U ʹja@sdZddlmZddlmZddlmZmZmZm Z ddl Z ddl m Z ddl mZmZmZmZddlmZmZmZmZmZmZmZmZmZmZmZddlmZmZd d l m!Z!d d l"m#Z#m$Z$d d l%m&Z&m'Z'm(Z(d gZ)Gddde j*Z+Gddde j*Z,Gddde j*Z-Gdd d e j*Z.dddZ/ddZ0dddZ1dddZ2dd d!Z3e&e2d"d#d$dd%e2d"d#d$dd%e2d&d"d$d#d'e2d(d"d$d#d'e2d)d"d$d#d'e2d*d"d$d#d'e2d+d"d$d#d'e2d,d"d#d$dd-e2d"d#d$dd%e2d.d"d#d$dd-e2d/d"d#d$dd-e2d0d"d#d$dd-e2d"d1d2dd3d4e2d"d1d2dd3d4e2d"d#d5d6e2d"d#d5d6e2d"d#d5d6e2d7d"d$dd'e2d8d"d$dd'e2d9d"d$dd'e2d:d"d$dd'e2d;d"d$dd'e2dd"d$dd'e2d?d"d$dd'e2d@d"d1d2dd3dAe2dBd"d1d2dd3dAe2dCd"d1d2dd3dAe2dDd"d1d2dd3dAe2dEd"d1d2dd3dAe2dFd"dGdHe2dId"dGdHe2dJd"dGdHe2dKd"dGdHe2dLd"dGdHe3dMd"d$dd'e3dNd"d1d2dd3dAe3dOd"d$dd'e3dPd"d1d2dd3dAe3dQd"d$dd'e3dRd"d1d2dd3dAe3dSd"d$dd'e3dTd"d1d2dd3dAe3dUd"d1d2dd3dAe3dVd"dWdXdd3dAe3dYd"d$d#d'e3dZd"d$d#d'e3d[d"d$d#d'e3d\d"d$dd'e3d]d"d$dd'e3d^d"d$dd'e3d_d"d$dd'e3d`d"d$dd'e3dad"ddHe3dbd"ddHe3dcd"ddHe3ddd"ddHe3ded"ddHe3dfd"ddHe3dgd"ddHe3dhd"ddHe2e2d"eedidjddke2d"eed1d2dd3dle2d"eedmdnddke2d"eed1d2dd3dle2d"eedidjddke2d"eed1d2ddke2d"eedidjddke2d"eed1d2dd3dle2d"eedidjddke2d"eed5didjddoe2d"eed5dmdnddoe2d"eed5d1d2dd3dpe2d"eed5d1d2dd3dpe2d"eed5didjddoe2dqdreedidjddsdte2dudreedidjddsdte2dvdreedidjddsdte2dwdreedmdnddsdte2dxdreedmdnddsdte2dydreedidjddzdte2d{dreedmdnddzdte2d|dreedmdnddzdte2d}dreedidjdd~dte2ddreedidjdd~dtdWZ4e'de.dddZ5e'de.dddZ6e'de.dddZ7e'de.dddZ8e'de.dddZ9e'de.dddZ:e'de.dddZ;e'de.dddZe'de.dddZ?e'de.dddZ@e'de.dddZAe'de.dddZBe'de.dddZCe'de.dddZDe'de.dddZEe'de.dddZFe'de.dddZGe'de.dddZHe'de.dddZIe'de.dddZJe'de.dddZKe'de.dddZLe'de.dddZMe(eNddddddddddddddddÜdS)ax ConvNeXt Papers: * `A ConvNet for the 2020s` - https://arxiv.org/pdf/2201.03545.pdf @Article{liu2022convnet, author = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie}, title = {A ConvNet for the 2020s}, journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2022}, } * `ConvNeXt-V2 - Co-designing and Scaling ConvNets with Masked Autoencoders` - https://arxiv.org/abs/2301.00808 @article{Woo2023ConvNeXtV2, title={ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders}, author={Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon and Saining Xie}, year={2023}, journal={arXiv preprint arXiv:2301.00808}, } Original code and weights from: * https://github.com/facebookresearch/ConvNeXt, original copyright below * https://github.com/facebookresearch/ConvNeXt-V2, original copyright below Model defs atto, femto, pico, nano and _ols / _hnf variants are timm originals. Modifications and additions for timm hacked together by / Copyright 2022, Ross Wightman ) OrderedDict)partial)CallableOptionalTupleUnionN)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDOPENAI_CLIP_MEANOPENAI_CLIP_STD) trunc_normal_ AvgPool2dSameDropPathMlpGlobalResponseNormMlp LayerNorm2d LayerNorm create_conv2d get_act_layermake_divisible to_ntuple)NormMlpClassifierHeadClassifierHead)build_model_with_cfg) named_applycheckpoint_seq)generate_default_cfgsregister_modelregister_model_deprecationsConvNeXtcs&eZdZdfdd ZddZZS) Downsamplercst|dkr|nd}|dks*|dkrX|dkr>|dkr>tntj}|d|ddd|_n t|_||kr~t||ddd|_n t|_dS)NrTF)Z ceil_modeZcount_include_pad)stride) super__init__r nnZ AvgPool2dpoolIdentityrconv)selfin_chsout_chsr#dilationZ avg_strideZ avg_pool_fn __class__S/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/models/convnext.pyr%<s  zDownsample.__init__cCs||}||}|SN)r'r)r*xr0r0r1forwardJs  zDownsample.forward)rr__name__ __module__ __qualname__r%r5 __classcell__r0r0r.r1r!:sr!csleZdZdZdeeeeeeeeeeffee e e eeee e fee ed fd d Z ddZ ZS) ConvNeXtBlockaa ConvNeXt Block There are two equivalent implementations: (1) DwConv -> LayerNorm (channels_first) -> 1x1 Conv -> GELU -> 1x1 Conv; all in (N, C, H, W) (2) DwConv -> Permute to (N, H, W, C); LayerNorm (channels_last) -> Linear -> GELU -> Linear; Permute back Unlike the official impl, this one allows choice of 1 or 2, 1x1 conv can be faster with appropriate choice of LayerNorm impl, however as model size increases the tradeoffs appear to change and nn.Linear is a better choice. This was observed with PyTorch 1.10 on 3090 GPU, it could change over time & w/ different HW. NrrrFTư>gelu) r+r, kernel_sizer#r- mlp_ratioconv_mlp conv_biasuse_grn ls_init_value act_layer norm_layer drop_pathc st|p|}td|}t| } | s6|r2tnt} t| r@tnt|d}||_ t |||||dd|d|_ | ||_ ||t ||| d|_| dk rt| t|nd|_||ks|dks|d|dkrt||||dd |_n t|_| d krt| nt|_dS) a[ Args: in_chs: Block input channels. out_chs: Block output channels (same as in_chs if None). kernel_size: Depthwise convolution kernel size. stride: Stride of depthwise convolution. dilation: Tuple specifying input and output dilation of block. mlp_ratio: MLP expansion ratio. conv_mlp: Use 1x1 convolutions for MLP and a NCHW compatible norm layer if True. conv_bias: Apply bias for all convolution (linear) layers. use_grn: Use GlobalResponseNorm in MLP (from ConvNeXt-V2) ls_init_value: Layer-scale init values, layer-scale applied if not None. act_layer: Activation layer. norm_layer: Normalization layer (defaults to LN if not specified). drop_path: Stochastic depth probability. r")Zuse_convrT)rBr#r-Z depthwisebias)rHNr)r#r-rA)r$r%rrrrrrr use_conv_mlprconv_dwnormintmlpr& ParametertorchZonesgammar!shortcutr(rrJ)r*r+r,rBr#r-rCrDrErFrGrHrIrJZ mlp_layerr.r0r1r%[s0!    "  zConvNeXtBlock.__init__cCs|}||}|jr*||}||}n4|dddd}||}||}|dddd}|jdk r||jdddd}||| |}|S)Nrr"r) rMrLrNrPZpermuterSmulreshaperJrT)r*r4rTr0r0r1r5s      zConvNeXtBlock.forward) Nr<rr=r>FTFr?r@NrA)r7r8r9__doc__rOrrrfloatboolstrrr%r5r:r0r0r.r1r;Ps:  ;r;c s&eZdZd fd d Zd d ZZS) ConvNeXtStager<r"r=N?FTr@cstd|_||ks0|dks0|d|dkr|dksH|d|dkrLdnd}|ddkr`dnd}t| |t|||||d|| d|_|}n t|_|pdg|}g}t|D]<}| t ||||d|||| | | | | r| n|d |}qtj||_ dS) NFrrr"Zsame)rBr#r-paddingrKrA) r+r,rBr-rJrGrDrErFrHrI) r$r%grad_checkpointingr& Sequentialr downsampler(rangeappendr;blocks)r*r+r,rBr#depthr-drop_path_ratesrGrDrErFrHrI norm_layer_clZds_kspadZ stage_blocksir.r0r1r%sJ        zConvNeXtStage.__init__cCs6||}|jr(tjs(t|j|}n ||}|Sr2)rbr`rRjitZ is_scriptingrrer3r0r0r1r5s   zConvNeXtStage.forward) r<r"r"r=Nr^FTFr@NNr6r0r0r.r1r]s:r]cseZdZdZd&eeeeeedfeedfeeeedffee eee e eee e e eee feeee fee e e dfdd Z e jjd'ddZe jjd(ddZe jjddZd)ddZdd Zd*e d!d"d#Zd$d%ZZS)+r zl ConvNeXt A PyTorch impl of : `A ConvNet for the 2020s` - https://arxiv.org/pdf/2201.03545.pdf rUavg rUrU rU`r<r?patchr>r^FNTr@rA.)in_chans num_classes global_pool output_stridedepthsdims kernel_sizesrG stem_type patch_sizehead_init_scalehead_norm_firsthead_hidden_sizerDrErFrHrInorm_eps drop_ratedrop_path_ratec"st|dksttd|}|dkr\t}|r6|nt}|dk rt||d}t||d}n$|shtd|}|dk rt||d}||_||_g|_ | dkst| dkrt t j ||d| | |d ||d|_ | }ndd | krt|dd n|d}t t j ||d d d |dt j ||dd d d |d||d|_ d}t |_ddtd|t||D}g}|d}|}d }tdD]}|d ks|dkrd nd }||kr|d kr||9}d }||9}|dkrd nd } ||}!|t||!|||| |f|||||||||||d|!}|j t||d|dg7_ qt j ||_||_| r| rht||j|_t|j|||jd|_n&t |_t|j|| ||j|dd|_ttt| d|dS)a Args: in_chans: Number of input image channels. num_classes: Number of classes for classification head. global_pool: Global pooling type. output_stride: Output stride of network, one of (8, 16, 32). depths: Number of blocks at each stage. dims: Feature dimension at each stage. kernel_sizes: Depthwise convolution kernel-sizes for each stage. ls_init_value: Init value for Layer Scale, disabled if None. stem_type: Type of stem. patch_size: Stem patch size for patch stem. head_init_scale: Init scaling value for classifier weights and biases. head_norm_first: Apply normalization before global pool + head. head_hidden_size: Size of MLP hidden layer in head if not None and head_norm_first == False. conv_mlp: Use 1x1 conv in MLP, improves speed for small networks w/ chan last. conv_bias: Use bias layers w/ all convolutions. use_grn: Use Global Response Norm (ConvNeXt-V2) in MLP. act_layer: Activation layer type. norm_layer: Normalization layer type. drop_rate: Head pre-classifier dropout rate. drop_path_rate: Stochastic depth drop rate. )rnr>N)epszcIf a norm_layer is specified, conv MLP must be used so all norm expect rank-4, channels-first input)rvoverlapoverlap_tieredrvr)rBr#rKZtieredr"rUr)rBr#r_rKcSsg|] }|qSr0)tolist).0r4r0r0r1 Gsz%ConvNeXt.__init__..)rr") rBr#r-rfrgrGrDrErFrHrIrhzstages.)Znum_chsZ reductionmodule) pool_typerr@)Z hidden_sizerrrIrH)r) r$r%AssertionErrorrrrrrxrZ feature_infor&raConv2dstemrstagesrRZlinspacesumsplitrcrdr]dictZ num_featuresnorm_prerheadr(rr _init_weights)"r*rwrxryrzr{r|r}rGr~rrrrrDrErFrHrIrrrrhZ stem_strideZmid_chsZdp_ratesrZprev_chsZ curr_strider-rjr#Zfirst_dilationr,r.r0r1r%s/           " $      zConvNeXt.__init__cCstd|r dndddgdS)Nz^stemz^stages\.(\d+))z^stages\.(\d+)\.downsample)r)z^stages\.(\d+)\.blocks\.(\d+)N)z ^norm_pre)i)rre)r)r*Zcoarser0r0r1 group_matcherszConvNeXt.group_matchercCs|jD] }||_qdSr2)rr`)r*enablesr0r0r1set_grad_checkpointings zConvNeXt.set_grad_checkpointingcCs|jjSr2)rfc)r*r0r0r1get_classifierszConvNeXt.get_classifierrcCs|j||dSr2)rreset)r*rxryr0r0r1reset_classifierszConvNeXt.reset_classifiercCs"||}||}||}|Sr2)rrrr3r0r0r1forward_featuress   zConvNeXt.forward_features pre_logitscCs|r|j|ddS||S)NTr)r)r*r4rr0r0r1 forward_headszConvNeXt.forward_headcCs||}||}|Sr2)rrr3r0r0r1r5s  zConvNeXt.forward)rUrlrmrnrorqr<r?rvr>r^FNFTFr@NNrArA)F)T)rN)F)r7r8r9rYrOr\rrrrZr[rr%rRrkignorerrrrrrr5r:r0r0r.r1r sp       r^cCst|tjr4t|jdd|jdk rtj|jnPt|tjrt|jddtj|j|rd|kr|jj ||jj |dS)Ng{Gz?)stdhead.) isinstancer&rr weightrKinitZzeros_ZLineardataZmul_)rnamerr0r0r1rs    rcCsd|ksd|kr|Sd|kr$|d}i}d|krdd|D}d|krp|d|d<t|djd |d <nDd |kr|d |d <|d |d<|d|d<t|djd |d <|Sd dl}|D]\}}|dd}|dd|}|dd|}|dd}|dd}d|krB|dd}|dd}||jd }|d!d"}|d#rf|d$d%}|j d&krd'|kr| |j}||}|||<q|S)(z Remap FB checkpoints -> timm zhead.norm.weightznorm_pre.weightmodelzvisual.trunk.stem.0.weightcSs(i|] \}}|dr|dd|qS)z visual.trunk.) startswithreplace)rkvr0r0r1 s z(checkpoint_filter_fn..zvisual.head.proj.weightzhead.fc.weightrz head.fc.biaszvisual.head.mlp.fc1.weightzhead.pre_logits.fc.weightzvisual.head.mlp.fc1.biaszhead.pre_logits.fc.biaszvisual.head.mlp.fc2.weightNzdownsample_layers.0.zstem.zstages.([0-9]+).([0-9]+)zstages.\1.blocks.\2z#downsample_layers.([0-9]+).([0-9]+)zstages.\1.downsample.\2ZdwconvrMZpwconvzmlp.fcZgrnzgrn.betaz mlp.grn.biasz grn.gammazmlp.grn.weightrVrzhead.fc.znorm.rNz head.normr"r) itemsrRZzerosshaperersubrXrndim state_dict)rrZout_dictrrrZ model_shaper0r0r1checkpoint_filter_fnsF               rFcKsB|dddkr|ddtt||fttdddd |}|S) NZpretrained_cfgrZfcmaeZpretrained_strictF)rrr"rUT)Z out_indicesZflatten_sequential)Zpretrained_filter_fnZ feature_cfg)get setdefaultrr rr)variant pretrainedkwargsrr0r0r1_create_convnexts  rrc Ks|dddddttddd |S) NrlrUrr<r<?bicubicstem.0head.fc) urlrx input_size pool_sizecrop_pct interpolationmeanr first_conv classifierrr rrr0r0r1_cfgsrcKs&|dddddttdddd d d d |S) Nrlrrrrrrz cc-by-nc-4.0zarXiv:2301.00808zGConvNeXt-V2: Co-designing and Scaling ConvNets with Masked Autoencodersz/https://github.com/facebookresearch/ConvNeXt-V2)rrxrrrrrrrrlicenseZ paper_idsZ paper_nameZ origin_urlrrr0r0r1_cfgv2s" rztimm/gffffff?)rU r) hf_hub_idrtest_input_size test_crop_pctzrhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_atto_d2-01bb0f51.pth)rrrrzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_atto_ols_a2-78d1c8f3.pthzshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_femto_d1-d71d5b4c.pthzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_femto_ols_d1-246bf2ed.pthzrhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_pico_d1-10ad7f0d.pthzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_pico_ols_d1-611f0ca7.pth)rrrrrzshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_nano_d1h-7eb4bdea.pthzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_nano_ols_d1h-ae424a9a.pthzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/convnext_tiny_hnf_a2h-ab7e9df2.pth)rUrtrt) rZsquash)rrrr crop_modei-.)rrrxzDhttps://dl.fbaipublicfiles.com/convnext/convnext_tiny_22k_1k_224.pthzEhttps://dl.fbaipublicfiles.com/convnext/convnext_small_22k_1k_224.pthzDhttps://dl.fbaipublicfiles.com/convnext/convnext_base_22k_1k_224.pthzEhttps://dl.fbaipublicfiles.com/convnext/convnext_large_22k_1k_224.pthzJhttps://dl.fbaipublicfiles.com/convnext/convnext_xlarge_22k_1k_224_ema.pthzDhttps://dl.fbaipublicfiles.com/convnext/convnext_tiny_1k_224_ema.pthzEhttps://dl.fbaipublicfiles.com/convnext/convnext_small_1k_224_ema.pthzDhttps://dl.fbaipublicfiles.com/convnext/convnext_base_1k_224_ema.pthzEhttps://dl.fbaipublicfiles.com/convnext/convnext_large_1k_224_ema.pthzDhttps://dl.fbaipublicfiles.com/convnext/convnext_tiny_22k_1k_384.pth)rrrrrrzEhttps://dl.fbaipublicfiles.com/convnext/convnext_small_22k_1k_384.pthzDhttps://dl.fbaipublicfiles.com/convnext/convnext_base_22k_1k_384.pthzEhttps://dl.fbaipublicfiles.com/convnext/convnext_large_22k_1k_384.pthzJhttps://dl.fbaipublicfiles.com/convnext/convnext_xlarge_22k_1k_384_ema.pthzAhttps://dl.fbaipublicfiles.com/convnext/convnext_tiny_22k_224.pthiQU)rrrxzBhttps://dl.fbaipublicfiles.com/convnext/convnext_small_22k_224.pthzAhttps://dl.fbaipublicfiles.com/convnext/convnext_base_22k_224.pthzBhttps://dl.fbaipublicfiles.com/convnext/convnext_large_22k_224.pthzChttps://dl.fbaipublicfiles.com/convnext/convnext_xlarge_22k_224.pthzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_nano_22k_224_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_nano_22k_384_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_tiny_22k_224_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_tiny_22k_384_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_base_22k_224_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_base_22k_384_ema.ptzXhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_large_22k_224_ema.ptzXhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_large_22k_384_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_huge_22k_384_ema.ptzWhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im22k/convnextv2_huge_22k_512_ema.pt)rUr)rzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_atto_1k_224_ema.ptzVhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_femto_1k_224_ema.ptzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_pico_1k_224_ema.ptzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_nano_1k_224_ema.ptzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_tiny_1k_224_ema.ptzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_base_1k_224_ema.ptzVhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_large_1k_224_ema.ptzUhttps://dl.fbaipublicfiles.com/convnext/convnextv2/im1k/convnextv2_huge_1k_224_ema.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_atto_1k_224_fcmae.ptz[https://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_femto_1k_224_fcmae.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_pico_1k_224_fcmae.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_nano_1k_224_fcmae.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_tiny_1k_224_fcmae.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_base_1k_224_fcmae.ptz[https://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_large_1k_224_fcmae.ptzZhttps://dl.fbaipublicfiles.com/convnext/convnextv2/pt_only/convnextv2_huge_1k_224_fcmae.pt)rUr)rr)rrrrrr)rrrrrrr)rU@r) r)rrrrxrrr)rrrrxrrrrz,laion/CLIP-convnext_base_w-laion2B-s13B-b82Kzopen_clip_pytorch_model.bin)rZhf_hub_filenamerrrrrrxz3laion/CLIP-convnext_base_w-laion2B-s13B-b82K-augregz4laion/CLIP-convnext_base_w-laion_aesthetic-s13B-b82Kz8laion/CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82Kz?laion/CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K-augregz5laion/CLIP-convnext_large_d.laion2B-s26B-b102K-augregruz5laion/CLIP-convnext_large_d_320.laion2B-s29B-b131K-ftz:laion/CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soupz9laion/CLIP-convnext_xxlarge-laion2B-s34B-b82K-augreg-soupz;laion/CLIP-convnext_xxlarge-laion2B-s34B-b82K-augreg-rewind)Wzconvnext_tiny.in12k_ft_in1kzconvnext_small.in12k_ft_in1kzconvnext_atto.d2_in1kzconvnext_atto_ols.a2_in1kzconvnext_femto.d1_in1kzconvnext_femto_ols.d1_in1kzconvnext_pico.d1_in1kzconvnext_pico_ols.d1_in1kzconvnext_nano.in12k_ft_in1kzconvnext_nano.d1h_in1kzconvnext_nano_ols.d1h_in1kzconvnext_tiny_hnf.a2h_in1kzconvnext_tiny.in12k_ft_in1k_384z convnext_small.in12k_ft_in1k_384zconvnext_nano.in12kzconvnext_tiny.in12kzconvnext_small.in12kconvnext_tiny.fb_in22k_ft_in1kconvnext_small.fb_in22k_ft_in1kconvnext_base.fb_in22k_ft_in1kconvnext_large.fb_in22k_ft_in1k convnext_xlarge.fb_in22k_ft_in1kzconvnext_tiny.fb_in1kzconvnext_small.fb_in1kzconvnext_base.fb_in1kzconvnext_large.fb_in1k"convnext_tiny.fb_in22k_ft_in1k_384#convnext_small.fb_in22k_ft_in1k_384"convnext_base.fb_in22k_ft_in1k_384#convnext_large.fb_in22k_ft_in1k_384$convnext_xlarge.fb_in22k_ft_in1k_384convnext_tiny.fb_in22kconvnext_small.fb_in22kconvnext_base.fb_in22kconvnext_large.fb_in22kconvnext_xlarge.fb_in22kz#convnextv2_nano.fcmae_ft_in22k_in1kz'convnextv2_nano.fcmae_ft_in22k_in1k_384z#convnextv2_tiny.fcmae_ft_in22k_in1kz'convnextv2_tiny.fcmae_ft_in22k_in1k_384z#convnextv2_base.fcmae_ft_in22k_in1kz'convnextv2_base.fcmae_ft_in22k_in1k_384z$convnextv2_large.fcmae_ft_in22k_in1kz(convnextv2_large.fcmae_ft_in22k_in1k_384z'convnextv2_huge.fcmae_ft_in22k_in1k_384z'convnextv2_huge.fcmae_ft_in22k_in1k_512zconvnextv2_atto.fcmae_ft_in1kzconvnextv2_femto.fcmae_ft_in1kzconvnextv2_pico.fcmae_ft_in1kzconvnextv2_nano.fcmae_ft_in1kzconvnextv2_tiny.fcmae_ft_in1kzconvnextv2_base.fcmae_ft_in1kzconvnextv2_large.fcmae_ft_in1kzconvnextv2_huge.fcmae_ft_in1kzconvnextv2_atto.fcmaezconvnextv2_femto.fcmaezconvnextv2_pico.fcmaezconvnextv2_nano.fcmaezconvnextv2_tiny.fcmaezconvnextv2_base.fcmaezconvnextv2_large.fcmaezconvnextv2_huge.fcmaezconvnextv2_small.untrainedz/convnext_base.clip_laion2b_augreg_ft_in12k_in1kz3convnext_base.clip_laion2b_augreg_ft_in12k_in1k_384z6convnext_large_mlp.clip_laion2b_soup_ft_in12k_in1k_320z6convnext_large_mlp.clip_laion2b_soup_ft_in12k_in1k_384z)convnext_base.clip_laion2b_augreg_ft_in1kz,convnext_base.clip_laiona_augreg_ft_in1k_384z.convnext_large_mlp.clip_laion2b_augreg_ft_in1kz2convnext_large_mlp.clip_laion2b_augreg_ft_in1k_384z*convnext_xxlarge.clip_laion2b_soup_ft_in1kz*convnext_base.clip_laion2b_augreg_ft_in12kz1convnext_large_mlp.clip_laion2b_soup_ft_in12k_320z3convnext_large_mlp.clip_laion2b_augreg_ft_in12k_384z1convnext_large_mlp.clip_laion2b_soup_ft_in12k_384z+convnext_xxlarge.clip_laion2b_soup_ft_in12kzconvnext_base.clip_laion2bz!convnext_base.clip_laion2b_augregzconvnext_base.clip_laionazconvnext_base.clip_laiona_320z$convnext_base.clip_laiona_augreg_320z&convnext_large_mlp.clip_laion2b_augregz&convnext_large_mlp.clip_laion2b_ft_320z+convnext_large_mlp.clip_laion2b_ft_soup_320z"convnext_xxlarge.clip_laion2b_soupz$convnext_xxlarge.clip_laion2b_rewind)returncKs,tdddd}tdd|it|f|}|S)Nr"r"r"(PrTr{r|rD convnext_attor)rrrrrZ model_argsrr0r0r1rzsrcKs.tddddd}tdd|it|f|}|S) NrrTrr{r|rDr~convnext_atto_olsr)rrrr0r0r1rsrcKs,tdddd}tdd|it|f|}|S)Nr0rrrsrtTrconvnext_femtor)rrrr0r0r1rsrcKs.tddddd}tdd|it|f|}|S) NrrTrrconvnext_femto_olsr)rrrr0r0r1rsrcKs,tdddd}tdd|it|f|}|S)Nr@rrTr convnext_picor)rrrr0r0r1rsrcKs.tddddd}tdd|it|f|}|S) NrrTrrconvnext_pico_olsr)rrrr0r0r1rsrcKs,tdddd}tdd|it|f|}|S)Nr"r"rr"rrrrTr convnext_nanor)r rrr0r0r1r sr cKs.tddddd}tdd|it|f|}|S) Nrr Trrconvnext_nano_olsr)r rrr0r0r1r sr cKs.tddddd}tdd|it|f|}|S)NrorqT)r{r|rrDconvnext_tiny_hnfr)r rrr0r0r1r sr cKs*tddd}tdd|it|f|}|S)Nrorqr{r| convnext_tinyr)rrrr0r0r1rs rcKs:tddddgddddgd}td d |it|f|}|S) NrUrrrsrtrur convnext_smallr)rrrr0r0r1rsrcKs:tddddgddddgd}td d |it|f|}|S) NrUrrrrrr  convnext_baser)rrrr0r0r1rsrcKs:tddddgddddgd}td d |it|f|}|S) NrUrrsrtrur convnext_larger)rrrr0r0r1rsrcKs<tddddgddddgdd}td d |it|f|}|S) NrUrrsrtrur)r{r|rconvnext_large_mlpr)rrrr0r0r1rsrcKs:tddddgddddgd}td d |it|f|}|S) NrUrrrrir convnext_xlarger)rrrr0r0r1rsrcKsDtddddgddddg|dd d }td d |it|f|}|S)NrUr>rtruri rgh㈵>)r{r|rconvnext_xxlarger)r)rpoprrr0r0r1rs&rcKs0tdddddd}tdd|it|f|}|S)NrrTr{r|rFrGrDconvnextv2_attor)rrrr0r0r1rsrcKs0tdddddd}tdd|it|f|}|S)NrrTrconvnextv2_femtor)rrrr0r0r1rsrcKs0tdddddd}tdd|it|f|}|S)NrrTrconvnextv2_picor)rrrr0r0r1rsrcKs0tdddddd}tdd|it|f|}|S)Nrr Trconvnextv2_nanor)rrrr0r0r1rsrcKs.tddddd}tdd|it|f|}|S)NrorqTr{r|rFrGconvnextv2_tinyr)rrrr0r0r1rsrcKs>tddddgddddgddd}td d |it|f|}|S) NrUrrrrsrtruTrconvnextv2_smallr)r rrr0r0r1r s r cKs>tddddgddddgddd}td d |it|f|}|S) NrUrrrrrTrconvnextv2_baser)r!rrr0r0r1r!%s r!cKs>tddddgddddgddd}td d |it|f|}|S) NrUrrsrtrurTrconvnextv2_larger)r"rrr0r0r1r",s r"cKs>tddddgddddgddd}td d |it|f|}|S) NrUri`iii Trconvnextv2_huger)r#rrr0r0r1r#3s r#rrrrrrrrrrrrrrr)Zconvnext_tiny_in22ft1kZconvnext_small_in22ft1kZconvnext_base_in22ft1kZconvnext_large_in22ft1kZconvnext_xlarge_in22ft1kZconvnext_tiny_384_in22ft1kZconvnext_small_384_in22ft1kZconvnext_base_384_in22ft1kZconvnext_large_384_in22ft1kZconvnext_xlarge_384_in22ft1kZconvnext_tiny_in22kZconvnext_small_in22kZconvnext_base_in22kZconvnext_large_in22kZconvnext_xlarge_in22k)Nr^)F)r)r)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)OrY collectionsr functoolsrtypingrrrrrRZtorch.nnr&Z timm.datarr r r Z timm.layersr r rrrrrrrrrrrZ_builderrZ _manipulaterr _registryrrr__all__Moduler!r;r]r rrrrrZ default_cfgsrrrrrrr r r rrrrrrrrrrrrr r!r"r#r7r0r0r0r1s|'   4 XE= *  u