U ʹj@sddlZddlmZddlmZmZmZddlZddlm Z ddl m Z m Z ddl mZmZmZmZmZmZmZddlmZddlmZdd lmZmZd d ZGd d d e jZGddde jZe j dfe!e!e je"e j#dddZ$Gddde jZ%Gddde jZ&Gddde jZ'Gddde jZ(Gddde jZ)Gddde jZ*Gd d!d!e jZ+Gd"d#d#e jZ,Gd$d%d%e jZ-Gd&d'd'e jZ.d@d)d*Z/ee/d+d,e/d+d,e/d+d,e/d+d,e/d+d,e/d+d,e/d+d-d.e/d+d,e/d+d,e/d+d,e/d+d,e/d+d,e/d+d,e/d+d-d.d/Z0dAd0d1Z1edBd2d3Z2edCd4d5Z3edDd6d7Z4edEd8d9Z5edFd:d;Z6edGdd?Z8dS)IN)partial)TupleOptionalUnionIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)DropPath trunc_normal_ create_conv2d ConvNormAct SqueezeExciteuse_fused_attnClassifierHead)build_model_with_cfg)checkpoint_seq)register_modelgenerate_default_cfgscCs$|sdS||dkst||SdS)Nrr)AssertionError) group_sizeZchannelsrR/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/models/fastvit.py num_groupssrcseZdZdZddddddddejf eeeeeeeeeeeejdd fdd Z e j e j d d d Z d dZ ee j e j fdddZeejejfee j e j fdddZZS)MobileOneBlocka#MobileOne building block. This block has a multi-branched architecture at train-time and plain-CNN style architecture at inference time For more details, please refer to our paper: `An Improved One millisecond Mobile Backbone` - https://arxiv.org/pdf/2206.04040.pdf rrFTN) in_chsout_chs kernel_sizestridedilationrinference_modeuse_seuse_actuse_scale_branchnum_conv_branches act_layerreturnc s&tt|_t||_|_|__|_ |_ | _ |rTt |ddnt _|r~t||||jdd_nd_||kr|dkrt j|dnd_| dkrt fdd tj D_nd_d_dkr | r tj j djjd d _| r| nt _dS) aConstruct a MobileOneBlock module. Args: in_chs: Number of channels in the input. out_chs: Number of channels produced by the block. kernel_size: Size of the convolution kernel. stride: Stride size. dilation: Kernel dilation factor. group_size: Convolution group size. inference_mode: If True, instantiates model in inference mode. use_se: Whether to use SE-ReLU activations. use_act: Whether to use activation. Default: ``True`` use_scale_branch: Whether to use scale branch. Default: ``True`` num_conv_branches: Number of linear conv branches. r)Z rd_divisorTrrrgroupsbiasN) num_featuresrc s(g|] }tjjjjddqS)Frrr( apply_act)r rrrr().0_rselfrr ksz+MobileOneBlock.__init__..Fr+)superr__init__r rr(rrrrrr$r nnIdentityser reparam_conv BatchNorm2didentityZ ModuleListrangeconv_kxk conv_scaler act) r0rrrrrrr r!r"r#r$r% __class__r/rr3*sV    zMobileOneBlock.__init__xr&cCs|jdk r ||||Sd}|jdk r8||}d}|jdk rP||}||}|jdk rz|jD]}|||7}qh|||S)zApply forward pass.Nr)r7r=r6r9r<r;)r0rAZ identity_outZ scale_outoutrcrrrforwards       zMobileOneBlock.forwardc Cs|jdk rdS|\}}t|j|j|j|j|j|jdd|_||jj _ ||jj _ | D]\}}d|krlqZ| qZ|d|dt|dr|dd|_dS)aFollowing works like `RepVGG: Making VGG-style ConvNets Great Again` - https://arxiv.org/pdf/2101.03697.pdf. We re-parameterize multi-branched architecture used at training time to obtain a plain CNN-like structure for inference. NT)Z in_channelsZ out_channelsrrrr(r)r7r;r<r9)r7_get_kernel_biasr rrrrrr(weightdatar)named_parametersdetach_ __delattr__hasattrr )r0kernelr)namepararrrreparameterizes.         zMobileOneBlock.reparameterizer&c Csd}d}|jdk rD||j\}}|jd}tjj|||||g}d}d}|jdk rf||j\}}d}d}|jdk rt |j D](}||j|\} } || 7}|| 7}q|||} |||} | | fS)zMethod to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83 Returns: Tuple of (kernel, bias) after fusing branches. rN) r<_fuse_bn_tensorrtorchr4 functionalpadr9r;r:r$) r0Z kernel_scaleZ bias_scalerUZkernel_identityZ bias_identityZ kernel_convZ bias_convZixZ_kernelZ_biasZ kernel_finalZ bias_finalrrrrEs(       zMobileOneBlock._get_kernel_bias)branchr&c Cst|tr<|jj}|jj}|jj}|jj}|jj}|jj}nt|t j sLt t |ds|j |j}tj|j ||j|jf|jj|jjd} t|j D]$} d| | | ||jd|jdf<q| |_|j}|j}|j}|j}|j}|j}||} || dddd} || |||| fS)a Method to fuse batchnorm layer with preceeding conv layer. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L95 Args: branch: Sequence of ops to be fused. Returns: Tuple of (kernel, bias) after fusing batchnorm. id_tensordtypedevicerrQ) isinstancer convrFbn running_mean running_varr)epsr4r8rrKrr(rSzerosrrYrZr:rWsqrtreshape) r0rVrLr_r`gammabetara input_dim kernel_valueistdtrrrrRs<     zMobileOneBlock._fuse_bn_tensor)__name__ __module__ __qualname____doc__r4GELUintboolModuler3rSTensorrDrOrrEr Sequentialr8rR __classcell__rrr>rr s>\#$rc seZdZdZdeeeeeeeeeejdd fdd Z e j e j ddd Z e e j e j fd d d Zdd d dZee j eje e j e j fdddZZS)ReparamLargeKernelConvzBuilding Block of RepLKNet This class defines overparameterized large kernel conv block introduced in `RepLKNet `_ Reference: https://github.com/DingXiaoH/RepLKNet-pytorch NF) rrrrr small_kernelr r%r&c stt|||_t|||_||_||_||_||_ |rXt ||||d|jdd|_ nRd|_ t ||||j|jdd|_ |dk r||kstdt ||||j|jdd|_|dk r|nt|_dS)a!Construct a ReparamLargeKernelConv module. Args: in_chs: Number of input channels. out_chs: Number of output channels. kernel_size: Kernel size of the large kernel conv branch. stride: Stride size. Default: 1 group_size: Group size. Default: 1 small_kernel: Kernel size of small kernel conv branch. inference_mode: If True, instantiates model in inference mode. Default: ``False`` act_layer: Activation module. Default: ``nn.GELU`` rTr'NFr+zDThe kernel size for re-param cannot be larger than the large kernel!)r2rwr3rrr(rrrrxr r7r large_convr small_convr4r5r=) r0rrrrrrxr r%r>rrr3sP   zReparamLargeKernelConv.__init__r@cCsF|jdk r||}n"||}|jdk r8|||}||}|SN)r7ryrzr=)r0rArBrrrrD\s     zReparamLargeKernelConv.forwardrPcCsn||jj|jj\}}t|drf||jj|jj\}}||7}|tj||j |j dgd7}||fS)zMethod to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepLKNet-pytorch Returns: Tuple of (kernel, bias) after fusing branches. rzrQ) _fuse_bnryr]r^rKrzr4rTrUrrx)r0eq_keq_bZsmall_kZsmall_brrrget_kernel_biasfs z&ReparamLargeKernelConv.get_kernel_biascCsb|\}}t|j|j|j|j|jdd|_||jj_ ||jj _ | dt |dr^| ddS)a Following works like `RepVGG: Making VGG-style ConvNets Great Again` - https://arxiv.org/pdf/2101.03697.pdf. We re-parameterize multi-branched architecture used at training time to obtain a plain CNN-like structure for inference. Trrr(r)ryrzN) rr rrrrr(r7rFrGr)rJrK)r0r~rrrrrOvs     z%ReparamLargeKernelConv.reparameterize)r]r^r&c Cs\|j}|j}|j}|j}|j}|j}||}||dddd} || ||||fS)zMethod to fuse batchnorm layer with conv layer. Args: conv: Convolutional kernel weights. bn: Batchnorm 2d layer. Returns: Tuple of (kernel, bias) after fusing batchnorm. r[r)rFr_r`r)rarcrd) r]r^rLr_r`rerfrarjrkrrrr}s  zReparamLargeKernelConv._fuse_bn)NFN)rlrmrnrorqrrrr4rsr3rSrtrDrrrO staticmethodr8r}rvrrr>rrws.B rwF)rrr%r r&c Cs@tt||dd||dt||ddd||dt||dd||dS)a,Build convolutional stem with MobileOne blocks. Args: in_chs: Number of input channels. out_chs: Number of output channels. inference_mode: Flag to instantiate model in inference mode. Default: ``False`` Returns: nn.Sequential object with stem elements. rQrrrrr%r r)rrrrrr%r )r4rur)rrr%r rrrconvolutional_stems6 rcsXeZdZUdZejjeed<d e e ee e ddfdd Z ej ej d d d Z ZS) AttentionzMulti-headed Self Attention module. Source modified from: https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py fused_attn FN)dimhead_dimqkv_bias attn_drop proj_dropr&cst||dkstd||_|||_|d|_t|_tj ||d|d|_ t ||_ t |||_ t ||_dS)a}Build MHSA module that can handle 3D or 4D input tensors. Args: dim: Number of embedding dimensions. head_dim: Number of hidden dimensions per head. Default: ``32`` qkv_bias: Use bias or not. Default: ``False`` attn_drop: Dropout rate for attention tensor. proj_drop: Dropout rate for projection tensor. rz#dim should be divisible by head_dimgr)r)N)r2r3rr num_headsscalerrr4LinearqkvDropoutrprojr)r0rrrrrr>rrr3s    zAttention.__init__r@c Cs |j\}}}}||}|ddd}||||d|j|jddddd}|d\}} } |j rt j j j || | |jr|jjndd }n8||j}|| dd} | jdd } || } | | }|dd|||}||}||}|dd||||}|S) NrQr[rrrr|r)Z dropout_pr)shapeflattenZ transposerrdrrZpermuteZunbindrrSr4rTZscaled_dot_product_attentionZtrainingrprZsoftmaxrr) r0rABCHWNrqkvZattnrrrrDsF      zAttention.forward)rFrr)rlrmrnrorSjitFinalrr__annotations__rqfloatr3rtrDrvrrr>rrs rc sTeZdZdZejddfeeeeejeeddfdd Z e j e j ddd Z Z S) PatchEmbedz$Convolutional patch embedding layer.FN) patch_sizerr embed_dimr% lkc_use_actr r&c sHttt||||dd||r&|nddt||dd||d|_dS)a{Build patch embedding layer. Args: patch_size: Patch size for embedding computation. stride: Stride for convolutional embedding layer. in_chs: Number of channels of input tensor. embed_dim: Number of embedding dimensions. inference_mode: Flag to instantiate model in inference mode. Default: ``False`` rrN)rrrrrrxr r%r)r2r3r4rurwrr)r0rrrrr%rr r>rrr3s*   zPatchEmbed.__init__r@cCs||}|Sr{)rr0rArrrrDBs zPatchEmbed.forward)rlrmrnror4rprqrsrrr3rSrtrDrvrrr>rrs)rcs&eZdZdfdd ZddZZS) LayerScale2dh㈵>Fcs.t||_t|t|dd|_dS)Nr)r2r3inplacer4 ParameterrSZonesre)r0rZ init_valuesrr>rrr3Hs zLayerScale2d.__init__cCs|jr||jS||jSr{)rZmul_rerrrrrDMszLayerScale2d.forward)rF)rlrmrnr3rDrvrrr>rrGsrcsJeZdZdZdedfdd Zejejdd d Zd d d dZ Z S)RepMixerzReparameterizable token mixer. For more details, please refer to our paper: `FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization `_ rrFr c st||_||_||_|rJtj|j|j|jd|jd|jdd|_nPd|_t|||ddddd|_ t|||ddd |_ |dk rt |||_ ntj |_ dS) aBuild RepMixer Module. Args: dim: Input feature map dimension. :math:`C_{in}` from an expected input of size :math:`(B, C_{in}, H, W)`. kernel_size: Kernel size for spatial mixing. Default: 3 layer_scale_init_value: Initial value for layer scale. Default: 1e-5 inference_mode: If True, instantiates model in inference mode. Default: ``False`` rrQTrrpaddingr(r)NFr)rr"r#r$)rr")r2r3rrr r4Conv2dr7rnormmixerr layer_scaler5)r0rrlayer_scale_init_valuer r>rrr3XsD   zRepMixer.__init__r@cCs8|jdk r||}n||||||}|Sr{)r7rrrrrrrrDs  zRepMixer.forwardNrPcCs |jr dS|j|jt|jtrx|jj|jj d|jj j |jj j }t |jj|jj j|jj j}n0|jj|jj j |jj j }|jj j|jj j}t|j|j|jd|jdd|_ ||j j _||j j_|D]\}}d|krq|q|d|d|d dS) ziReparameterize mixer and norm into a single convolutional layer for efficient inference. Nr[rTrr7rrr)r rrOrr\rrrWreZ unsqueezer7rFrSZsqueezer)r rrrGrHrIrJ)r0wbrMrNrrrrOsF         zRepMixer.reparameterize)rrF) rlrmrnrorrr3rSrtrDrOrvrrr>rrQs 5rcsleZdZdZddejdfeeeeeeje ddfdd Z ejdddd Z e j e j d d d ZZS) ConvMlpzConvolutional FFN Module.Nr)rhidden_channelsrr%dropr&csvt|p|}|p|}t||d|dd|_tj||dd|_||_tj||dd|_t ||_ | |j dS)a_Build convolutional FFN module. Args: in_chs: Number of input channels. hidden_channels: Number of channels after expansion. Default: None out_chs: Number of output channels. Default: None act_layer: Activation layer. Default: ``GELU`` drop: Dropout rate. Default: ``0.0``. F)rr(r,r)rN) r2r3r r]r4rfc1r=fc2rrapply _init_weights)r0rrrr%rr>rrr3s  zConvMlp.__init__mr&cCs8t|tjr4t|jdd|jdk r4tj|jddS)N{Gz?rjr)r\r4rr rFr)init constant_r0rrrrrs  zConvMlp._init_weightsr@cCs@||}||}||}||}||}||}|Sr{)r]rr=rrrrrrrDs      zConvMlp.forward)rlrmrnror4rprqrrsrr3rrSrtrDrvrrr>rrs!rcsdeZdZdZdeeeeeeeeffddfdd Ze j e j dd d Z dd d d Z Z S)RepConditionalPosEnca"Implementation of conditional positional encoding. For more details refer to paper: `Conditional Positional Encodings for Vision Transformers `_ In our implementation, we can reparameterize this module to eliminate a skip connection. NrrF)rdim_out spatial_shaper&c stt|t|tr&t|gd}t|tsDtdt|dt |dksdtdt |d||_ ||_ |pv||_ ||_ |rtj|j |j |j d|dd|j dd|_n2d |_tj|j |j |dt|dd|j dd |_d S) atBuild reparameterizable conditional positional encoding Args: dim: Number of input channels. dim_out: Number of embedding dimensions. Default: 768 spatial_shape: Spatial shape of kernel for positional encoding. Default: (7, 7) inference_mode: Flag to instantiate block in inference mode. Default: ``False`` rQz/"spatial_shape" must by a sequence or int, get z instead.z+Length of "spatial_shape" should be 2, got rrTrN)r(r))r2rr3r\rqtuplerrtypelenrrrr(r4rr7pos_enc)r0rrrr r>rrr3sB     zRepConditionalPosEnc.__init__r@cCs(|jdk r||}n|||}|Sr{)r7rrrrrrD4s  zRepConditionalPosEnc.forwardrPc Cs|j|j}tj|j||jd|jdf|jjj|jjjd}t |jD],}d|||||jdd|jddf<qH|}||jj}|jj }t j |j|j |jdt|jdd|jdd|_||jj_||jj _|D]\}}d|krq|q|ddS) NrrrXrQTrr7r)rr(rSrbrrrFrYrZr:r)r4rrrqr7rGrHrIrJ) r0rgrhrirWZw_finalZb_finalrMrNrrrrO;sL        z#RepConditionalPosEnc.reparameterize)NrF)rlrmrnrorqrrrr3rSrtrDrOrvrrr>rrs 6rc sPeZdZdZddejddddfeeeejeeee dfdd Z d d Z Z S) RepMixerBlockzImplementation of Metaformer block with RepMixer as token mixer. For more details on Metaformer structure, please refer to: `MetaFormer Is Actually What You Need for Vision `_ r@rrF)rr mlp_ratior%r drop_pathrr c sttt||||d|_t|t||||d|_|dk rLt|||_n t |_|dkrft |nt |_ dS)a,Build RepMixer Block. Args: dim: Number of embedding dimensions. kernel_size: Kernel size for repmixer. Default: 3 mlp_ratio: MLP expansion ratio. Default: 4.0 act_layer: Activation layer. Default: ``nn.GELU`` proj_drop: Dropout rate. Default: 0.0 drop_path: Drop path rate. Default: 0.0 layer_scale_init_value: Layer scale value at initialization. Default: 1e-5 inference_mode: Flag to instantiate block in inference mode. Default: ``False`` )rrr rrr%rNr) r2r3r token_mixerrrqmlprrr4r5r r) r0rrrr%rrrr r>rrr3ps"   zRepMixerBlock.__init__cCs(||}|||||}|Sr{)rrrrrrrrrDs zRepMixerBlock.forward) rlrmrnror4rprqrrsrrr3rDrvrrr>rris& -rc sPeZdZdZdejejdddfeeej ej eeedfdd Z dd Z Z S) AttentionBlockzImplementation of metaformer block with MHSA as token mixer. For more details on Metaformer structure, please refer to: `MetaFormer Is Actually What You Need for Vision `_ rrr)rrr% norm_layerrrrcst|||_t|d|_|dk r6t|||_n t|_|dkrPt |nt|_ t |t ||||d|_ |dk rt|||_n t|_|dkrt |nt|_dS)aBuild Attention Block. Args: dim: Number of embedding dimensions. mlp_ratio: MLP expansion ratio. Default: 4.0 act_layer: Activation layer. Default: ``nn.GELU`` norm_layer: Normalization layer. Default: ``nn.BatchNorm2d`` proj_drop: Dropout rate. Default: 0.0 drop_path: Drop path rate. Default: 0.0 layer_scale_init_value: Layer scale value at initialization. Default: 1e-5 rNrr)r2r3rrrr layer_scale_1r4r5r drop_path1rrqr layer_scale_2 drop_path2)r0rrr%rrrrr>rrr3s"      zAttentionBlock.__init__c Cs>||||||}|||||}|Sr{)rrrrrrrrrrrrDs zAttentionBlock.forward) rlrmrnror4rpr8rqrrsr3rDrvrrr>rrs" ,rcsteZdZddddddejejdddd d f eeeeeeee ej ee ej ej e e e e d fd d Z d dZ ZS) FastVitStageTrrQNrrrrF)rrdepthtoken_mixer_type downsampledown_patch_size down_stride pos_emb_layerrrr%rproj_drop_ratedrop_path_raterc std|_|r.t||||| ||d|_n||ks:tt|_|dk r\|||d|_n t|_g}t |D]h}|dkr| t || | | | ||||dqr|dkr| t || | | | |||dqrt d |qrtj||_dS) aQFastViT stage. Args: dim: Number of embedding dimensions. depth: Number of blocks in stage token_mixer_type: Token mixer type. kernel_size: Kernel size for repmixer. mlp_ratio: MLP expansion ratio. act_layer: Activation layer. norm_layer: Normalization layer. proj_drop_rate: Dropout rate. drop_path_rate: Drop path rate. layer_scale_init_value: Layer scale value at initialization. inference_mode: Flag to instantiate block in inference mode. F)rrrrr%rr Nrrepmixer)rrr%rrrr attention)rr%rrrrz"Token mixer type: {} not supported)r2r3grad_checkpointingrrrr4r5pos_embr:appendrr ValueErrorformatrublocks)r0rrrrrrrrrrr%rrrrrr rZ block_idxr>rrr3sX#       zFastVitStage.__init__cCs@||}||}|jr2tjs2t|j|}n ||}|Sr{)rrrrSrZ is_scriptingrrrrrrrD3s    zFastVitStage.forward)rlrmrnr4rpr8rqstrrrrrsrr3rDrvrrr>rrs>VrcsheZdZUejjeed<ddddddddd d d d d d d dddej ej ddfe e e dfe e dfe e dfe edfe edfe e e eejdfe e eeeeeee ejejeeddfdd ZejddddZejjddZejjd,ddZejjd-ddZejjd d!Zd.d"d#Zejejd$d%d&Zd/ejed'd(d)Zejejd$d*d+ZZS)0FastVit fork_featrrQrQrQrrrr@ir|r|r|r|)FTTT)NNNNrrQrrFg@avg.N)in_chanslayers token_mixers embed_dims mlp_ratios downsamplesrepmixer_kernel_size num_classespos_embsrr drop_raterrrr cls_ratio global_poolrr%rr r&c#st|rdn||_||_||_g|_t||d|||_|d}d}ddt d|t | |D}g}t t |D]}||p|||k}t||||||| | | ||||||||| |||||d}||||}|r|d9}|jt|d|d|d g7_q|tj||_||_|jrdddd g|_t|jD]P\}}|dkrxtjd drxt} n |||} d |}!||!| qLnHt|d ||_}"t|d |"d dd|d|dd |_t|"||| d|_ |!|j"dS)NrrcSsg|] }|qSr)tolist)r-rArrrr1nsz$FastVit.__init__..)rrrrrrrrrrr%rrrrrr rQr|zstages.)Znum_chsZ reductionmodulerZ FORK_LAST3rr[T) rrrrrr r!r%r$)Z pool_typer)#r2r3rrrZ feature_inforstemrSZlinspacesumsplitr:rrrdictr4rustagesr* out_indices enumerateosenvirongetr5Z add_modulerqr final_convrheadrr)#r0rrrrrrrrrrrrrrrrrrrr%rr Zprev_dimrZdprrrirZstageZi_embZi_layerlayerZ layer_nameZfinal_featuresr>rrr3Ds " &     zFastVit.__init__rcCsDt|tjr@t|jddt|tjr@|jdk r@tj|jddS)zInit. for classificationrrNr)r\r4rr rFr)rrrrrrrs zFastVit._init_weightscCstSr{)setr0rrrno_weight_decayszFastVit.no_weight_decaycCstd|r dndddgdS)Nz^stemz^stages\.(\d+))z^stages\.(\d+).downsampler)z^stages\.(\d+).pos_embr)z^stages\.(\d+)\.\w+\.(\d+)N)r r)r )r0Zcoarserrr group_matcherszFastVit.group_matcherTcCs|jD] }||_qdSr{)rr)r0enablesrrrset_grad_checkpointings zFastVit.set_grad_checkpointingcCs|jjSr{)rfcrrrrget_classifierszFastVit.get_classifiercCs||_|j||dSr{)rrreset)r0rrrrrreset_classifierszFastVit.reset_classifierr@cCst||}g}t|jD]B\}}||}|jr||jkrt|d|}||}||q|jrf|S||}|S)Nr)r rrrrgetattrrr)r0rAZoutsidxblockrZx_outrrrforward_featuress    zFastVit.forward_features)rA pre_logitscCs|r|j|ddS||S)NT)r')r)r0rAr'rrr forward_headszFastVit.forward_headcCs"||}|jr|S||}|Sr{)r&rr(rrrrrDs   zFastVit.forward)F)T)N)F)rlrmrnrSrrrrrr4r8rprqrrrrrsr3rignorerrrr r"rtr&r(rDrvrrr>rr=sz      p     rc Ks|dddddttddd |S) Nr)rrr)r+g?Zbicubic)zstem.0.conv_kxk.0.convzstem.0.conv_scale.convzhead.fc) urlrZ input_sizeZ pool_sizecrop_pct interpolationZmeanrjZ first_conv classifierr)r,kwargsrrr_cfgs r1ztimm/) hf_hub_idgffffff?)r2r-)zfastvit_t8.apple_in1kzfastvit_t12.apple_in1kzfastvit_s12.apple_in1kzfastvit_sa12.apple_in1kzfastvit_sa24.apple_in1kzfastvit_sa36.apple_in1kzfastvit_ma36.apple_in1kzfastvit_t8.apple_dist_in1kzfastvit_t12.apple_dist_in1kzfastvit_s12.apple_dist_in1kzfastvit_sa12.apple_dist_in1kzfastvit_sa24.apple_dist_in1kzfastvit_sa36.apple_dist_in1kzfastvit_ma36.apple_dist_in1kcKs0|dd}tt||fdtd|di|}|S)Nr)rrrQrZ feature_cfgT)Zflatten_sequentialr)poprrr )variant pretrainedr0rmodelrrr_create_fastvit,s  r7cKs*tddddd}tdd|it|f|S) z%Instantiate FastViT-T8 model variant.)rQrQr|rQ)0`irrrrrrrrr fastvit_t8r5)r=r r7r5r0Z model_argsrrrr=8sr=cKs*tddddd}tdd|it|f|S) z&Instantiate FastViT-T12 model variant.rrr;rr< fastvit_t12r5)r@r>r?rrrr@Dsr@cKs*tddddd}tdd|it|f|S) z&Instantiate FastViT-S12 model variant.rrrrr< fastvit_s12r5)rAr>r?rrrrAPsrAc Ks<tddddddttddfdd}td d |it|f|S) z'Instantiate FastViT-SA12 model variant.rrrNrrrrrrrrrrr fastvit_sa12r5)rEr rrr7r?rrrrE\srEc Ks<tddddddttddfdd}td d |it|f|S) z'Instantiate FastViT-SA24 model variant.)r|r| r|rrNrrBrCrD fastvit_sa24r5)rHrFr?rrrrHisrHc Ks<tddddddttddfdd}td d |it|f|S) z'Instantiate FastViT-SA36 model variant.rrrrrNrrBrCrD fastvit_sa36r5)rKrFr?rrrrKvsrKc Ks<tddddddttddfdd}td d |it|f|S) z'Instantiate FastViT-MA36 model variant.rI)Li0i`rNrrBrCrD fastvit_ma36r5)rNrFr?rrrrNsrN)r*)F)F)F)F)F)F)F)F)9r functoolsrtypingrrrrSZtorch.nnr4Z timm.datarrZ timm.layersr r r r r rrZ_builderrZ _manipulater _registryrrrrsrrwrprqrrrurrrrrrrrrrrr1Z default_cfgsr7r=r@rArErHrKrNrrrrs  $   r -D1 p4t:9a7 )