U ʹjd@sdZddlZddlmZmZmZddlZddlmm Z ddlmZddl m Z m Z mZmZmZmZddlmZddlmZdd lmZmZmZdd lmZmZmZmZmZm Z dd l!m"Z#gZ$dqd dZ%drddZ&dsddZ'dtddZ(e)ee%ddddde%ddddde&dddddddd e&dddd!dddd e&dd!dd"dddd fdd#d$d$d%d&d'ee%dd(dd)e%dddd)e&dddd"ddd*e&dd!dd+ddd*e&dd"dd,ddd*fdd#d$d$d%d-d'ee%dd(dd)e%dddd)e&dd"dd,ddd*e&dd.dd/ddd*e&dd0dd1ddd*fdd#d$d$d%d2d'ee%dd(dd)e%dddd)e&dd"dd,ddd*e&dd.dd/ddd*e&dd0dd1ddd*fdd#d$d$d3e)d4d5d2d6e(de(d7e(d8e(de(d9e(d:e(dd; Z*eGdd?d?ej+Z-Gd@dAdAej+Z.eGdBdCdCej+Z/edDe,edEe/dudGdHZ0dvdIdJZ1dwdKdLZ2ee2dMdNe2dMdNe2dMdNe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdQdRddSe2dMdQdRddSe2dMdQdRddSdTZ3edxedUdVdWZ4edyedUdXdYZ5edzedUdZd[Z6ed{edUd\d]Z7ed|edUd^d_Z8ed}edUd`daZ9ed~edUdbdcZ:ededUdddeZ;ededUdfdgZdjdkdldmdndodpdS)a  MobileViT Paper: V1: `MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer` - https://arxiv.org/abs/2110.02178 V2: `Separable Self-attention for Mobile Vision Transformers` - https://arxiv.org/abs/2206.02680 MobileVitBlock and checkpoints adapted from https://github.com/apple/ml-cvnets (original copyright below) License: https://github.com/apple/ml-cvnets/blob/main/LICENSE (Apple open source) Rest of code, ByobNet, and Transformer block hacked together by / Copyright 2022, Ross Wightman N)CallableTupleOptional)nn) to_2tuplemake_divisible GroupNorm1ConvMlpDropPath is_exportable)build_model_with_cfg)register_notrace_module)register_modelgenerate_default_cfgsregister_model_deprecations)register_block ByoBlockCfg ByoModelCfgByobNetLayerFn num_groups)Block@c Cstd|||d|tddddS)NZbottler T)Z bottle_inZ linear_out)typedcsgsbr block_kwargs)rdictrrrrr#T/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/models/mobilevit.py_inverted_residual_block!s r%c Cs,t||||dtdd|dt|||ddfS)Nr" mobilevitr )transformer_dimtransformer_depth patch_size)rrrrr r%rr!rrrr(r)r*rr#r#r$_mobilevit_block(sr-@?c Cs.t||||dtdd|d|dt||ddfS)Nr" mobilevit2r )r)r*)rrrrrrr r+)rrrr)r*rZtransformer_brr#r#r$_mobilevitv2_block6sr2?c sd}dkr"tfdd|D}ttd|ddddtd |dd ddtd|d d d d td|d d d d td|d d d d ftd ddddd}|S)N)@ir3csg|]}t|qSr#)int).0r multiplierr#r$ Fsz$_mobilevitv2_cfg..r rr/r"r.)rrrr)r& 3x3silu)blocksstem_chs stem_type stem_pool downsample act_layer)tuplerr%r2r8)r;Zchscfgr#r:r$_mobilevitv2_cfgCs" rJr"r=0r4r,P`r?r@rAi@)rBrCrDrErFrG num_featuresr>)rrr)rrrr(r)r*xr7r5iseg?)Zrd_ratio)rBrCrDrErFZ attn_layerZ attn_kwargsrPg?g?g?g?) mobilevit_xxs mobilevit_xs mobilevit_sZ semobilevit_smobilevitv2_050mobilevitv2_075mobilevitv2_125mobilevitv2_100mobilevitv2_150mobilevitv2_175mobilevitv2_200cseZdZdZdddddddddd d d d d d dejfeeeeeeeee eefeeeeeeeee ee e d fdd Z ejejdddZZS)MobileVitBlockzS MobileViT block Paper: https://arxiv.org/abs/2110.02178?context=cs.LG Nr=r r3r r r/r.r&F)in_chsout_chs kernel_sizestride bottle_ratio group_sizedilation mlp_ratior(r)r* num_heads attn_dropdrop no_fusiondrop_path_ratelayerstransformer_norm_layerc stt|ptt||}|p(|}p8t||j||||||dd|_tj |ddd|_ tj fddt | D|_ |_j|ddd|_|rd|_nj||||dd|_t| |_|jd|jd|_dS) Nrrgrhgroupsrkr Frgbiasc s(g|] }tdjd qS)T)rlrmZqkv_biasrn proj_drop drop_pathrG norm_layer)TransformerBlockactr9_rnrorqrrrlrmr(rsr#r$r<s z+MobileVitBlock.__init__..rgrh)superra__init__rrr conv_norm_actconv_kxkrConv2dconv_1x1 Sequentialrange transformernorm conv_proj conv_fusionrr* patch_area)selfrerfrgrhrirjrkrlr(r)r*rmrnrorprqrrrskwargsru __class__rr$rs0      zMobileVitBlock.__init__xreturncCs|}||}||}|j\}}|j\}}}}t|||t|||} } | || |} } | | } d}| |ks| |krtj|| | fddd}d}|||| || | dd}|||| |j  dd||j | d}| |}| |}| ||j | d}| dd||| | ||}| dd||| || |}|rntj|||fddd}||}|jdk r|tj||fdd }|S) NFbilinearsizemodeZ align_cornersTr r.r=dim)rrr*shapemathceilF interpolatereshapeZ transposerrr contiguousviewrrtorchcat)rrZshortcutpatch_hpatch_wBCHWnew_hnew_w num_patch_h num_patch_w num_patchesrr#r#r$forwards2   & *      zMobileVitBlock.forward)__name__ __module__ __qualname____doc__rZ LayerNormr8rfloatrboolrrrrTensorr __classcell__r#r#rr$rasN `_ Args: embed_dim (int): :math:`C_{in}` from an expected input of size :math:`(B, C_{in}, P, N)` mlp_ratio (float): Inner dimension ratio of the FFN relative to embed_dim drop (float): Dropout rate. Default: 0.0 attn_drop (float): Dropout rate for attention in multi-head attention. Default: 0.0 drop_path (float): Stochastic depth rate Default: 0.0 norm_layer (Callable): Normalization layer. Default: layer_norm_2d Shape: - Input: :math:`(B, C_{in}, P, N)` where :math:`B` is batch size, :math:`C_{in}` is input embedding dim, :math:`P` is number of pixels in a patch, and :math:`N` is number of patches, - Output: same shape as the input r/rdN)rrlrornryrcsrt|ptj}|pt}|||_t|||d|_t||_ |||_ t |t ||||d|_ t||_dS)N)rrnrx)Z in_featuresZhidden_featuresrGro)rrrZSiLUrnorm1rattnr drop_path1norm2r r8mlp drop_path2)rrrlrornryrGrzrr#r$rs      zLinearTransformerBlock.__init__rcCsj|dkr$|||||}n(|}||}|||}|||}|||||}|S)N)rrrrrr)rrrresr#r#r$rs  zLinearTransformerBlock.forward)r/rdrdrdNN)N) rrrrr8rrrrrrrr#r#rr$rs rcseZdZdZddddddddd d d d defeeeeeeeeeefeeeeeeeee e d fd d Z e j e j dddZZS)MobileVitV2Blockz8 This class defines the `MobileViTv2 block <>`_ Nr=r3r rbr/r.rcrd)rerfrgrirjrkrlr(r)r*rnrorqrrrsc stt|ptt||}|p(|}p8t||j|||d||dd|_tj |ddd|_ tj fddt | D|_ |_j|dddd|_t| |_|jd|jd|_t|_dS) Nr rrtFrvc s$g|]}tjdqS))rlrnroryrGrz)rr|r}rnrorqrrrlr(rsr#r$r<s z-MobileVitV2Block.__init__..)rgrhZ apply_act)rrrrrrrrrrrrrrrrrr*rr coreml_exportable)rrerfrgrirjrkrlr(r)r*rnrorqrrrsrrurrr$rs,     zMobileVitV2Block.__init__rc Cs~|j\}}}}|j\}}t|||t|||}} ||| |} } | | } ||ksh| |kr~tj||| fddd}||}||}|jd}|jrtj |||f||fd}n$| ||| || | ddddd d }| ||d | }| |}| |}|jr4| ||||| | }tj||d }n<| ||||| |  ddd d dd}| ||| || |}||}|S) NrTrr rrr=r.r&r)Zupscale_factor)rr*rrrrrrrZunfoldrZpermuterrZ pixel_shuffler) rrrrrrrrrrrrrr#r#r$rs. &   $  $ zMobileVitV2Block.forward)rrrrrr8rrrrrrrrrrr#r#rr$rsB 2rr'r1FcKs0tt||f|st|nt|tddd|SNT)Zflatten_sequential)Z model_cfgZ feature_cfgr r model_cfgsr!variantZ cfg_variant pretrainedrr#r#r$_create_mobilevit#srcKs0tt||f|st|nt|tddd|Srrrr#r#r$_create_mobilevit2+src Ks |ddddddddd d d |S) Ni)r=r6r6)rcrcg?Zbicubic)rdrdrd)r3r3r3z stem.convzhead.fcF) urlZ num_classes input_size pool_sizecrop_pct interpolationZmeanZstdZ first_conv classifierZfixed_input_sizer#)rrr#r#r$_cfg3srztimm/) hf_hub_idg"~j?)rr)r=r7r7) r)rrrr)zmobilevit_xxs.cvnets_in1kzmobilevit_xs.cvnets_in1kzmobilevit_s.cvnets_in1kzmobilevitv2_050.cvnets_in1kzmobilevitv2_075.cvnets_in1kzmobilevitv2_100.cvnets_in1kzmobilevitv2_125.cvnets_in1kzmobilevitv2_150.cvnets_in1kzmobilevitv2_175.cvnets_in1kzmobilevitv2_200.cvnets_in1k$mobilevitv2_150.cvnets_in22k_ft_in1k$mobilevitv2_175.cvnets_in22k_ft_in1k$mobilevitv2_200.cvnets_in22k_ft_in1k(mobilevitv2_150.cvnets_in22k_ft_in1k_384(mobilevitv2_175.cvnets_in22k_ft_in1k_384(mobilevitv2_200.cvnets_in22k_ft_in1k_384)rcKstdd|i|S)NrWr)rWrrrr#r#r$rWosrWcKstdd|i|S)NrXr)rXrrr#r#r$rXtsrXcKstdd|i|S)NrYr)rYrrr#r#r$rYysrYcKstdd|i|S)NrZr)rZrrr#r#r$rZ~srZcKstdd|i|S)Nr[r)r[rrr#r#r$r[sr[cKstdd|i|S)Nr]r)r]rrr#r#r$r]sr]cKstdd|i|S)Nr\r)r\rrr#r#r$r\sr\cKstdd|i|S)Nr^r)r^rrr#r#r$r^sr^cKstdd|i|S)Nr_r)r_rrr#r#r$r_sr_cKstdd|i|S)Nr`r)r`rrr#r#r$r`sr`rrrrrr)Zmobilevitv2_150_in22ft1kZmobilevitv2_175_in22ft1kZmobilevitv2_200_in22ft1kZmobilevitv2_150_384_in22ft1kZmobilevitv2_175_384_in22ft1kZmobilevitv2_200_384_in22ft1k)r)r&r)r.r/r0)r3)NF)NF)r@)F)F)F)F)F)F)F)F)F)F)?rrtypingrrrrZtorch.nn.functionalrZ functionalrZ timm.layersrrrr r r Z_builderr Z _features_fxr _registryrrrZbyobnetrrrrrrZvision_transformerrr{__all__r%r-r2rJr!rModulerarrrrrrZ default_cfgsrWrXrYrZr[r]r\r^r_r`rr#r#r#r$sp               Lkw:]     1