U ʹj\ @s(dZddlmZmZmZddlmZddlmZm Z m Z m Z m Z ddl Z ddlmZddlmZmZddlmZmZmZmZmZmZmZdd lmZdd lmZmZdd l m!Z!m"Z"d gZ#eGd ddZ$ddZ%eGdddZ&eGdddZ'dddZ(Gdddej)Z*Gdddej)Z+Gdd d ej)Z,Gd!d"d"ej)Z-Gd#d$d$ej)Z.Gd%d&d&ej)Z/d'd(d'd)d*d*ej0ej1df d+d,Z2d-d.Z3d/d0Z4d1d2Z5e'e6e7e e8efd3d4d5Z9Gd6d d ej)Z:dd7d8Z;ed?d@dAdBdCdDdEe'e$dFd'd;dd?d@dAdBddCdGdEe'e$dFd'd;ddHd@ddIdBdCdGdEe'e$d9d:d;ddHd@d(dddBdCdJdEe'e$d(d'dd*d=e&dKdLd)dMdNdOdCddPdEe'e$d(d'dd*d=e&dQdLdRdSdTdddUdEe'e$d(d'dd*d=e&dVdLdRdSdTdddUdEe'e$d(d'dd*d=e&dVdLd)dBddWdddXdEe'e$d(d'dd*d=e&dKdLd)dBddddUdEe'e$d(d'dd*d=e&dKdLd)dBddCdddYdEe(dBdBdZe(d[d\dZe(e(d]d^dZe(dBdBdCd_e(d[d\dCd_e(dCd`e(d]d^dCd_e(dWeddpdqZ?e"e?drdsdte?e?e?drdudte?drdvdte?e?e?e?drdwdxdyddze?drd{dydd|e?dxd}e?drd~dxdyddze?drddxdyddze?drddxddydde?dxd}e?drddxdyddze?drddxdyddze?dxd}e?drddxdyddze?drddxdyddze?dxd}e?drddxdyddze?drddxdddddZ@e!de:dddZAe!de:dddZBe!de:dddZCe!de:dddZDe!de:dddZEe!de:dddZFe!de:dddZGe!de:dddZHe!de:dddZIe!de:dddZJe!de:dddZKe!de:dddZLe!de:dddZMe!de:dddZNe!de:dddZOe!de:dddZPe!de:dddZQe!de:dddZRe!de:dddZSe!de:dddZTe!de:dddZUe!de:dddZVe!de:dddZWdS)aPyTorch CspNet A PyTorch implementation of Cross Stage Partial Networks including: * CSPResNet50 * CSPResNeXt50 * CSPDarkNet53 * and DarkNet53 for good measure Based on paper `CSPNet: A New Backbone that can Enhance Learning Capability of CNN` - https://arxiv.org/abs/1911.11929 Reference impl via darknet cfg files at https://github.com/WongKinYiu/CrossStagePartialNetworks Hacked together by / Copyright 2020 Ross Wightman ) dataclassasdictreplace)partial)AnyDictOptionalTupleUnionNIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)ClassifierHead ConvNormAct ConvNormActAaDropPathget_attncreate_act_layermake_divisible)build_model_with_cfg) named_applyMATCH_PREV_GROUP)register_modelgenerate_default_cfgsCspNetc@sveZdZUdZeeeedffed<dZeeeedffed<dZ eed<dZ eee fed <dZ e e ed <d S) CspStemCfg .out_chsstride kernel_sizepaddingpoolN)__name__ __module__ __qualname__rr intr __annotations__r r"r$strr%rr,r,Q/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/models/cspnet.pyrs  rcCsNt|ttfs|f}t|}||}|dkr8|d|St||df|S)Nr) isinstancetuplelistlen)xnZcurr_nZpad_nr,r,r-_pad_arg(s r5c@seZdZUdZeedfed<dZeedfed<dZe eeedffed<dZ e eeedffed <d Z e e ee dffed <d Z e e ee dffed <d Ze eeedffed<dZee eeedffed<dZee eeefed<dZe eeefed<dZe eeefed<d Ze e ee dffed<d Ze eeedffed<d Ze eeedffed<ddZdS) CspStagesCfgr!r!r.depthrrr rgroups? block_ratio bottle_ratioFavg_downN attn_layer attn_kwargscsp stage_typebottle block_type expand_ratio cross_linear down_growthcCst|j}t|j|kstt|j||_t|j||_t|j||_t|j||_t|j ||_ t|j ||_ t|j ||_ t|j ||_ t|j ||_ t|j||_t|j||_t|j||_dSN)r2r9rAssertionErrorr5r r?rArBrCrDrErGrIrJrKrL)selfr4r,r,r- __post_init__Fs zCspStagesCfg.__post_init__)r&r'r(r9r r)r*rr r r?rAfloatrBrCboolrDrr+rErrGrIrJrKrLrPr,r,r,r-r63s  r6c@sReZdZUeed<eed<dZeed<dZe ed<dZ e ed<d Z e e ed <d S) CspModelCfgstemstagesTzero_init_lastZ leaky_relu act_layerZ batchnorm norm_layerNaa_layer) r&r'r(rr*r6rVrRrWr+rXrYrr,r,r,r-rSXs    rSr@Fsiludarkc s|r ttdddddd} n"ttfdddDd ddd } t| ttfd dd Dtfd ddDd|d|||d|d |dS)N@rr#)rr"r r$r%csg|]}t|qSr,r.0cwidth_multiplierr,r- ssz_cs3_cfg..rr\r!rr"r r%csg|]}t|qSr,r^r_rbr,r-rdxsr:csg|]}t|qSr,)r))r`d)depth_multiplierr,r-rdys)r!r] r!?cs3) rr9r rBrArCrDrErGrIrTrUrW)rrr0rSr6) rcrhrCrWfocusrDrErBrIZstem_cfgr,)rhrcr-_cs3_cfgbs<   rnc sHeZdZdZdddejejddddf fdd Zd d Zd d Z Z S) BottleneckBlockz ResNe(X)t Bottleneck Block r?FNc stt|tt||} t||d} | dk o4|}| dk oB| }t|| fddi| |_t| | fd||| d| |_|r| | |dnt |_ t| |fddd| |_ |r| ||dnt |_ | rt| nt |_t||_dS) NrWrXr"rr!r"dilationr?Z drop_layerrWFr"Z apply_act)superro__init__r)rounddictrconv1conv2nnIdentityattn2conv3attn3r drop_pathract3)rOin_chsrrtrBr?rWrXZ attn_lastrD drop_blockrmid_chsckwargsZ attn_first __class__r,r-rxs,  zBottleneckBlock.__init__cCstj|jjjdSrM)r}initzeros_rbnweightrOr,r,r-rVszBottleneckBlock.zero_init_lastcCsR|}||}||}||}||}||}|||}||}|SrM)r{r|rrrrrrOr3Zshortcutr,r,r-forwards      zBottleneckBlock.forward r&r'r(__doc__r}ReLU BatchNorm2drxrVr __classcell__r,r,rr-rosrocsFeZdZdZdddejejdddffdd Zdd Zd d Z Z S) DarkBlockz DarkNet Block rrjNrqc stt|tt||} t||d} t|| fddi| |_|dk rV|| |dnt |_ t| |fd||| d| |_ | rt | nt |_ dS)Nrrr"rrur!rs)rwrrxr)ryrzrr{r}r~attnr|rr rOrrrtrBr?rWrXrDrrrrrr,r-rxs"  zDarkBlock.__init__cCstj|jjjdSrMr}rrr|rrrr,r,r-rVszDarkBlock.zero_init_lastcCs4|}||}||}||}|||}|SrMr{rr|rrr,r,r-rs    zDarkBlock.forwardrr,r,rr-rsrcsFeZdZdZdddejejdddffdd Zdd Zd d Z Z S) EdgeBlockzZ EdgeResidual / Fused-MBConv / MobileNetV1-like 3x3 + 1x1 block (w/ activated output) rrjNrqc stt|tt||} t||d} t|| fd||| d| |_|dk r\|| |dnt |_ t| |fddi| |_ | rt | nt |_ dS)Nrrr!rsrur"r)rwrrxr)ryrzrr{r}r~rr|rrrrr,r-rxs"  zEdgeBlock.__init__cCstj|jjjdSrMrrr,r,r-rVszEdgeBlock.zero_init_lastcCs4|}||}||}||}|||}|SrMrrr,r,r-rs    zEdgeBlock.forwardrr,r,rr-rsrc s>eZdZdZdddddddddef fdd Zdd ZZS) CrossStagez Cross Stage.r@rNFc stt|| p|} | r|n|}tt|||_}tt||}t|d|dd}|dd}|dks|| |kr| rt |dkrt dnt t ||fdd| d||_n t ||fd|| | |d ||_|}nt |_|}t||fd| d ||_|d}t |_t|D]F}|jt||f||||| |dk rV||nd d ||}q&t||dfd di||_t||fd di||_dSNrWrXrrrYrrr"r r?r!r"r rtr?rYrvrqrrrtrBr?rr")rwrrxr)ry expand_chsrzgetpopr} Sequential AvgPool2dr~r conv_downrconv_expblocksrange add_moduler+conv_transition_bconv_transitionrOrrr rtr9rArBrJr?first_dilationrCrLrK block_dprblock_fn block_kwargsZdown_chsZexp_chs block_out_chs conv_kwargsrYprev_chsirr,r-rx sZ      zCrossStage.__init__cCs`||}||}|j|jddd\}}||}||}|tj ||gdd}|SNrr)dim) rrsplitrrr contiguousrtorchcat)rOr3Zxsxboutr,r,r-rMs   zCrossStage.forwardr&r'r(rrorxrrr,r,rr-r sArc s>eZdZdZdddddddddef fdd Zdd ZZS) CrossStage3z`Cross Stage 3. Similar to CrossStage, but with only one transition conv for the output. r@rNFc stt|| p|} | r|n|}tt|||_}tt||}t|d|dd}|dd}|dks|| |kr| rt |dkrt dnt t ||fdd| d||_n t ||fd|| | |d ||_|}n d|_|}t||fd| d ||_|d}t |_t|D]F}|jt||f||||| |dk rR||nd d ||}q"t||fd di||_dSr)rwrrxr)ryrrzrrr}rrr~rrrrrrrr+rrrr,r-rx[sX     zCrossStage3.__init__cCsR||}||}|j|jddd\}}||}|tj||gdd}|Sr)rrrrrrrr)rOr3x1Zx2rr,r,r-rs    zCrossStage3.forwardrr,r,rr-rWs >rcs8eZdZdZdddddedffdd Zdd ZZS) DarkStagezDarkNet stage.r@rNFc  stt|| p|} t| d| dd}| dd}| rzt|dkrTtdnt t ||fdd|d||_ n t ||fd|| ||d ||_ |}t t ||}t|_t|D]B}|jt|| f|||||| dk r| |nd d | |}qdS) NrWrXrrrYrrrr!rrqr)rwrrxrzrrr}rrr~rrr)ryrrrr+)rOrrr rtr9rArBr?rrCrrrrrYrrrrr,r-rxsH    zDarkStage.__init__cCs||}||}|SrM)rrrOr3r,r,r-rs  zDarkStage.forwardrr,r,rr-rs .rr!rrr#c  Cst} g} t|ttfs |g}t|} | s0t|dks.rrcsg|]}tt|qSr,)rzzipkeys)r`valuesZcfg_dictr,r-rd=srrrrrr rrr)r rrtrrYrDrzstages.r)rrUr2r9rZlinspacesumrrrrzrWrXrrrrrrrYr}r)rrrrZ num_stagesrrrtZ net_striderrrrUZ stage_idxrrZattn_fnr rr,rr-create_csp_stages3sZ  (         rcseZdZdZdedfd d Zejjdd d Z ejjdddZ ejjddZ dddZ ddZ dedddZddZZS) raCross Stage Partial base model. Paper: `CSPNet: A New Backbone that can Enhance Learning Capability of CNN` - https://arxiv.org/abs/1911.11929 Ref Impl: https://github.com/WongKinYiu/CrossStagePartialNetworks NOTE: There are differences in the way I handle the 1x1 'expansion' conv in this impl vs the darknet impl. I did it this way for simplicity and less special cases. r!ravgrqT)rc  st||_||_|dks"tt|f| }t|j|j|j d} g|_ t |ft |j | \|_ } |j | ddt|||| dd\|_} | dd} |j | | |_t| |||d|_ttt|d|dS) a Args: cfg (CspModelCfg): Model architecture configuration in_chans (int): Number of input channels (default: 3) num_classes (int): Number of classifier classes (default: 1000) output_stride (int): Output stride of network, one of (8, 16, 32) (default: 32) global_pool (str): Global pooling type (default: 'avg') drop_rate (float): Dropout rate (default: 0.) drop_path_rate (float): Stochastic depth drop-path rate (default: 0.) zero_init_last (bool): Zero-init last weight of residual path kwargs (dict): Extra kwargs overlayed onto cfg )r)rWrXrYNr.)rrrr)Z in_features num_classes pool_type drop_rate)rV)rwrxrrrNrrzrWrXrYrrrrTextendrrU num_featuresrheadrr _init_weights)rOrrrr global_poolrrrVkwargsZ layer_argsZstem_feat_infoZstage_feat_inforrr,r-rxrs:      zCspNet.__init__FcCs"td|r dn ddtfdgd}|S)Nz^stem^stages\.(\d+))z^stages\.(\d+)\.blocks\.(\d+)Nz^stages\.(\d+)\..*transition)r)r)rTr)rzr)rOZcoarseZmatcherr,r,r- group_matcherszCspNet.group_matchercCs|r tddS)Nz$gradient checkpointing not supported)rN)rOenabler,r,r-set_grad_checkpointingszCspNet.set_grad_checkpointingcCs|jjSrM)rfcrr,r,r-get_classifierszCspNet.get_classifiercCst|j|||jd|_dS)N)rr)rrrr)rOrrr,r,r-reset_classifierszCspNet.reset_classifiercCs||}||}|SrMrTrUrr,r,r-forward_featuress  zCspNet.forward_features pre_logitscCs|j||dS)Nr)r)rOr3rr,r,r- forward_headszCspNet.forward_headcCs||}||}|SrM)rrrr,r,r-rs  zCspNet.forward)r!rrrrqrqT)F)T)r)F)r&r'r(rrSrxrZjitignorerrrrrrRrrrr,r,rr-rhs* :    cCst|tjr:tjj|jddd|jdk rtj|jnPt|tjrttjj |jddd|jdk rtj|jn|rt |dr| dS)NZfan_outZrelu)modeZ nonlinearityrqg{Gz?)meanstdrV) r/r}ZConv2drZkaiming_normal_rZbiasrZLinearZnormal_hasattrrV)rnamerVr,r,r-rs    rr\rmaxrfr7r:r@rjT)r9rr rJrBrKr)rrr\)r9rr rJrBrArK)r<r=r>irp)r9rr r?rJrBrArK)rrrrr)r\r;r<r=r>)rr@)rjr@)r@rj)r9rr rJrBrArLrI)rrrrr)r)rj)r@)r9rr rBrArGrI)rrrrrse)r9rr rBrArDrGrI)r9rr rBrArCrGrI)rcrhg?gq= ףp?g?gHzG?)rcrhrm)rm)Zrd_ratio)rDrE)rDrcrhre)r!r] r)rrr<r=)r9rr r?rBrArDrlg?r)rcrhrBrI)rcrhrBrIrDrE) cspresnet50 cspresnet50d cspresnet50w cspresnext50 cspdarknet53 darknet17 darknet21 sedarknet21 darknet53 darknetaa53 cs3darknet_s cs3darknet_m cs3darknet_l cs3darknet_xcs3darknet_focus_scs3darknet_focus_mcs3darknet_focus_lcs3darknet_focus_xcs3sedarknet_lcs3sedarknet_xcs3sedarknet_xdw cs3edgenet_xcs3se_edgenet_xcKsP|ds|drd}nd}|d|}tt||ft|td|dd|S) NZdarknetZ cspdarknet)rrrr!rr8)rrrr!r out_indicesT)Zflatten_sequentialr)Z model_cfgZ feature_cfg) startswithrrr model_cfgsrz)variant pretrainedrZdefault_out_indicesrr,r,r-_create_cspnets  r#c Ks|dddddttddd |S) Nr)r!r<r<)rrgMb?Zbilinearzstem.conv1.convzhead.fc) urlrZ input_sizeZ pool_sizecrop_pct interpolationrrZ first_conv classifierr )r$rr,r,r-_cfgsr(ztimm/zlhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/cspresnet50_ra-d3e8d487.pth) hf_hub_idr$zqhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/cspresnext50_ra_224-648b4713.pthzqhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/cspdarknet53_ra_256-d05c7c21.pthzthttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/darknet53_256_c2ns-3aeff817.pthZbicubic)r! r*)r)r$r&test_input_size test_crop_pctzrhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/darknetaa53_c2ns-5c28ec8a.pth)r)r$r+r,)r&zshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3darknet_m_c2ns-43f06604.pthgffffff?zshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3darknet_l_c2ns-16220c5d.pthzshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3darknet_x_c2ns-4e4490aa.pth)r)r$r&r%r+r,zyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3darknet_focus_m_c2ns-e23bed41.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3darknet_focus_l_c2ns-65ef8888.pthzuhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3sedarknet_l_c2ns-e8d1dc13.pthzuhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3sedarknet_x_c2ns-b4d0abc0.pthzqhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3edgenet_x_c2-2e1610a9.pthzvhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-tpu-weights/cs3se_edgenet_x_c2ns-76f8e3ac.pth)r!@r-)zcspresnet50.ra_in1kzcspresnet50d.untrainedzcspresnet50w.untrainedzcspresnext50.ra_in1kzcspdarknet53.ra_in1kzdarknet17.untrainedzdarknet21.untrainedzsedarknet21.untrainedzdarknet53.c2ns_in1kzdarknetaa53.c2ns_in1kzcs3darknet_s.untrainedzcs3darknet_m.c2ns_in1kzcs3darknet_l.c2ns_in1kzcs3darknet_x.c2ns_in1kzcs3darknet_focus_s.untrainedzcs3darknet_focus_m.c2ns_in1kzcs3darknet_focus_l.c2ns_in1kzcs3darknet_focus_x.untrainedzcs3sedarknet_l.c2ns_in1kzcs3sedarknet_x.c2ns_in1kzcs3sedarknet_xdw.untrainedzcs3edgenet_x.c2_in1kzcs3se_edgenet_x.c2ns_in1k)returncKstdd|i|S)Nrr")rr#r"rr,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nr r")r r/r0r,r,r-r sr cKstdd|i|S)Nr r")r r/r0r,r,r-r sr cKstdd|i|S)Nr r")r r/r0r,r,r-r sr cKstdd|i|S)Nr r")r r/r0r,r,r-r sr cKstdd|i|S)Nr r")r r/r0r,r,r-r sr cKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-r srcKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-rsrcKstdd|i|S)Nrr")rr/r0r,r,r-r#srcKstdd|i|S)Nrr")rr/r0r,r,r-r(srcKstdd|i|S)Nrr")rr/r0r,r,r-r-srcKstdd|i|S)Nrr")rr/r0r,r,r-r2srcKstdd|i|S)Nrr")rr/r0r,r,r-r7srcKstdd|i|S)Nrr")rr/r0r,r,r-r<srcKstdd|i|S)Nrr")rr/r0r,r,r-rAsrcKstdd|i|S)Nrr")rr/r0r,r,r-rFsrcKstdd|i|S)Nrr")rr/r0r,r,r-rKsrcKstdd|i|S)Nrr")rr/r0r,r,r-rPsr) r@r@FrZFNNr@r[)F)F)r#)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)Xr dataclassesrrr functoolsrtypingrrrr r rZtorch.nnr}Z timm.datar r Z timm.layersrrrrrrrZ_builderrZ _manipulaterr _registryrr__all__rr5r6rSrnModulerorrrrrrrrrrrrQr)r+rrrrzr r#r(Z default_cfgsrrr r r r r rrrrrrrrrrrrrrrrr,r,r,r-s  $  $  %3((MK8 6   5i                 "  J