U ʹjr2r!rrr^s&  zDepthwiseSeparableConv.__init__cCs.|dkrtdd|jjdStd|jjdSdS)Nr?rS forward_prer@r-rC)rDrS in_channelsrErFrrrrHssz#DepthwiseSeparableConv.feature_infocCsN|}||}||}||}||}||}|jrJ|||}|SrI)rQr6rRrSrTr4r8rJrrrr&ys     zDepthwiseSeparableConv.forwardrLrrr!rr Ys csReZdZdZdddddddddejejdddffd d Zd d Zd dZ Z S)r a Inverted residual block w/ optional SE Originally used in MobileNet-V2 - https://arxiv.org/abs/1801.04381v4, this layer is often referred to as 'MBConv' for (Mobile inverted bottleneck conv) and is also used in * MNasNet - https://arxiv.org/abs/1807.11626 * EfficientNet - https://arxiv.org/abs/1905.11946 * MobileNet-V3 - https://arxiv.org/abs/1905.02244 r$rr-F?Nr.cstt|t| | }|pi}t|| }t||}||koJ|dkoJ| |_t||| fd|i||_||dd|_ t|||f||||d||_ ||dd|_ |r||| dnt |_t||| fd|i||_||dd|_|rt|nt |_dS) Nrr3Trr/rNFrP)rr rrrrr4rrSr6rQrTrr7rRconv_pwlbn3rr8)rrr9rUr0r1rr:rV exp_ratioexp_kernel_sizerWr r<rX conv_kwargsr=r>mid_chsr2r!rrrs2   zInvertedResidual.__init__cCs.|dkrtdd|jjdStd|jjdSdSNr?r]rYr@r-rCrDr]rZrErFrrrrHszInvertedResidual.feature_infocCsb|}||}||}||}||}||}||}||}|jr^|||}|SrI) rSr6rQrTrRr]r^r4r8rJrrrr&s       zInvertedResidual.forwardrLrrr!rr s$ csJeZdZdZdddddddddejejddd ffd d Zd d ZZ S)r z, Inverted residual block w/ CondConv routingr$rr-Fr[Nrr.csV||_t|jd}tt|j|||||||| || | | || ||dt||j|_dS)N) num_experts)rUr0r1rr:r rVr_r`rWrXr<rar=)rerDrr rrZLinear routing_fn)rrr9rUr0r1rr:rVr_r`rWr r<rXrer=rar!rrrs*  zCondConvResidual.__init__cCs|}t|dd}t||}|||}||}|||}| |}| |}| ||}| |}|j r|||}|S)Nr)FZadaptive_avg_pool2dflattentorchZsigmoidrfrSr6rQrTrRr]r^r4r8)rr%rKZ pooled_inputsZrouting_weightsrrrr&s       zCondConvResidual.forward) r'r(r)r*rr+rMrr&r,rrr!rr s"c sPeZdZdZdddddddddejejdd f fd d Zd d ZddZ Z S)r a( Residual block with expansion convolution followed by pointwise-linear w/ stride Originally introduced in `EfficientNet-EdgeTPU: Creating Accelerator-Optimized Neural Networks with AutoML` - https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html This layer is also called FusedMBConv in the MobileDet, EfficientNet-X, and EfficientNet-V2 papers * MobileDet - https://arxiv.org/abs/2004.14525 * EfficientNet-X - https://arxiv.org/abs/2102.05610 * EfficientNet-V2 - https://arxiv.org/abs/2104.00298 r$rrr-Fr[Nr.c stt|t| | }|dkr.t|| }n t|| }t||}||koX|dkoX| |_t|||||||d|_||dd|_ |r||| dnt |_ t||| |d|_ ||dd |_|rt|nt |_dS) Nrrr/TrrNrOFr\)rr rrrrr4rconv_expr6rr7rRr]rTrr8)rrr9r`r0r1rr:Z force_in_chsrVr_rWr r<rXr=r>rbr2r!rrrs*   zEdgeResidual.__init__cCs.|dkrtdd|jjdStd|jjdSdSrcrdrFrrrrH szEdgeResidual.feature_infocCsN|}||}||}||}||}||}|jrJ|||}|SrI)rjr6rRr]rTr4r8rJrrrr&s     zEdgeResidual.forwardrLrrr!rr s" )r*riZtorch.nnrrrgZ timm.layersrrrrr__all__rModulerr r r r r rrrrs$   !,;#