U ʹj@sVddlZddlZddlZddlmZddZdd d Zdd d ZdddZddZ dS)N)_calculate_fan_in_and_fan_outcCsdd}||d|ks(||d|kr6tjddd||||}||||}|d|dd|d|||td|||j||d|S) NcSsdt|tddS)N?@)matherfsqrt)xr V/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/layers/weight_init.pynorm_cdf sz _trunc_normal_..norm_cdfzjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelr)minmax) warningswarnuniform_Zerfinv_mul_rradd_Zclamp_)tensormeanstdabr lur r r _trunc_normal_s  rrrc Cs0tt|||||W5QRSQRXdS)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this impl is similar to the PyTorch trunc_normal_, the bounds [a, b] are applied while sampling the normal with mean/std applied, therefore a, b args should be adjusted to match the range of mean, std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) N)torchno_gradrrrrrrr r r trunc_normal_+s r#c Cs8t&t|dd|||||W5QRX|S)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsquently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) rr)r r!rrrr"r r r trunc_normal_tf_Fs r$fan_innormalc Cst|\}}|dkr|}n"|dkr(|}n|dkr<||d}||}|dkrdt|t|ddnx|dkrt|jt|dW5QRXnH|d krtd |}t|| |W5QRXntd |dS) Nr%fan_outZfan_avgr truncated_normalg۶%?)rr&uniformzinvalid distribution ) rr$rrr r!Znormal_r ValueError) rscalemode distributionr%r'denomZvarianceboundr r r variance_scaling_cs$    r1cCst|ddddS)Nr%r()r-r.)r1)rr r r lecun_normal_|sr2)rrrr)rrrr)rr%r&) r rrZ torch.nn.initrrr#r$r1r2r r r r s #