U ʹj@sLddlZddlZddlmZddlmZGdddeZGdddeZdS)N)Samplerc@s*eZdZdZd ddZddZddZdS) OrderedDistributedSampleraSampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.DistributedDataParallel`. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it. .. note:: Dataset is assumed to be of constant size. Arguments: dataset: Dataset used for sampling. num_replicas (optional): Number of processes participating in distributed training. rank (optional): Rank of the current process within num_replicas. NcCs|dkr tstdt}|dkr@ts8tdt}||_||_||_tt t |jd|j|_ |j |j|_ dS)N,Requires distributed package to be availableg?)dist is_available RuntimeErrorget_world_sizeget_rankdataset num_replicasrankintmathceillen num_samples total_size)selfr r r r\/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/data/distributed_sampler.py__init__s z"OrderedDistributedSampler.__init__cCslttt|j}||d|jt|7}t||jks>t||j|j|j}t||jksdtt |SN) listrangerr rAssertionErrorr r riter)rindicesrrr__iter__%s z"OrderedDistributedSampler.__iter__cCs|jSr)rrrrr__len__2sz!OrderedDistributedSampler.__len__)NN)__name__ __module__ __qualname____doc__rrrrrrrrs  rc@s2eZdZdZdddZd d Zd d Zd dZdS)RepeatAugSampleraSampler that restricts data loading to a subset of the dataset for distributed, with repeated augmentation. It ensures that different each augmented version of a sample will be visible to a different process (GPU). Heavily based on torch.utils.data.DistributedSampler This sampler was taken from https://github.com/facebookresearch/deit/blob/0c4b8f60/samplers.py Used in Copyright (c) 2015-present, Facebook, Inc. NTrcCs|dkr tstdt}|dkr@ts8tdt}||_||_||_||_||_ d|_ t t t|j||j|_|j|j|_|p|}|rt t t|j||||_nt t t|j||_dS)Nrr)rrrrr r r r shuffle num_repeatsepochr rrrrrfloornum_selected_samples)rr r r r'r(Zselected_roundZselected_ratiorrrrAs,   zRepeatAugSampler.__init__cs"t}|jjr0tjtj|d}ntjdtjd}t j t rj st j tj}|tfddt|D}ntj|tj dd}|}jt|}|dkr||d|7}t|jkst|jjj}t|jkstt|djS)N) generatorr)startendcsg|]}t|jqSr)r r().0irrr rsz-RepeatAugSampler.__iter__..)Zrepeatsdim)torch GeneratorZ manual_seedr)r'Zrandpermrr Zarange isinstancer(float is_integerrrZtensorrZrepeat_interleaver tolistrrr r rrr+)rgrZ repeat_sizeZ padding_sizerrrres" "zRepeatAugSampler.__iter__cCs|jSr)r+rrrrrszRepeatAugSampler.__len__cCs ||_dSr)r))rr)rrr set_epochszRepeatAugSampler.set_epoch)NNTr%r&r)r r!r"r#rrrr:rrrrr$6s  $r$) rr3Ztorch.utils.datarZtorch.distributed distributedrrr$rrrrs   /