U ʹj6@sjddlZddlmZmZmZddlZddlmZmZddl m Z dgZ edddZ Gd ddee ZdS) N)TypeVarOptionalIterator)SamplerDatasetDistributedSamplerT_coT) covariantc @sdeZdZdZdeeeeeeeeddddZe e d d d Z ed d d Z eddddZ dS)ra' Sampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.DistributedDataParallel`. In such a case, each process can pass a :class:`~torch.utils.data.DistributedSampler` instance as a :class:`~torch.utils.data.DataLoader` sampler, and load a subset of the original dataset that is exclusive to it. .. note:: Dataset is assumed to be of constant size and that any instance of it always returns the same elements in the same order. Args: dataset: Dataset used for sampling. num_replicas (int, optional): Number of processes participating in distributed training. By default, :attr:`world_size` is retrieved from the current distributed group. rank (int, optional): Rank of the current process within :attr:`num_replicas`. By default, :attr:`rank` is retrieved from the current distributed group. shuffle (bool, optional): If ``True`` (default), sampler will shuffle the indices. seed (int, optional): random seed used to shuffle the sampler if :attr:`shuffle=True`. This number should be identical across all processes in the distributed group. Default: ``0``. drop_last (bool, optional): if ``True``, then the sampler will drop the tail of the data to make it evenly divisible across the number of replicas. If ``False``, the sampler will add extra indices to make the data evenly divisible across the replicas. Default: ``False``. .. warning:: In distributed mode, calling the :meth:`set_epoch` method at the beginning of each epoch **before** creating the :class:`DataLoader` iterator is necessary to make shuffling work properly across multiple epochs. Otherwise, the same ordering will be always used. Example:: >>> # xdoctest: +SKIP >>> sampler = DistributedSampler(dataset) if is_distributed else None >>> loader = DataLoader(dataset, shuffle=(sampler is None), ... sampler=sampler) >>> for epoch in range(start_epoch, n_epochs): ... if is_distributed: ... sampler.set_epoch(epoch) ... train(loader) NTrF)dataset num_replicasrankshuffleseed drop_lastreturncCs|dkr tstdt}|dkr@ts8tdt}||ksP|dkrjtd|d|dd||_||_||_d|_ ||_ |j rt |j|jdkrt t |j|j|j|_nt t |j|j|_|j|j|_||_||_dS)Nz,Requires distributed package to be availablerz Invalid rank z%, rank should be in the interval [0, r])distZ is_available RuntimeErrorZget_world_sizeZget_rank ValueErrorr r r epochrlenmathceil num_samples total_sizerr)selfr r r rrrr[/var/www/html/bwave/python/venv/lib/python3.8/site-packages/torch/utils/data/distributed.py__init__>s2zDistributedSampler.__init__)rcCs|jr:t}||j|jtjt|j|d }nt t t|j}|j s|j t|}|t|kr~||d|7}q||t|t|d|7}n|d|j }t||j kst||j|j |j}t||jkstt|S)N) generator)rtorch GeneratorZ manual_seedrrZrandpermrr tolistlistrangerrrrAssertionErrorr r riter)rgindicesZ padding_sizerrr__iter__`s $zDistributedSampler.__iter__cCs|jS)N)r)rrrr__len__{szDistributedSampler.__len__)rrcCs ||_dS)a) Sets the epoch for this sampler. When :attr:`shuffle=True`, this ensures all replicas use a different random ordering for each epoch. Otherwise, the next iteration of this sampler will yield the same ordering. Args: epoch (int): Epoch number. N)r)rrrrr set_epoch~s zDistributedSampler.set_epoch)NNTrF)__name__ __module__ __qualname____doc__rrintboolrrr r*r+r,rrrrr s 0  ")rtypingrrrr!rrZtorch.distributed distributedr__all__r rrrrrs