U ʹj<)@s0dZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z ddlZddlZddlZddlmZmZdd lmZmZdd lmZmZdd lmZdd lmZdd lm Z e!e"Z#ddZ$ddZ%GdddZ&d'ddZ'dddddddddddddddddddeeddddddddej(e)ddddddf$e eefe e*e e*e*fe e*e*e*ffe*e+e+e,e-e*e+e e-e e e,e,fe e e,e,fe,e,e,e e,e,e,e e-e*e*e-e e,dfe e,dfe*e+e e,e e-e e*e e e+e+ej.ej)e+e+e+e-e+d 'd!d"Z/Gd#d$d$ej0j1j2Z3Gd%d&d&e4Z5dS)(a Loader Factory, Fast Collate, CUDA Prefetcher Prefetcher and Fast Collate inspired by NVIDIA APEX example at https://github.com/NVIDIA/apex/commit/d5e2bb4bdeedd27b1dfaf5bb2b24d6c000dee9be#diff-cf86c282ff7fba81fad27a559379d5bf Hacked together by / Copyright 2019, Ross Wightman N)suppress)partial)repeat)CallableOptionalTupleUnion)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)IterableImageDataset ImageDataset)OrderedDistributedSamplerRepeatAugSampler) RandomErasing)FastCollateMixup)create_transformc Cs,t|dtstt|}t|ddtrt|dd}||}tj|tjd}tj|f|dddjtjd}t |D]l}t||d|kstt |D]F}||d||||<||||t ||d|7<qq||fSt|ddt j rtj dd|Dtjd}t||ks8ttj|f|ddjtjd}t |D]$}||t ||d7<q`||fSt|ddtjrtj dd|Dtjd}t||ksttj|f|ddjtjd}t |D]}||||dq||fSds(tdS) zd A fast collation function optimized for uint8 images (np array or torch) and int64 targets (labels)r)dtyper cSsg|] }|dqSr .0brrO/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/data/loader.py /sz fast_collate..cSsg|] }|dqSrrrrrrr6sFN) isinstancetupleAssertionErrorlentorchZzerosZint64shapeZuint8rangeZ from_numpynpZndarraytensorZTensorZcopy_)batch batch_sizeZinner_tuple_sizeZflattened_batch_sizetargetsr#ijrrr fast_collates8$  .  "  r)cCsnt|ttfstt||}nLt||krVt|}|f|}t d|dnt||ksjt d|S)Nz@Pretrained mean/std different shape than model, using avg value .z-normalization stats must match image channels) rrlistrrr"meanitem_loggerwarningr)xnZx_meanrrr adapt_to_chs@s  r2c @steZdZeededejdddddf dd Zd d Z d d Z e ddZ e ddZ e ddZejddZdS)PrefetchLoadercudaFconstr rc Cst||}t||}d|ddf} ||_||_|r6tj}||_tjdd|D||d| |_tjdd|D||d| |_ |dkrt || | | |d|_ nd|_ tj o|jdk|_dS) Nr cSsg|] }|dqSrrr0rrrrgsz+PrefetchLoader.__init__..)devicercSsg|] }|dqSr8rr:rrrrisr6)Z probabilitymode max_countZ num_splitsr;r5)r2loaderr;rZfloat16 img_dtyper#viewr,stdrrandom_erasingr5Z is_availabletypeis_cuda) selfr>r,rAchannelsr;r?fp16re_probre_modere_count re_num_splitsZnormalization_shaperrr__init__Ns@      zPrefetchLoader.__init__c csd}|jr&tj}ttjj|d}nd}t}|jD]\}}|V|j|j dd}|j|j dd}||j  |j  |j}|jdk r||}W5QRX|s||fVnd}|dk rtj||}|}q4||fVdS)NT)stream)r;Z non_blockingF)rDrr5ZStreamrrMrr>tor;r?Zsub_r,Zdiv_rArBZcurrent_streamZ wait_stream)rEfirstrMZstream_contextZ next_inputZ next_targetinputtargetrrr__iter__vs*   zPrefetchLoader.__iter__cCs t|jSN)rr>rErrr__len__szPrefetchLoader.__len__cCs|jjSrS)r>samplerrTrrrrVszPrefetchLoader.samplercCs|jjSrS)r>datasetrTrrrrWszPrefetchLoader.datasetcCs t|jjtr|jjjSdSdS)NFrr> collate_fnr mixup_enabledrTrrrrZs zPrefetchLoader.mixup_enabledcCst|jjtr||jj_dSrSrX)rEr0rrrrZsN)__name__ __module__ __qualname__r r rr;float32rLrRrUpropertyrVrWrZsetterrrrrr3Ls* (   r3allcCs|tjj}|j|kstt|trR||}t |t |t j |dn&|dks^t|dkrxt j |j ddS)Nl)rapartra) rutilsdataZget_worker_infoidrrrrandomseedZ manual_seedr")Z worker_idworker_seedingZ worker_inforgrrr _worker_inits     riFr6r7g?g?Zbilinearr5T.)'rW input_sizer% is_trainingno_augrHrIrJre_splittrain_crop_modescaleratiohflipvflip color_jittercolor_jitter_probgrayscale_probgaussian_blur_prob auto_augmentnum_aug_repeatsnum_aug_splits interpolationr,rA num_workers distributedcrop_pct crop_modecrop_border_pixelsrY pin_memoryrGr?r;use_prefetcheruse_multi_epochs_loaderpersistent_workersrhtf_preprocessingc'.Csd}'|r|pd}'t|||| | | | | |||||||||||||||'|&|"|dkd|_t|trh|j|dd}(|rt|tjjjs|r|rt ||d}(qtjjj |}(qt |}(n|dkst d|dkr|"rtn tjjjj}tjjj})|#rt})t|t|tjjj o|(dko|||(|||tt|%d|$d }*z|)|f|*}+Wn:tk rz},z|*d |)|f|*}+W5d},~,XYnX|"r|r|s|nd }-t|+|||d|!|| |-|||'d }+|+S) a Args: dataset: The image dataset to load. input_size: Target input size (channels, height, width) tuple or size scalar. batch_size: Number of samples in a batch. is_training: Return training (random) transforms. no_aug: Disable augmentation for training (useful for debug). re_prob: Random erasing probability. re_mode: Random erasing fill mode. re_count: Number of random erasing regions. re_split: Control split of random erasing across batch size. scale: Random resize scale range (crop area, < 1.0 => zoom in). ratio: Random aspect ratio range (crop ratio for RRC, ratio adjustment factor for RKR). hflip: Horizontal flip probability. vflip: Vertical flip probability. color_jitter: Random color jitter component factors (brightness, contrast, saturation, hue). Scalar is applied as (scalar,) * 3 (no hue). color_jitter_prob: Apply color jitter with this probability if not None (for SimlCLR-like aug grayscale_prob: Probability of converting image to grayscale (for SimCLR-like aug). gaussian_blur_prob: Probability of applying gaussian blur (for SimCLR-like aug). auto_augment: Auto augment configuration string (see auto_augment.py). num_aug_repeats: Enable special sampler to repeat same augmentation across distributed GPUs. num_aug_splits: Enable mode where augmentations can be split across the batch. interpolation: Image interpolation mode. mean: Image normalization mean. std: Image normalization standard deviation. num_workers: Num worker processes per DataLoader. distributed: Enable dataloading for distributed training. crop_pct: Inference crop percentage (output size / resize size). crop_mode: Inference crop mode. One of ['squash', 'border', 'center']. Defaults to 'center' when None. crop_border_pixels: Inference crop border of specified # pixels around edge of original image. collate_fn: Override default collate_fn. pin_memory: Pin memory for device transfer. fp16: Deprecated argument for half-precision input dtype. Use img_dtype. img_dtype: Data type for input image. device: Device to transfer inputs and targets to. use_prefetcher: Use efficient pre-fetcher to load samples onto device. use_multi_epochs_loader: persistent_workers: Enable persistent worker processes. worker_seeding: Control worker random seeding at init. tf_preprocessing: Use TF 1.0 inference preprocessing for testing model ports. Returns: DataLoader r)rkrlrnrorprqrrrsrtrurvrwrzr,rAr}r~rrHrIrJrKrrZseparate)r{N)Z num_repeatszORepeatAugment not currently supported in non-distributed or IterableDataset use)rh) r%shuffler{rVrYrZ drop_lastZworker_init_fnrrr6) r,rArFr;rGr?rHrIrJrK)rZ transformrr Zset_loader_cfgrrcrdZIterableDatasetrr|ZDistributedSamplerrrr)Z dataloaderZdefault_collate DataLoaderMultiEpochsDataLoaderdictrri TypeErrorpopr3).rWrjr%rkrlrHrIrJrmrnrorprqrrrsrtrurvrwrxryrzr,rAr{r|r}r~rrYrrGr?r;rrrrhrrKrV loader_classZ loader_argsr>eZprefetch_re_probrrr create_loadersW        rcs,eZdZfddZddZddZZS)rcsNtj||d|_|jdkr,t|j|_n t|j|_d|_t|_dS)NFT)superrLZ_DataLoader__initialized batch_sampler_RepeatSamplerrVrRiterator)rEargskwargs __class__rrrLts  zMultiEpochsDataLoader.__init__cCs |jdkrt|jSt|jjSrS)rrrVrTrrrrU~szMultiEpochsDataLoader.__len__ccs"tt|D]}t|jVq dSrS)r!rnextr)rEr'rrrrRszMultiEpochsDataLoader.__iter__)r[r\r]rLrUrR __classcell__rrrrrrs rc@s eZdZdZddZddZdS)rzH Sampler that repeats forever. Args: sampler (Sampler) cCs ||_dSrS)rV)rErVrrrrLsz_RepeatSampler.__init__ccst|jEdHqdSrS)iterrVrTrrrrRsz_RepeatSampler.__iter__N)r[r\r]__doc__rLrRrrrrrsr)ra)6rloggingrf contextlibr functoolsr itertoolsrtypingrrrrrZtorch.utils.datanumpyr" constantsr r rWr r Zdistributed_samplerrrrBrZmixuprZtransforms_factoryr getLoggerr[r.r)r2r3rir^r;intboolfloatstrrrrcrdrrobjectrrrrrs       # a     6