U ʹj) @sddlZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZddlZddlmZddlmZddd d d d d dddg Zd(ejedddZd)e ejeeejdddZd*ejeeeddd Zd+ejeeeddd ZdZd,e e eefee e feeddd Zd-ejeedddZd.ejeedd d Zd/e e eejfeeee ed#ffeee e ejfd$d%dZ d0d&dZ!d'd Z"dS)1N) defaultdict)chain)AnyCallableDictIteratorTupleTypeUnion)nn) checkpointmodel_parameters named_apply named_modulesnamed_modules_with_paramsadapt_input_convgroup_with_matcher group_modulesgroup_parametersflatten_modulescheckpoint_seqFmodelZ exclude_headcCs*|rdd|DddS|SdS)NcSsg|]}|qSr).0prrV/var/www/html/bwave/python/venv/lib/python3.8/site-packages/timm/models/_manipulate.py sz$model_parameters..) parametersrrrrr sT)fnmodule depth_first include_rootreturncCsf|s|r|||d|D]0\}}|r6d||fn|}t||||ddq|rb|rb|||d|S)N)r"name.T)r!r"r&r#r$)named_childrenjoinr)r!r"r&r#r$ child_name child_modulerrrrs  r"r&r#r$ccsf|s|r||fV|D]4\}}|r4d||fn|}t|||ddEdHq|rb|rb||fVdSNr'Tr,)r(r)rr"r&r#r$r*r+rrrr(s ccsr|jr|s|r||fV|D]4\}}|r:d||fn|}t|||ddEdHq |jrn|rn|rn||fVdSr-) _parametersr(r)rr.rrrr8s )i) named_objects group_matcher return_valuesreversecsjttrg}tD]h\}\}}|dkr0qt|ttfrj|D]$}|t|d|f|dfg7}qBq|t||fdfg7}q|fdd} tt} |D]"\} } | | |  |r| n| qtt} d}t t dd| D]:} |dks| dt dkr|d7}| || | q|rf|r6tdi}| D]\}}|D]}|||<qNqB|S| S) Nrc stttfrbD]D\}}}||}|r|||f}ttttt d|SqtdfS|}t|t j j s~|fSt|SdS)Ninf) isinstancelisttuplematchgroupsmapfloatr from_iterablefilter collectionsabcIterable)r&Zmatch_fnprefixsuffixrpartsordr1rr _get_grouping`s   z)group_with_matcher.._get_groupingcSs|dk S)Nr)xrrrwz$group_with_matcher..z-reverse mapping only sensible for name output)r6dict enumerateitemsr8r7recompilerappendsortedr>keysMATCH_PREV_GROUPextendAssertionError)r0r1r2r3compiledZ group_ordinal group_nameZmspecZsspecrHgroupingkvZlayer_id_to_paramlidZparam_to_layer_idZlmnrrGrrKs8 $  )r"r2r3cCst||||dSN)r2r3)rZnamed_parametersr"r1r2r3rrrrs cCstt||||dSr_)rrr`rrrrs r4 sequential.)rdepthrB module_typesccst|t}t|tr6|dkr.tjtjtjf}ntjf}|D]v\}}|rzt||rzt||d|rh|fn||dEdHq:|r||f}||fVq:|rd ||g}||fVq:dS)N containerr4)rBrcr') r6r8strr SequentialZ ModuleListZ ModuleDictrr(r))rrbrBrcZprefix_is_tupler&r"rrrrs&      c Csdd}t|tjjr|}|r,t|}t|ttfsBt|}t |}|rV|d8}d}t d||D]0} t | |d|d}t || ||||d}qf|r||dt |d||S|S)aA helper function for checkpointing sequential models. Sequential models execute a list of modules/functions in order (sequentially). Therefore, we can divide such a sequence into segments and checkpoint each segment. All segments except run in :func:`torch.no_grad` manner, i.e., not storing the intermediate activations. The inputs of each checkpointed segment will be saved for re-running the segment in the backward pass. See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works. .. warning:: Checkpointing currently only supports :func:`torch.autograd.backward` and only if its `inputs` argument is not passed. :func:`torch.autograd.grad` is not supported. .. warning: At least one of the inputs needs to have :code:`requires_grad=True` if grads are needed for model inputs, otherwise the checkpointed part of the model won't have gradients. Args: functions: A :class:`torch.nn.Sequential` or the list of modules or functions to run sequentially. x: A Tensor that is input to :attr:`functions` every: checkpoint every-n functions (default: 1) flatten (bool): flatten nn.Sequential of nn.Sequentials skip_last (bool): skip checkpointing the last function in the sequence if True preserve_rng_state (bool, optional, default=True): Omit stashing and restoring the RNG state during each checkpoint. Returns: Output of running :attr:`functions` sequentially on :attr:`*inputs` Example: >>> model = nn.Sequential(...) >>> input_var = checkpoint_seq(model, input_var, every=2) csfdd}|S)Ncs$tdD]}||}q|S)Nr4)range)Z_xjend functionsstartrrforwardsz5checkpoint_seq..run_function..forwardr)rlrjrkrmrrir run_functionsz$checkpoint_seq..run_functionr4rIr)preserve_rng_state) r6torchr rfchildrenrr=r8r7lenrgminr ) rkrJZeveryflattenZ skip_lastrornZnum_checkpointedrjrlrrrrs", cCs|j}|}|j\}}}}|dkrx|dkrh|jdddksBt|||dd||}|jddd}q|jddd}nh|dkr|dkrtdnNtt |d}| d|ddddd|ddddf}|dt|9}| |}|S) Nr4rF)dimZkeepdimTz*Weight format not supported by conversion.) Zdtyper<shaperWZreshapesumNotImplementedErrorintmathceilrepeatto)Zin_chansZ conv_weightZ conv_typeOIJKr~rrrrs" , )F)r TF)r TF)r TF)FF)FF)FF)r4r ra)r4FFT)#collections.abcr?r|rPr itertoolsrtypingrrrrrr r rpr Ztorch.utils.checkpointr __all__Moduleboolr rrerrrUrrrr{rrrrrrrs  $         @   ! F