U ʹj@sddlZddlmZddlmZddlmZmZmZm Z m Z ddl m Z ddl mZdddZeZeZGd d d Zefe ed d d Ze jeeefdddZdS)N) OrderedDictwraps)CallableDictListOptionalType)_State__composable_api_state_keycCs|dttS)N_)struuiduuid4)stringre/var/www/html/bwave/python/venv/lib/python3.8/site-packages/torch/distributed/_composable/contract.pygenerate_state_key src@s eZdZdS) RegistryItemN)__name__ __module__ __qualname__rrrrrsr state_clscstfdd}|S)aJ Decorate a function as a composable distributed API, where the first argument of the function must be an :class:`nn.Module` instance. The decorator verifies that the wrapped function does not modify parameter, buffer or sub-module fully-qualified names (FQN). When a function ``func`` is decorated by ``@contract()``, a ``.state(module: nn.Module)`` method will be installed to the decorated function. Then you can retrieve and modify the state on a module by calling ``func.state(module)``. Example:: >>> # xdoctest: +SKIP >>> import torch.nn as nn >>> >>> class MyModel(nn.Module): >>> def __init__(self): >>> super().__init__() >>> self.l1 = nn.Linear(10, 10) >>> self.l2 = nn.Linear(10, 10) >>> >>> def forward(self, x): >>> return self.l2(self.l1(x)) >>> >>> @contract() >>> def my_feature(module: nn.Module) -> nn.Module: >>> my_feature.state(module).some_state = "any value" >>> return module >>> >>> model = MyModel() >>> my_feature(model.l1) >>> assert my_feature.state(model.l1).some_state == "any value" >>> my_feature(model.l2) >>> model(torch.randn(2, 10)).sum().backward() csJttjttjdfdd }tjttdfdd }||_|S)Nmodulereturncst}|jt|}t|ts&tdt}|jt|}t|tsLtd|kr^j|ksttdjd|||jt t| }t|j dd}t|j dd} |f||} | dkr|} t| } t| j dd} t| j dd} t| t js&tdt| tttttdd d }|t|t| d |t|t| d |t| t| d | S)Nz+Distributed composable API states corruptedz-Distributed composable API registry corruptedzOEach distinct composable distributed API can only be applied to a module once. z3 has already been applied to the following module. F)Zremove_duplicatezPOutput of composable distributed APIs must be either None or nn.Module, but got ) orig_fqnsnew_fqns check_keycSst||kr dSt|t|}}||}||}t|s>t|rXt|d|d|nt|d|d|dS)NzVComposable distributed API implementations cannot modify FQNs. Only in original FQNs: z, Only in new FQNs: z[Composable distributed API implementations cannot modify the order of FQNs. Original FQNs: z New FQNs: )setlen RuntimeError)rrrZ orig_fqn_setZ new_fqn_setZ orig_onlyZnew_onlyrrr check_fqn}sz;contract..inner..wrapper..check_fqnzCheck parameters, zCheck buffer, zCheck modules, )r__dict__ setdefault STATE_KEY isinstancedictAssertionError REGISTRY_KEYrrZnamed_parametersZ named_buffersZ named_modulesnnModuletyperr listkeys)rargskwargsZdefault_all_stateZ all_statedefault_registryregistryZorig_named_paramsZorig_named_buffersZorig_named_modulesupdatedZnew_named_paramsZnew_named_buffersZnew_named_modulesr#)funcrrrwrapperBs              z(contract..inner..wrappercs|jtiS)N)r$r%r&get)rr5rr get_states z*contract..inner..get_state)rr+r,rr state)r5r6r9rr8rinner@s  jzcontract..innerr)rr;rrrcontracts&xr<rcCs2t|td}|dkr*t}t|t||S|SdS)zz Get an ``OrderedDict`` of composable APIs that have been applied to the ``module``, indexed by the API name. N)getattrr*rsetattr)rr3r2rrr _get_registrys   r?)r )r collectionsr functoolsrtypingrrrrr Ztorch.nnr+Z#torch.distributed._composable_stater rr&r*rr<r,r r?rrrrs     #