U ʹjd@sddlmZddlmZmZmZmZddlZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZddlmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,ddl-Z-ddl.Z.ddd d d d d ddddddddddddddddddd d!d"d#d$gZ/Gd%ddedd&d'gZ0Gd(ddedd&d'gZ1e0e%e(d)Z2e0e(e!d*Z3e0e%e&d)Z4e0e"e(d)Z5e0ej6ej7d+d,ej6ej7d-d)Z8e0ej6ej7d-ej6ej7d-d)Z9e0e"e&d)Z:e0e'e#d)Z;e0e'e$d)Ze0ej j?ed)Z@e0e ej j?d)ZAe0eed)ZBe0e+ed)ZCdLd/dZDe0ej6ejEd0d1d2e)d)ZFe0ej6ejEd0d1d2e*d)ZGe0ej6ejHd-ed)ZIe0ej6ejHd-ed)ZJe0eed)ZKdMd3d ZLe0ej6ed4d5ejEd0d1d6ed)ZMe0ej6ed4d5ejEd0d1d6ed)ZNe0ej6ejHd-ej6ejHd-d)ZOe0ej6ejPd-dd)ZQdNd7d!ZRdOd8d"ZSee0ej jTdd9d:d;ZUee0ZVdd?ZXeeee,ee fZYeYeYd@dAdBZZeed@dCdDZ[eVeVdEdFd$Z\e0dGdHdIZ]ee0dGdJdKZ^dS)P) namedtuple)OptionalAnyUnionTypeN) FakeQuantizeFakeQuantizeBasedefault_fake_quantdefault_dynamic_fake_quant%default_per_channel_weight_fake_quantdefault_weight_fake_quantdefault_fused_act_fake_quantdefault_fused_wt_fake_quantFusedMovingAvgObsFakeQuantize'default_fused_per_channel_wt_fake_quantdefault_embedding_fake_quant!default_embedding_fake_quant_4bit(fused_wt_fake_quant_range_neg_127_to_1274fused_per_channel_wt_fake_quant_range_neg_127_to_127)_PartialWrapperMinMaxObserverHistogramObserverMovingAverageMinMaxObserver NoopObserverPlaceholderObserverReuseInputObserverdefault_debug_observerdefault_dynamic_quant_observerdefault_float_qparams_observer#default_float_qparams_observer_4bitdefault_observer#default_per_channel_weight_observerdefault_placeholder_observerdefault_weight_observer$weight_observer_range_neg_127_to_1270per_channel_weight_observer_range_neg_127_to_127default_reuse_input_observer ObserverBaseQConfigQConfigDynamicdefault_qconfigdefault_debug_qconfigdefault_per_channel_qconfigdefault_dynamic_qconfigfloat16_dynamic_qconfigfloat16_static_qconfigper_channel_dynamic_qconfig!float_qparams_weight_only_qconfig&float_qparams_weight_only_qconfig_4bitdefault_quint8_weight_qconfigdefault_qat_qconfigdefault_dynamic_qat_qconfigdefault_weight_only_qconfigdefault_activation_only_qconfigdefault_qat_qconfig_v2default_reuse_input_qconfig!default_symmetric_qnnpack_qconfig-default_per_channel_symmetric_qnnpack_qconfig%default_symmetric_qnnpack_qat_qconfig1default_per_channel_symmetric_qnnpack_qat_qconfigdefault_embedding_qat_qconfig"default_embedding_qat_qconfig_4bitget_default_qconfigget_default_qat_qconfigget_default_qconfig_dictget_default_qat_qconfig_dict QConfigAnyqconfig_equalscs eZdZdZfddZZS)r)a Describes how to quantize a layer or a part of the network by providing settings (observer classes) for activations and weights respectively. Note that QConfig needs to contain observer **classes** (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization preparation function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with `with_args` method (that behaves like functools.partial):: my_qconfig = QConfig( activation=MinMaxObserver.with_args(dtype=torch.qint8), weight=default_observer.with_args(dtype=torch.qint8)) cs0t|tjst|tjr tdt|||S)NzQConfig received observer instance, please pass observer class instead. Use MyObserver.with_args(x=1) to override arguments to constructor if needed) isinstancennModule ValueErrorsuper__new__cls activationweight __class__\/var/www/html/bwave/python/venv/lib/python3.8/site-packages/torch/ao/quantization/qconfig.pyrLeszQConfig.__new__)__name__ __module__ __qualname____doc__rL __classcell__rSrSrQrTr)RsrOrPcs.eZdZdZejjejjffdd ZZS)r*a Describes how to dynamically quantize a layer or a part of the network by providing settings (observer classes) for weights. It's like QConfig, but for dynamic quantization. Note that QConfigDynamic needs to contain observer **classes** (like MinMaxObserver) or a callable that returns instances on invocation, not the concrete observer instances themselves. Quantization function will instantiate observers multiple times for each of the layers. Observer classes have usually reasonable default arguments, but they can be overwritten with `with_args` method (that behaves like functools.partial):: my_qconfig = QConfigDynamic(weight=default_observer.with_args(dtype=torch.qint8)) cs.t|tjrtdtdt|||S)NzQConfigDynamic received observer instance, please pass observer class instead. Use MyObserver.with_args(x=1) to override arguments to constructor if neededzTQConfigDynamic is going to be deprecated in PyTorch 1.12, please use QConfig instead)rGrHrIrJwarningswarnrKrLrMrQrSrTrL}s  zQConfigDynamic.__new__) rUrVrWrXtorchrHIdentityrLrYrSrSrQrTr*msrOrP)rPrOT)dtypeZ is_dynamic)r_x86cCsddddg}||kr.tdt|d||dkr|dkrTttjdd td }q|dkrrttjd d td }q|dkrtj st d ttjd d td }q|dkrttjdd td }qt }ntd t|d|S)z Returns the default PTQ qconfig for the specified backend. Args: * `backend` (str): a string representing the target backend. Currently supports `x86` (default), `fbgemm`, `qnnpack` and `onednn`. Return: qconfig fbgemmr`qnnpackonednn backend: ' not supported. backend must be one of rT) reduce_ranger^FzDefault qconfig of oneDNN backend with reduce_range of false may have accuracy issues on CPU without Vector Neural Network Instruction support.Version number: zB in get_default_qconfig is not supported. Version number must be 0) AssertionErrorstrr)r with_argsr"r$r\cpuZ_is_cpu_support_vnnirZr[r+backendversionZsupported_backendsqconfigrSrSrTrAsB        Fg0?)r_rfepscCsddddg}||kr.tdt|d||dkr|dkrZttjtddd d td }nn|dkr~ttjtddd d td }nJ|dkrttjtddd td }n(|dkrttjtddd d td }nt}n|dkrn|dkrtt jtddd d t d }nt|dkrtt jtddd d t d }nN|dkrBtt jtddd t d }n*|dkrhtt jtddd d t d }nt }ntdt|d|S)aL Returns the default QAT qconfig for the specified backend. Args: * `backend` (str): a string representing the target backend. Currently supports `x86` (default), `fbgemm`, `qnnpack` and `onednn`. * `version`: version, for backwards compatibility. Can be `None` or `1`. Return: qconfig rar`rbrcrdrerT)observer quant_min quant_maxrfr^F)rrrsrtrrgzJin get_default_qat_qconfig is not supported. Version number must be 0 or 1) rhrir)rrjrr r r5rrrr9rlrSrSrTrB9s       i)rrrsrtr_rfrpcCstdtjj||S)Nztorch.ao.quantization.get_default_qconfig_dict is deprecated and will be removed in a future version. Please use torch.ao.quantization.get_default_qconfig_mapping instead.)rZr[r\ao quantizationZget_default_qconfig_mappingto_dictrmrnrSrSrTrCscCstdtjj||S)Nztorch.ao.quantization.get_default_qat_qconfig_dict is deprecated and will be removed in a future version. Please use torch.ao.quantization.get_default_qat_qconfig_mapping instead.)rZr[r\rvrwZget_default_qat_qconfig_mappingrxryrSrSrTrDs)romodreturncCsl|dkr dSt|tjjtjjtjjf}|rh|jdkr:dS|}t|tjjj tjjj f}|rht ddS)z0 Verifies that this `qconfig` is valid. NzGPer channel weight observer is not supported yet for ConvTranspose{n}d.) rGr\rHZConvTranspose1dZConvTranspose2dZConvTranspose3drPrvrwZPerChannelMinMaxObserverZ%MovingAveragePerChannelMinMaxObserverrh)rorzZis_conv_transpose_modZexample_observerZis_per_channelrSrSrT_assert_valid_qconfigs  r|ztorch.ao.quantization.qconfig)romoduler{csTdks|dks|jdkr|Sfddfdd}||j}||j}t||S)aPThis is a helper function for use in quantization prepare that updates a qconfig so that the constructors stored in the qconfig will create observers on the same device that 'module' is on. This is intended to be used when the qconfigs are propagated to each module in order to avoid potential device alignment issues. Args: qconfig: QConfig with obs constructors stored in activation and weight module: module which the qconfig is related to Return: qconfig: configured so that obs constructors set to construct on the same device as module Nr^csfttjjstddDddDB}t|dkrNtt |nd}|dkr^dSd|iS)NcSsh|] }|jqSrSdevice.0prSrSrT szd_add_module_to_qconfig_obs_ctr..get_factory_kwargs_based_on_module_device..cSsh|] }|jqSrSr~rrSrSrTrsrr) rGr\rHrIrh parametersbufferslennextiter)Zdevicesr)r}rSrT)get_factory_kwargs_based_on_module_devices zQ_add_module_to_qconfig_obs_ctr..get_factory_kwargs_based_on_module_devicecsTz |jdd}||jdWStk r8|YStk rN|YSXdS)N)Zfactory_kwargs)rjZwith_callable_argsAttributeError TypeError)Zoriginal_constructorcheck)rrSrT1configure_constructor_to_put_obs_on_module_devices zY_add_module_to_qconfig_obs_ctr..configure_constructor_to_put_obs_on_module_device)_fieldsrOrPr))ror}rrOrPrS)rr}rT_add_module_to_qconfig_obs_ctrs   r obs_or_fq1 obs_or_fq2cCs&t|trt|trt||S||kSN)rGr_partial_wrapper_equalsrrSrSrT_obs_or_fq_ctr_equalss rcCst|jj}t|jj}d}d|krZd|krZ|oDt|d|d}|d|d|od||k}|jj|jjko|jj|jjko|S)z< Return whether the two partial wrappers are equal, Trr)copyrkeywordsrpopfuncargs)rrZobs_or_fq1_keywordsZobs_or_fq2_keywordsZkeywords_equalrSrSrTrs   r)q1q2cCst|dks|dkr||kS|dk r(|dk s,tz&t|j|j}t|j|j}|oP|WStk rn||kYSXdS)zD Returns `True` if `q1` equals `q2`, and `False` otherwise. N)rhrrOrPr)rrZactivation_sameZ weight_samerSrSrTrFs rocCs:dd}|}t|tr.t|dr.||jS||SdS)z Return whether the observer for activations defined in the given QConfig is memoryless. This means a MovingAverage observer with averaging constant equal to 1. cSst|do|jdkS)Naveraging_constantr)hasattrr)rrrSrSrT_is_memoryless%sz1_activation_is_memoryless.._is_memorylessactivation_post_processN)rOrGrrr)rorZactrSrSrT_activation_is_memoryless s  rcCs$|dk o"t|to"t|tSr)rGrOrrPrrrSrSrT_is_reuse_input_qconfig-s   r)r`r)r`r)r`r)r`r)_ collectionsrtypingrrrrr\Ztorch.nnrHZ#torch.ao.quantization.fake_quantizerrr r r r r rrrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(rZr__all__r)r*r+r,r-r.rjZfloat16r/r0r1r2r3r5r6r]r7r8r9r:rAZqint8r;r<Zfloat32r?r@r4rBr=r>Z!_default_fp32_placeholder_qconfigZquint8Z#_default_quint8_placeholder_qconfigrCrDrIr|rErVrZ"_ObserverOrFakeQuantizeConstructorrrrFrrrSrSrSrTs0  @T#    G    S         *