U ^{j @sxdZddlZddlmZmZddlmZmZddlZ ddl m Z ddl m Z mZmZmZmZmZddlmZdd lmZmZdd lmZmZdd lmZdd lmZmZdd l m!Z!m"Z"dddgZ#eedkrddl m$Z%n ddl m%Z%ddZ&d+ddZ'ddZ(d,dd Z)d!d"Z*Gd#d$d$eeeee ed%Z+Gd&dde+Z,Gd'dde+Z-Gd(d)d)e+Z.Gd*ddeee Z/dS)-zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)IntegralReal)svd) BaseEstimatorClassNamePrefixFeaturesOutMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context)ConvergenceWarning) check_arraycheck_consistent_length)Interval StrOptions)svd_flip) parse_version sp_version) FLOAT_DTYPEScheck_is_fitted PLSCanonical PLSRegressionPLSSVDz1.7)pinv)pinv2c Cst|ddd\}}}|jj}ddd}t|||t|j}t||k}|ddd|f}||d|}t t t ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssumZ transpose conjugatedot)ausZvhtfactorZcondZrankr0_/var/www/html/bwave/python/venv/lib/python3.8/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_old)s  r2Aư>Fc st|jjztfdd|jD}Wn,tk rV}ztd|W5d}~XYnXd}|dkrvt|t|} } t|D]} |dkrt | |} nt |j|t ||} | t t | | } t || } |dkrt | | }nt |j| t | j| }|r*|t t ||}t ||t ||}| |}t |||ksp|j ddkrvq|| }q~| d}||krt dt| ||fS) a?Return the first left and right singular vectors of X'Y. Provides an alternative to the svd(X'Y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3s&|]}tt|kr|VqdSN)r$anyabs).0colr'r0r1 Hsz;_get_first_singular_vectors_power_method..Y residual is constantNdBz$Maximum number of iterations reached)r$r&r!r'nextT StopIterationr2ranger*sqrtshapewarningswarnr)XYmodemax_itertolnorm_y_weightsZy_scoreeZ x_weights_oldZX_pinvZY_pinvi x_weightsZx_score y_weightsZx_weights_diffZn_iterr0r;r1(_get_first_singular_vectors_power_method;s8   "  rScCs@t|j|}t|dd\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'Y. Here the whole SVD is computed. FrNr)r$r*rBr)rIrJCU_Vtr0r0r1_get_first_singular_vectors_svdvsrYTcCs|jdd}||8}|jdd}||8}|rr|jddd}d||dk<||}|jddd}d||dk<||}n t|jd}t|jd}||||||fS)z{Center X, Y and scale if the scale parameter==True Returns ------- X, Y, x_mean, y_mean, x_std, y_std raxisr@)r[Zddofg?)ZmeanZstdr$ZonesrF)rIrJscaleZx_meanZy_meanZx_stdZy_stdr0r0r1_center_scale_xys     r^cCs2tt|}t||}||9}||9}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r$Zargmaxr8sign)r,vZbiggest_abs_val_idxr_r0r0r1 _svd_flip_1dsrac @seZdZUdZeeddddgdgeddhged d hged d hgeeddddgeed dddgdgdZe e d<e d$ddd d ddddddZ e ddddZd%ddZd&ddZd'ddZd(d d!Zd"d#ZdS))_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf r@Nleftclosedboolean regression canonicalr3r?rnipalsr n_componentsr]deflation_moderK algorithmrLrMcopy_parameter_constraintsrTr4r5)r]rlrKrmrLrMrnc Cs4||_||_||_||_||_||_||_||_dSr6)rkrlrKr]rmrLrMrn) selfrkr]rlrKrmrLrMrnr0r0r1__init__s z _PLS.__init__Zprefer_skip_nested_validationc Cs t|||j|tj|jdd}t|dtj|jdd}|jdkrTd|_|dd}nd|_|j d }|j d}|j d}|j }|j d kr|n t |||}||krt d |d |d |j dk|_|j}t|||j\} } |_|_|_|_t||f|_t||f|_t||f|_t||f|_t||f|_t||f|_g|_t| jj} t |D]} |j!dkr(tj"t#| d| kd d} d| dd| f<z$t$| | |j%|j&|j'|d\}}}WnPt(k r}z0t)|dkrt*+d| WY qW5d}~XYnX|j,|n|j!dkrBt-| | \}}t.||t/| |}|rdd}n t/||}t/| ||}t/|| t/||}| t0||8} |j dkrt/|| t/||}| t0||8} |j d krt/|| t/||}| t0||8} ||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<qft/|jt1t/|jj2|jdd|_3t/|jt1t/|jj2|jdd|_4t/|j3|jj2|_5|j5|jj2|_5|j|_6|j3j d|_7|S)Fit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. Y : array-like of shape (n_samples,) or (n_samples, n_targets) Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : object Fitted model. rr!rnZensure_min_samplesrJF input_namer!rn ensure_2dr@Trrg`n_components` upper bound is . Got instead. Reduce `n_components`.rhri rZr\N)rKrLrMrNr=z$Y residual is constant at iteration r)r)8r_validate_datar$float64rnrndim _predict_1dreshaperFrkrlmin ValueErrorZ_norm_y_weightsr^r]_x_mean_y_mean_x_std_y_stdZzeros x_weights_ y_weights_ _x_scores _y_scores x_loadings_ y_loadings_n_iter_r&r!r'rDrmallr8rSrKrLrMrCstrrGrHappendrYrar*outerrrB x_rotations_ y_rotations_coef_ intercept__n_features_out)rprIrJnpqrkrank_upper_boundrNZXkZYkZY_epskZYk_maskrQrRrrOx_scoresZy_ssy_scoresZ x_loadingsZ y_loadingsr0r0r1fits               z_PLS.fitcCst||j||tdd}||j8}||j}t||j}|dk rt|dd|td}|j dkrl| dd}||j 8}||j }t||j }||fS|S)a.Apply the dimension reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to transform. Y : array-like of shape (n_samples, n_targets), default=None Target vectors. copy : bool, default=True Whether to copy `X` and `Y`, or perform in-place normalization. Returns ------- x_scores, y_scores : array-like or tuple of array-like Return `x_scores` if `Y` is not given, `(x_scores, y_scores)` otherwise. Frnr!resetNrJ)rvrwrnr!r@rx)rr}rrrr$r*rrrrrrr)rprIrJrnrrr0r0r1 transformms(      z_PLS.transformcCst|t|dtd}t||jj}||j9}||j7}|dk r|t|dtd}t||j j}||j 9}||j 7}||fS|S)aeTransform data back to its original space. Parameters ---------- X : array-like of shape (n_samples, n_components) New data, where `n_samples` is the number of samples and `n_components` is the number of pls components. Y : array-like of shape (n_samples, n_components) New target, where `n_samples` is the number of samples and `n_components` is the number of pls components. Returns ------- X_reconstructed : ndarray of shape (n_samples, n_features) Return the reconstructed `X` data. Y_reconstructed : ndarray of shape (n_samples, n_targets) Return the reconstructed `X` target. Only returned when `Y` is given. Notes ----- This transformation will only be exact if `n_components=n_features`. rI)rvr!NrJ) rrrr$matmulrrBrrrrr)rprIrJZX_reconstructedZY_reconstructedr0r0r1inverse_transforms    z_PLS.inverse_transformcCsRt||j||tdd}||j8}||j}||jj|j}|jrN| S|S)aUPredict targets of given samples. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples. copy : bool, default=True Whether to copy `X` and `Y`, or perform in-place normalization. Returns ------- y_pred : ndarray of shape (n_samples,) or (n_samples, n_targets) Returns predicted values. Notes ----- This call requires the estimation of a matrix of shape `(n_features, n_targets)`, which may be an issue in high dimensional space. Fr) rr}rrrrrBrrZravel)rprIrnZYpredr0r0r1predicts   z _PLS.predictcCs|||||S)aLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples, n_targets), default=None Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : ndarray of shape (n_samples, n_components) Return `x_scores` if `Y` is not given, `(x_scores, y_scores)` otherwise. rrrprIyr0r0r1 fit_transformsz_PLS.fit_transformcCs dddS)NTF)Z poor_scoreZ requires_yr0)rpr0r0r1 _more_tagssz_PLS._more_tags)r)NT)N)T)N)__name__ __module__ __qualname____doc__rrrrrodict__annotations__rrqr rrrrrrr0r0r0r1rbs:        ' ,  rb) metaclasscs`eZdZUdZejZeed<dD]Ze eq"d dddddfd d Z fd d Z Z S)ra PLS regression. PLSRegression is also known as PLS2 or PLS1, depending on the number of targets. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_target, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, Y) PLSRegression() >>> Y_pred = pls2.predict(X) rorlrKrmrTr4r5r]rLrMrnc s tj||ddd|||ddS)Nrgr3rirjsuperrqrprkr]rLrMrn __class__r0r1rqcszPLSRegression.__init__cs"t|||j|_|j|_|S)rs)rrrZ x_scores_rZ y_scores_)rprIrJrr0r1rqszPLSRegression.fit)r) rrrrrbrorrparampoprqr __classcell__r0r0rr1rs b csVeZdZUdZejZeed<dD]Ze eq"d dddddd fd d Z Z S) ra Partial Least Squares transformer and regressor. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, Y) PLSCanonical() >>> X_c, Y_c = plsca.transform(X, Y) ro)rlrKrTrir4r5)r]rmrLrMrnc s tj||dd||||ddS)Nrhr3rjr)rprkr]rmrLrMrnrr0r1rqs zPLSCanonical.__init__)r rrrrrbrorrrrrqrr0r0rr1rs _ csTeZdZUdZejZeed<dD]Ze eq"d dddddfd d Z Z S) CCAat Canonical Correlation Analysis, also known as "Mode B" PLS. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, Y) CCA(n_components=1) >>> X_c, Y_c = cca.transform(X, Y) rorrTr4r5rc s tj||ddd|||ddS)Nrhr?rirjrrrr0r1rqhsz CCA.__init__)rrr0r0rr1r s W rc@speZdZUdZeeddddgdgdgdZeed<dd d d d d Z e d dddZ dddZ dddZ dS)raPartial Least Square SVD. This transformer simply performs a SVD on the cross-covariance matrix `X'Y`. It is able to project both the training data `X` and the targets `Y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> Y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, Y) >>> X_c, Y_c = pls.transform(X, Y) >>> X_c.shape, Y_c.shape ((4, 2), (4, 2)) r@Nrcrdrfrkr]rnrorT)r]rncCs||_||_||_dSr6r)rprkr]rnr0r0r1rqszPLSSVD.__init__rrc Cs"t|||j|tj|jdd}t|dtj|jdd}|jdkrL|dd}|j}t |j d|j d|j d}||krt d |d |d t |||j \}}|_|_|_|_t|j|}t|dd \}}}|d d d |f}|d |}t||\}}|j} ||_| |_|jj d|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. Y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. rrtrJFrur@rxrryrzr{rTN)rr}r$r~rnrrrrkrrFrr^r]rrrrr*rBrrrrr) rprIrJrkrrUrVr-rXVr0r0r1rsJ    z PLSSVD.fitcCst||j|tjdd}||j|j}t||j}|dk rt|ddtjd}|j dkrh| dd}||j |j }t||j }||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. Y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. F)r!rNrJ)rvrwr!r@rx)rr}r$r~rrr*rrrrrrr)rprIrJZXrrZYrrr0r0r1rs  zPLSSVD.transformcCs|||||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. rrr0r0r1rszPLSSVD.fit_transform)r)N)N)rrrrrrrorrrqr rrrr0r0r0r1rws D 6 )r3r4r5F)T)0rrGabcrrnumbersrrnumpyr$Z scipy.linalgrbaserr r r r r exceptionsrutilsrrZutils._param_validationrrZ utils.extmathrZ utils.fixesrrZutils.validationrr__all__rrr2rSrYr^rarbrrrrr0r0r0r1sR        ;  Tk