classic vs. singular value method for ridge regression

[dj-gauthier]

I've notice that the ridge_regression estimator now has two options: "classic," which uses np.linalg.pinv, and "svd," which uses an explicit SVD method. Actually, the np.linalg.pinv function is based on singular value decomposition (see here, for example: https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse). I believe is also does some nice checking for small singular values. So not sure the "svd" approach is needed here or would even be desired given they already try to handle close to singular matrices. Thoughts anyone?

[wilsonrljr]

Hi @dj-gauthier! You have a good point. When you pushed your implementation, another contributor was working on the same method, so decided to move with both implementation and check later regarding the best approach. I'll take a look at this

[dj-gauthier]

Dear Wilsonrljr/, For the problem I was studying, both methods give the same result for the elements of \therta out to several decimal places. But pinv has built-in ways to handle ill -conditioned matrices. But the other approach uses the SVD routine, which probably has similar error checking and fixes. There is likely no difference. I would pick one or the other so a user is not confused or to make a choice without little information on which to make the choice. Up to you how to handle this. dj-Gauthier

…

On Tue, Sep 3, 2024 at 12:27 PM Wilson Rocha ***@***.***> wrote: Hi @dj-gauthier <https://github.com/dj-gauthier>! You have a good point. When you pushed your implementation, another contributor was working on the same method, so decided to move with both implementation and check later regarding the best approach. I'll take a look at this — Reply to this email directly, view it on GitHub <#137 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALEY24UYG3UUPQCMWVSQL2TZUXPOBAVCNFSM6AAAAABNNDSLPOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJTGQZTQMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>