what does lightning do with intercept?
mathurinm opened this issue · comments
We comment saying that lightning always fits an intercept:
https://github.com/benchopt/benchmark_lasso/blob/main/solvers/lightning.py#L10
and indeed it has an attribute intercept_ after fitting.
but if I fit with a constant y, I don't get the solution I'd expect with an intercept:
import numpy as np
from numpy.linalg import norm
from lightning.regression import CDRegressor
from sklearn.linear_model import Lasso
np.random.seed(0)
X = np.random.randn(20, 50)
y = 12 * np.ones(X.shape[0])
alpha = norm(X.T @ y, ord=np.inf) / 10
clf = CDRegressor(C=0.5, alpha=alpha, penalty='l1',
tol=1-30, random_state=0, permute=False,
shrinking=False).fit(X, y)
las = Lasso(fit_intercept=True, alpha=alpha/len(y), max_iter=100_000,
tol=1e-10).fit(X, y)
print("intercepts:")
print(las.intercept_) # 12
print(clf.intercept_) # not 12, 0
print("coef norm:")
print(norm(las.coef_)) # is 0 as expected
print(norm(clf.coef_)) # not zerooutput:
intercepts:
12.0 # sklearn
[0.]
coef norm:
0.0 # sklearn
8.548795604837427
yes it could be ported there at some point, but I'm putting it here for now because we have a comment in our own codebase saying that lightning fits an intercept, so I'm asking where this comes from.
I'm into this right now for PR #94. I get the same behavior as @mathurinm.
As far as I understand from their code, the intercept is just set to zero in any case. Therefore, it is not really handled (confirming what both of you already mentioned).
Concerning the comment we have in our code saying that lightning always fits an intercept, I will remove (or correct) it in #94, if you both agree.