recurze / LassoByDescent

Solving Lasso Regression by descent algorithms

Problem

You want to solve Lasso regression but know only Gradient Descent and can't compute gradient of lasso regularizer.

Solution

• Proximal Gradient (PG) descent
• Accelerated Proximal Gradient (APG) descent (Nesterov's)
• Accelerated Proximal Gradient with Restart (APGRestart) descent
• Coordinate descent (CD)

Optimizations

• Running delta: instead of computing fresh delta at every iteration, compute incrementally by difference.
• matrix.dot(sparse_vector): filter out zero rows of the vector