Hadisalman / L0Learn

Efficient Algorithms for L0 Regularized Learning

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

L0Learn: Fast Best Subset Selection Build Status

Hussein Hazimeh and Rahul Mazumder

Massachusetts Institute of Technology

Downloads from Rstudio:

Introduction

L0Learn is a highly efficient framework for solving L0-regularized learning problems. It can (approximately) solve the following three problems, where the empirical loss is penalized by combinations of the L0, L1, and L2 norms:

We support both regression (using squared error loss) and classification (using logistic or squared hinge loss). Optimization is done using coordinate descent and local combinatorial search over a grid of regularization parameter(s) values. Many computational tricks and heuristics are used to speed up the algorithms and improve the solution quality. These heuristics include warm starts, active set convergence, correlation screening, greedy cycling order, and efficient methods for updating the residuals through exploiting sparsity and problem dimensions. Moreover, we employed a new computationally efficient method for dynamically selecting the regularization parameter λ in the path. We describe the details of the algorithms in our paper: Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms (arXiv link).

The toolkit is implemented in C++11 and can often run faster than popular sparse learning toolkits (see our experiments in the paper above). We also provide an easy-to-use R interface; see the section below for installation and usage of the R package.

R Package Installation and Usage

The latest version of L0Learn (v1.2.0) can be installed from Github:

library(devtools)
install_github("hazimehh/L0Learn")

Alternatively, it can be installed from CRAN:

install.packages("L0Learn", repos = "http://cran.rstudio.com")

For a tutorial, please refer to L0Learn's Vignette. For a detailed description of the API, check the Reference Manual.

L0Learn's changelog can be accessed from here. For users who have been using L0Learn before July 1, 2018, please check this Wiki page for more information on the changes introduced in the new CRAN version.

Citing L0Learn

If you find L0Learn useful in your research, please consider citing the following paper:

@article{hazimeh_mazumder_2019,
   title={Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms},
   author={Hazimeh, Hussein and Mazumder, Rahul},
   journal={Operations Research},
   url={https://arxiv.org/abs/1803.01454},
   keywords = {Statistics - Computation, Mathematics - Optimization and Control, Statistics - Machine Learning},
   year={2019}
}

About

Efficient Algorithms for L0 Regularized Learning

License:Other


Languages

Language:C++ 75.9%Language:R 20.6%Language:M4 3.4%Language:C 0.1%Language:Shell 0.0%