A-Riemannian-SubGradient-Solver-for-Least-Absolute-Distance-Problem
This repository contains implementations (C++/Python/Matlab) of a Riemannian SubGraident (RSG) solver proposed in the NeurIPS 2019 paper "A Linearly Convergent Method for Non-Smooth Non-Convex Optimization on the Grassmannian with Applications to Robust Subspace and Dictionary Learning" for solving the least absolute distance problem with the following formulation:
where X is the data matrix with shape D x L, and B is the variable whose columns are constrained to be orthonormal and ideally lie in a dual space which is orthogonal to the subspace spaned by the samples (columns of X).
In above formulations, we denote
- D is the number of features, or number of dimensions in the feature space
- L is the number of samples
- c is the number of dual directions we aim to compute, and must satisfy 0 < c < D
In particular, when c is 1, the original problem becomes the following one on the sphere:
In this case, we are only interested in finding a single dual direction that is orthogonal to the samples as much as possible.
C++ Users
The C++ version is implemented as a header file for ease of reuse, which is located in ./c++/RSG.h. Note that the implementation relies on two external C++ scientific computing libraries, i.e., Eigen and libigl, to help with relevant numerical tasks. Since both of them are header-only libraries, we include them in the directory so that users do not need to spend extra effort downloading/installing them.
The code has been tested with g++ 4.2.1. For testing, run
cd c++/
g++ demo.cpp -std=c++11 -o demo
./demo
After running the demo, simple numerical results should be available in the standard output.
Python Users
The Python version is implemented as a module, which is located in ./python/RSG.py.
The code has been tested with Python 3.7. For testing, run
cd python/
python demo.py
Matlab Users
The Matlab version is implemented as multiple function files located in ./matlab/.
The code has been tested with MATLAB_R2018b. For testing, open Matlab, change the working directory, and run demo.m.
Synthetic Experiments
For interested users, we also include the code for synthetic experiments used in the paper, e.g., how do the various parameters affect the performance of the algorithm. See ./synthetic_experiments for details.
Citation
If you find the code or results useful, please cite the following paper:
@inproceedings{zhu2019linearly,
title={A linearly convergent method for non-smooth non-convex optimization on the grassmannian with applications to robust subspace and dictionary learning},
author={Zhu, Zhihui and Ding, Tianyu and Robinson, Daniel and Tsakiris, Manolis and Vidal, Ren{\'e}},
booktitle={Advances in Neural Information Processing Systems},
pages={9437--9447},
year={2019}
}