Parallelisation
niekverw opened this issue · comments
What would be the best way to parallelise it, or incorporate parallelisation? Are there any plans to parallelise CorShrink?
As a method, CorShrink learns its shrinkage strengths from all the correlations in the correlation matrix and the is not readily parallelizable. For large data matrices however, one can learn subset the correlation matrix randomly and then run parallelizations to estimate shrinkage separately and combine them, with the hope that learning of shrinkage rates would be close to the non-parallel approach. We are working on it and are yet to compare it with the non-parallel version in terms of performance.
Also, in case this query stems from the fact that the computational speed is slow for a large matrix, another option you may consider is to increase the tol argument (the default for which is 1e-06) - that should ensure faster convergence and results are often robust to it.
Great, indeed I encountered low speeds using a large matrix; I will try the tol argument. Many thanks.