LUP decomposition for singular matrices
AtheMathmo opened this issue · comments
There is some discussion around this in #94 .
Currently we only support LUP decomposition for non-singular matrices. This is an unnecessary restriction which we should aim to remove in the future.
The existing implementation uses partial pivoting but in order to decompose a non-invertible matrix safely we should use full pivoting. There is a brief explanation in the Eigen docs.
This is a low priority issue and should not be tackled until the outstanding on LUP decomposition is complete.
Is anyone working on this? I'd like to take a stab at it, if that's alright.
@Prismatica: not to my knowledge! Would be cool if you could take a look :)
Actually, I noticed the description here is not quite right. We do not want to replace partial pivoting with full pivoting, but we want to provide both. Whereas the pivoting for partial pivoting is very cheap compared to the floating point operations, the pivoting involved in full pivoting is much more expensive. Most matrices are invertible and relatively well-conditioned, in which case partial pivoting is almost always sufficient. However, for perfectly singular matrices or badly conditioned matrices, partial pivoting breaks down, whereas full pivoting also handles this case well.
So, to summarize: for most applications partial pivoting is sufficient and the fastest option. But in the case of badly conditioned matrices, or if you want to decompose a singular matrix, then full pivoting is the way to go.
We currently have PartialPivLu. The task would be to additionally implement FullPivLu, which would correspond to full pivoting. IIRC the floating point operations are the same - only the pivoting is different. Whereas partial pivoting only looks for the largest element in the current column at each iteration, full pivoting considers the entire lower right submatrix. A good resource is the monograph Matrix Computations by Golub and Van Loan.
Let me know if you have any questions or suggestions :)
Oh, and there's usually some nice people who keep a PDF of such monographs lying around in their publicly accessible, cough googleable cough web directories!
Thanks for the details! I was planning on (very loosely) modelling the interface off of the Eigen FullPivLU (as well as the existing rulinalg PartialPivLU, of course). I think this will mostly mean having the same sort of interface as the PartialPivLU, but adding the column-interchange permutation matrix and a few extra functions like rank and maybe nullSpace if I'm feeling ambitious.
I actually own a copy of Golub and Van Loan, but during my poor student days, I may have taken those nice people up on their implied offer. :)
@Prismatica: I was just thinking that I should have mentioned that rank is highly desirable! Great plan :)
Also, when you're done it would be great if you could add your implementation to the table in the decomposition module docs!
I've been (slowly) working on a revamp of the decompositions in rulinalg, but I've been very busy lately and haven't had much time, so I'm very happy to see someone else taking an interest in working on decompositions.
In a few weeks I should have more time to work on the other decompositions (QR, Eigendecomposition, SVD, ...). If you finish the FullPivLu implementation and want more work, let me know if you're interested in helping out here!
I just want to chime in to thank you both for the discussion here. FullPivLU would be a huge contribution @Prismatica - please don't hesitate to post here or elsewhere if you need any help getting this working.
It's also a huge relief to have both you and @Andlon tackling the decomposition stuff. It is one of the weaker areas in the library because I lack the expertise. The recent improvements have made a huge difference and I'm excited to see even more positive changes :)
It is one of the weaker areas in the library because I lack the expertise.
It also represents a tremendous amount of work, and the fact that you've singlehandedly implemented them all by yourself is really impressive. It's a lot easier to improve on something than it is to build from scratch. The fact that we have these foundations to build on and improve really makes the task much more manageable.