Segmentation Fault for Ultra-High-Dimensional Data

[Dario Strbenac]

Let's say there is a CyTOF data set with 45 million cells and 60 markers.

library(BiocSingular)
test <- matrix(rnorm(45000000*60), ncol = 60)
test <- runSVD(test, k = 5) 

 *** caught segfault ***
address 0x7f0ae16da618, cause 'memory not mapped'
Error in La.svd(x, nu, nv) : 
  long vectors not supported yet: ../../src/include/Rinlinedfuns.h:537

Can it fail fast instead (or perhaps handle long vectors)?

[Aaron Lun]

That's really a question for base::svd, not BiocSingular. I note that you have a very small number of actual dimensions, so you could ask BiocSingular to compute the SVD via a cross-product instead - see the ?fold argument.

[Dario Strbenac]

Touché.