Error running scDblFinder
malonzm1 opened this issue · comments
malonzm1 commented
Hi,
When running scDBlFinder I run into the error: Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102
The code is the following:
from glob import glob
import os
import scanpy as sc
import anndata2ri
import logging
import rpy2.rinterface_lib.callbacks as rcb
import rpy2.robjects as ro
rcb.logger.setLevel(logging.ERROR)
ro.pandas2ri.activate()
anndata2ri.activate()
%load_ext rpy2.ipython
%%R
library(Seurat)
library(scater)
library(scDblFinder)
library(BiocParallel)
gse = 'GSE134649'
infolder = '/scratch/cs/pan-autoimmune/data/counts/scRNAseq/%s/table'%gse
folders = glob('%s/GSM*/'%infolder)
gsms = []
for folder in folders:
gsm = folder.split('/')[-2]
gsms.append(gsm)
adatas1 = [sc.read_10x_mtx('%s/%s'%(infolder,gsm),cache=True, var_names='gene_symbols') for gsm in gsms]
%%R -i data_mat -o doublet_score -o doublet_class
set.seed(123)
sce = scDblFinder(
SingleCellExperiment(
list(counts=data_mat),
)
)
and the error trace is:
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102
---------------------------------------------------------------------------
RRuntimeError Traceback (most recent call last)
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:385, in RMagics.eval(self, code)
383 try:
384 # Need the newline in case the last line in code is a comment.
--> 385 value, visible = ro.r("withVisible({%s\n})" % code)
386 except (ri.embedded.RRuntimeError, ValueError) as exception:
387 # Otherwise next return seems to have copy of error.
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/__init__.py:459, in R.__call__(self, string)
458 p = rinterface.parse(string)
--> 459 res = self.eval(p)
460 return conversion.get_conversion().rpy2py(res)
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/functions.py:208, in SignatureTranslatedFunction.__call__(self, *args, **kwargs)
207 kwargs[r_k] = v
--> 208 return (super(SignatureTranslatedFunction, self)
209 .__call__(*args, **kwargs))
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/functions.py:131, in Function.__call__(self, *args, **kwargs)
130 new_kwargs[k] = cv.py2rpy(v)
--> 131 res = super(Function, self).__call__(*new_args, **new_kwargs)
132 res = cv.rpy2py(res)
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/rinterface_lib/conversion.py:45, in _cdata_res_to_rinterface.<locals>._(*args, **kwargs)
44 def _(*args, **kwargs):
---> 45 cdata = function(*args, **kwargs)
46 # TODO: test cdata is of the expected CType
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/rinterface.py:817, in SexpClosure.__call__(self, *args, **kwargs)
816 if error_occured[0]:
--> 817 raise embedded.RRuntimeError(_rinterface._geterrmessage())
818 return res
RRuntimeError: Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102
During handling of the above exception, another exception occurred:
RInterpreterError Traceback (most recent call last)
Cell In[8], line 1
----> 1 get_ipython().run_cell_magic('R', '-i data_mat -o doublet_score -o doublet_class', '\nset.seed(123)\nsce = scDblFinder(\n SingleCellExperiment(\n list(counts=data_mat),\n ) \n)\ndoublet_score = sce$scDblFinder.score\ndoublet_class = sce$scDblFinder.class\n')
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/IPython/core/interactiveshell.py:2493, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
2491 with self.builtin_trap:
2492 args = (magic_arg_s, cell)
-> 2493 result = fn(*args, **kwargs)
2495 # The code below prevents the output from being displayed
2496 # when using magics with decorator @output_can_be_silenced
2497 # when the last Python token in the expression is a ';'.
2498 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:943, in RMagics.R(self, line, cell, local_ns)
941 if not e.stdout.endswith(e.err):
942 print(e.err)
--> 943 raise e
944 finally:
945 if self.device in DEVICES_STATIC:
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:923, in RMagics.R(self, line, cell, local_ns)
921 return_output = False
922 else:
--> 923 text_result, result, visible = self.eval(code)
924 text_output += text_result
925 if visible:
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:389, in RMagics.eval(self, code)
386 except (ri.embedded.RRuntimeError, ValueError) as exception:
387 # Otherwise next return seems to have copy of error.
388 warning_or_other_msg = self.flush()
--> 389 raise RInterpreterError(code, str(exception),
390 warning_or_other_msg)
391 text_output = self.flush()
392 return text_output, value, visible[0]
RInterpreterError: Failed to parse and evaluate line '\nset.seed(123)\nsce = scDblFinder(\n SingleCellExperiment(\n list(counts=data_mat),\n ) \n)\ndoublet_score = sce$scDblFinder.score\ndoublet_class = sce$scDblFinder.class\n'.
R error message: "Error in h(simpleError(msg, call)) : \n error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102"
Please advise.
Thanks and good day.
Pierre-Luc commented
Hi,
I don't really understand how exactly your code passes the data to R, but
- my guess would be that somewhere in there your matrix is converted to dense format and that gets too large.
- it looks like you have multiple samples merged together and then pass the whole thing to scDblFinder without telling it about the samples. If that's the case, that's very bad. You should either pass the samples one by one, or provide the
samples
argument.
Pierre-Luc commented
Will close this issue unless you have an update.
malonzm1 commented
Hi,
Thanks. I filtered the data before running scDblFinder and the error stopped.