RAM usage in SCISSOR_TCGA

[FANGLIANGZI MENG]

Hi, thank you for your work. When I was performing SCISSOR_TCGA step, I encountered an issue requiring over three thousand GB RAM. I wanted to inquire about how much memory resources you used when performing SCISSOR_TCGA?

[Gregor Sturm]

I was working on a system where memory wasn't that much of an issue, so I can't say exactly, but I think you should be good with ~30 GB per process. Since there will be several hundred scissor processes be lauched at the same time, 3000GB doesn't sound too unrealistic.

If you have memory issues, I recommend adding

memory = 30.GB

to the process configuration here:

luca/conf/downstream_analyses.config

Lines 165 to 173 in 5ffb0a4

 withName: ".*scissor:SCISSOR_TCGA" { 
     // errorStrategy = 'ignore' 
     ext.ignore_error = true 
     container = "${baseDir}/containers/2020-pircher-scissor2_2022-04-13.sif" 
     publishDir = [ 
         path: { "${params.outdir}/scissor_tcga/scissor_by_sample/${id}" }, 
         mode: params.publish_dir_mode 
     ] 
 } 

Then nextflow should take care of not launching more processes simultaneously than there's memory available.

[FANGLIANGZI MENG]

Thank you for your response. I found that the problem might be due to my incorrect usage of the data.