core dumped bad_alloc in seqan

Question

core dumped bad_alloc in seqan

vodkatad opened this issue 5 months ago · comments

After managing to run the test via docker we've tried with our data, but without success:

docker run -v /mnt/cold1/snaketree/prj/snakegatk/local/share/data/second_shipment/:/data/ fred2/optitype -i PRJ0003S1_SA_L001_R1_001.fastq.gz PRJ0003S1_SA_L001_R2_001.fastq.gz -d -o /data/
/data/seqan-src/include/seqan/basic/basic_exception.h:363 FAILED!  (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
  0          [0x8917fd]  /usr/local/bin/razers3()
  1    [0x7ffa997556b6]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)
  2    [0x7ffa997546a9]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c6a9)
  3    [0x7ffa99755005]  __gxx_personality_v0 + 0x2e5
  4    [0x7ffa98f96f83]  /lib/x86_64-linux-gnu/libgcc_s.so.1(+0xff83)
  5    [0x7ffa98f972eb]  _Unwind_RaiseException + 0xfb
  6    [0x7ffa9975590c]  __cxa_throw + 0x5c
  7    [0x7ffa99755ebc]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8debc)
  8          [0x62c455]  seqan::Size<seqan::String<unsigned int, seqan::Alloc<void> > >::Type seqan::_Resize_String<seqan::Tag<seqan::TagExact_> >::resize_<seqan::String<unsigned int, seqan::Alloc<void> > >(seqan::String<unsigned int, seqan::Alloc<void> >&, seqan::Size<seqan::String<unsigned int, seqan::Alloc<void> > >::Type) + 0x65
  9          [0x803c77]  void seqan::_patternInit<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, double, void>(seqan::Pattern<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, seqan::Pigeonhole<void> >&, double) + 0x507
 10          [0x803ff4]  void seqan::_applyFilterOptions<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, void, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> > >(seqan::Pattern<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, seqan::Pigeonhole<void> >&, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> > const&) + 0x144
 11          [0x5b988a]  /usr/local/bin/razers3()
 12    [0x7ffa991a8cbf]  GOMP_parallel + 0x3f
 13          [0x8111e6]  void seqan::initializeThreadLocalStoragesSingle<seqan::String<seqan::ThreadLocalStorage<seqan::MapSingleReads<seqan::String<seqan::MatchRecord<unsigned long>, seqan::Alloc<void> >, seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >, seqan::Finder<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::Pigeonhole<void> >, seqan::Pattern<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, seqan::Pigeonhole<void> >, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >, seqan::RazerSMode<seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_> > >, seqan::Alloc<void> >, seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >, seqan::String<unsigned int, seqan::Alloc<void> >, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape> const, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> > >(seqan::String<seqan::ThreadLocalStorage<seqan::MapSingleReads<seqan::String<seqan::MatchRecord<unsigned long>, seqan::Alloc<void> >, seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >, seqan::Finder<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::Pigeonhole<void> >, seqan::Pattern<seqan::Index<seqan::StringSet<seqan::Segment<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::InfixSegment> const, seqan::Owner<seqan::Tag<seqan::Default_> > >, seqan::IndexQGram<seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::Tag<seqan::OpenAddressing_> > >, seqan::Pigeonhole<void> >, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >, seqan::RazerSMode<seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_> > >, seqan::Alloc<void> >&, seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::String<unsigned int, seqan::Alloc<void> > const&, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape> const&, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >&) + 0x116
 14          [0x811f2e]  int seqan::_mapSingleReadsParallel<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >, seqan::RazerSSpec<false, false>, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape>, seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_, seqan::Pigeonhole<void> >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >&, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >&, seqan::Shape<seqan::SimpleType<unsigned char, seqan::Dna_>, seqan::OneGappedShape> const&, seqan::RazerSMode<seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_> const&, seqan::Pigeonhole<void>) + 0x14e
 15          [0x83d610]  int seqan::_mapReads<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >, seqan::RazerSSpec<false, false>, seqan::RazerSMode<seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_> >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >&, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >&, seqan::RazerSMode<seqan::RazerSGlobal, seqan::RazerSGapped, seqan::RazerSErrors, seqan::NMatchesNone_> const&) + 0x520
 16          [0x84aff9]  int seqan::_mapReads<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >, seqan::RazerSSpec<false, false> >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::String<seqan::String<unsigned short, seqan::Alloc<void> >, seqan::Alloc<void> >&, seqan::RazerSCoreOptions<seqan::RazerSSpec<false, false> >&) + 0x79
 17          [0x88ec11]  int mapReads<seqan::RazerSSpec<false, false> >(seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x1371
 18          [0x572e65]  main + 0x845

Aborted (core dumped)
[E::hts_open_format] Failed to open file /data/2024_02_23_16_26_26/2024_02_23_16_26_26_2.bam
Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 313, in <module>
    pos2, read_details2 = ht.pysam_to_hdf(bam_paths[1])
  File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf
    sam = pysam.AlignmentFile(samfile, sam_or_bam)
  File "pysam/libcalignmentfile.pyx", line 728, in pysam.libcalignmentfile.AlignmentFile.__cinit__
  File "pysam/libcalignmentfile.pyx", line 918, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file `/data/2024_02_23_16_26_26/2024_02_23_16_26_26_2.bam`: No such file or directory

RAM should not be an issue. Can anyone give us pointers?
Thanks,
E.

Hupingdong · Answer 1 · Sun May 26 2024 21:41:51 GMT+0800 (China Standard Time)

Hi, have you solver the error? I met the same issue with the conda installed optitype

Elena Grassi · Answer 2 · Wed May 29 2024 21:26:19 GMT+0800 (China Standard Time)

We ended up finding that indeed it was a RAM issue - the aligner razers3 had not enough ram (250Gb) to run with both R1 and R2 reads from 30x WGS. We ended up using R1 and R2 separately, apparently (but I am not sure) optitype does not use the information from read pairs so it should be fine (even if a single allele ended up having different calls with R1/R2, but they are very similar alleles...).

Hupingdong · Answer 3 · Thu May 30 2024 13:25:59 GMT+0800 (China Standard Time)

Thank you for your reply. When I run it on the example data, razers3 used more 100G RAM. and then I got that error. Do you think bwa-mem can be used instead of razers3 for the alignament?

Elena Grassi · Answer 4 · Thu May 30 2024 21:24:31 GMT+0800 (China Standard Time)

We ended up using bwa with these parameters:

bwa mem -M -t $(nproc) $HLA_REF $R1 2> err.bwa-hla.$OUT | \
        samtools view -F4 -C -T $HLA_REF -@10

Hupingdong · Answer 5 · Thu May 30 2024 22:32:28 GMT+0800 (China Standard Time)

OK, thank you so much!

b-schubert · Answer 6 · Mon Jun 17 2024 18:27:25 GMT+0800 (China Standard Time)

HLA-Locus prefiltering helps. And yes Optitype uses the paired information when constructing the binary hit matrix