Unicycler 0.4.8.0 and Spades 3.12.0+galaxy1 fail at usegalaxy.org

[Jennifer Hillman-Jackson]

Workaround for end-users:

1. Reset the metadata on the fastq read inputs for paired-end datasets

• Step 1: Merge the paired reads with Fastq Interlacer
• Step 2: Split the paired reads with Fastq De-Interlacer
• Step 3: Input the "still paired" reads to the assembly tool
• Note: Both tools have always required matched paired inputs (downstream from Trimmomatic or via other methods). That is currently not enough for tool success. Interlace/De-interlace the existing match paired data as additional steps. The result will not appear to not differ from the original inputs, however underlying metadata is reset and allows the tools to function correctly. Finding out why this new behavior presents is part of resolving the tool errors when run at usegalaxy.org.

2. Reset the metadata on the fastq read inputs for single-end datasets

• pending tests

3. Alternatively, try a different public Galaxy server until resolved. usegalaxy.eu and usegalaxy.org.au are good choices.

Issue: Both tools fail at usegalaxy.org (only). Suspect some underlying dependency issue or a fa.fai index issue during runtime. Unicyler incorporates Spades, and Spades also fails when run directly.

End-user reports of errors: Several. Review galaxy-bugs mailing list reports of submitted tool errors for these two tools.

Test histories: Both use tutorial inputs/methods from GTN https://training.galaxyproject.org/training-material/topics/assembly/tutorials/unicycler-assembly/tutorial.html

• usegalaxy.org: https://usegalaxy.org/u/jen-galaxyproject/h/test-history-unicycler
• usegalaxy.eu: https://usegalaxy.eu/u/jenj/h/test-unicycler

Example of error for Unicycler: (abbreviated, the stderr is always very long since it includes a full trace, this is the end):

=== Stack Trace ===
/cvmfs/main.galaxyproject.org/deps/_conda/envs/__unicycler@0.4.8/bin/spades-core(+0x14159a) [0x55eac3e1e59a]
/cvmfs/main.galaxyproject.org/deps/_conda/envs/__unicycler@0.4.8/bin/spades-core(+0x435eef) [0x55eac4112eef]
/cvmfs/main.galaxyproject.org/deps/_conda/envs/__unicycler@0.4.8/bin/spades-core(+0x4370a6) [0x55eac41140a6]
/cvmfs/main.galaxyproject.org/deps/_conda/envs/__unicycler@0.4.8/bin/../lib/libgomp.so.1(+0x146d5) [0x7fc243ff16d5]
/lib64/libpthread.so.0(+0x7dd5) [0x7fc24387cdd5]
/lib64/libc.so.6(clone+0x6d) [0x7fc2435a602d]
spades-core: /opt/conda/conda-bld/spades_1566422302166/work/src/common/utils/kmer_mph/kmer_index_builder.hpp:205: size_t utils::KMerDiskCounter<Seq, traits>::MergeKMers(const string&, const string&) [with Seq = RuntimeSeq<128>; traits = utils::kmer_index_traits<RuntimeSeq<128> >; size_t = long unsigned int; std::__cxx11::string = std::__cxx11::basic_string<char>]: Assertion `std::is_sorted(beg, end, adt::array_less<typename Seq::DataType>())' failed.
== Error ==  system call for: "['/cvmfs/main.galaxyproject.org/deps/_conda/envs/__unicycler@0.4.8/bin/spades-core', '/pylon5/mc48nsp/xcgalaxy/main/staging/31998120/working/spades_assembly/assembly/K77/configs/config.info']" finished abnormally, err code: -6
In case you have troubles running SPAdes, you can write to [e-mail]
or report an issue on our GitHub repository github.com/ablab/spades
Please provide us with params.txt and spades.log files from the output directory.

Insert size mean: 249.3 bp
Insert size stdev: 10.8 bp


-----------------------------------------------------------------------------
job info:
Remote job server indicated a problem running or monitoring this job.
-----------------------------------------------------------------------------

ping @Delphine-L @NickSto @natefoo

[Jennifer Hillman-Jackson]

Tools likely impacted. Not all runs, just those that use uploaded input fastq that hasn't undergone any other pre-processing steps. Which "pre-processing" is not clearly defined. Interlace/De-interlace is one of them. Trimmomatic is probably another (Jen: double-check that and add to to the workaround). Coping datasets to a new history is definitely not one of those.

[Jennifer Hillman-Jackson]

Reruns started, same test history, tagged "dec18".

Status: 12/21 am -- queued at bridges

usegalaxy.org: https://usegalaxy.org/u/jen-galaxyproject/h/test-history-unicycler

[Jennifer Hillman-Jackson]

Fixed, closing, thanks all