_csv.Error: field larger than field limit (131072)
s4251484 opened this issue · comments
Hi @metageni and the team
i encountered this issue where superfocus successfully generated the .m8 file but failed to generate the excel files. FYI, I am analysing the contigs with superfocus diamond 98.
Verbose output:
.......
478 queries aligned.
diamond v0.9.32.133 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org
#CPU threads: 24
Loading subject IDs... [2.603s]
Scoring parameters: (Matrix=custom Lambda=0.267 K=0.041 Penalties=11/1)
DB sequences = 5234971
DB sequences used = 9623
DB letters = 1880813436
#Target sequences to report alignments for: 25
Generating output... [12.417s]
[2020-05-20 11:25:15,035 - INFO] Parsing Alignments
Traceback (most recent call last):
File "/RDS/Q1034/conda/superfocus/bin/superfocus", line 12, in
sys.exit(main())
File "/RDS/Q1034/conda/superfocus/lib/python3.8/site-packages/superfocus_app/superfocus.py", line 339, in main
results, binning_reads = parse_alignments(alignment_name, results, normalise_output, len(query_files),
File "/RDS/Q1034/conda/superfocus/lib/python3.8/site-packages/superfocus_app/do_alignment.py", line 150, in parse_alignments
for row in alignment_reader:
_csv.Error: field larger than field limit (131072)
understand that its a limit with the csv - how could I possibly fix it - seems like it is going to generate some huge excel files.
@s4251484 Thanks for reporting this.
One thing to be aware - unrelated to what you reported. Make sure you predict the ORFs for the contigs before running SUPER-FOCUS. Otherwise, the tool will assign the best hit which will not represent all the functions on the contig.
I will need to take a deeper look into your problem and I will get back to you.
@s4251484
The solution seems to be simple. I will push it into master and will you be able to re-run it?
Best
https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072
hi!
yea i modified the do.alignment.py accordingly.
yea it is a fix!
meanwhile I shall work on my ORF! legit advice! thanks!
@s4251484
The solution seems to be simple. I will push it into master and will you be able to re-run it?Best
https://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072