select_by_key segfault with 3.3

Question

select_by_key segfault with 3.3

dputhier opened this issue 7 years ago · comments

It seems that we are getting back to some instability of select_by_key when using select_by_key under OSX.

    67262  cd ../libgtftk/
    67263  git pull
    67264  cp -Rf  ./* ../gtftk/gtftk/src/libgtftk/
    67265  cd ..
    67266  cd gtftk
    67267  make install
    67268  gtftk select_by_key -i gtftk/data/simple/simple.gtf -k feature -v transcript

    $ gtftk select_by_key -i gtftk/data/simple/simple.gtf -k feature -v transcript
    chr1    gtftk   transcript  125 138 .   +   .   gene_id "G0001"; transcript_id "G0001T002";
    chr1    gtftk   transcript  125 138 .   +   .   gene_id "G0001"; transcript_id "G0001T001";
    chr1    gtftk   transcript  180 189 .   +   .   gene_id "G0002"; transcript_id "G0002T001";
    chr1    gtftk   transcript  50  61  .   -   .   gene_id "G0003"; transcript_id "G0003T001";
    chr1    gtftk   transcript  65  76  .   +   .   gene_id "G0004"; transcript_id "G0004T002";
    chr1    gtftk   transcript  65  76  .   +   .   gene_id "G0004"; transcript_id "G0004T001";
    chr1    gtftk   transcript  33  47  .   -   .   gene_id "G0005"; transcript_id "G0005T001";
    chr1    gtftk   transcript  22  35  .   -   .   gene_id "G0006"; transcript_id "G0006T001";
    chr1    gtftk   transcript  28  35  .   -   .   gene_id "G0006"; transcript_id "G0006T002";
    chr1    gtftk   transcript  107 116 .   +   .   gene_id "G0007"; transcript_id "G0007T001";
    chr1    gtftk   transcript  107 116 .   +   .   gene_id "G0007"; transcript_id "G0007T002";
    chr1    gtftk   transcript  210 222 .   -   .   gene_id "G0008"; transcript_id "G0008T001";
    chr1    gtftk   transcript  3   14  .   -   .   gene_id "G0009"; transcript_id "G0009T002";
    chr1    gtftk   transcript  3   14  .   -   .   gene_id "G0009"; transcript_id "G0009T001";
    chr1    gtftk   transcript  176 186 .   +   .   gene_id "G0010"; transcript_id "G0010T001";
    Segmentation fault: 11

Denis Puthier · Answer 1 · Sat Nov 11 2017 04:28:36 GMT+0800 (China Standard Time)

Not observed under Linux... OSX specific thus...

fafa13 · Answer 2 · Mon Nov 13 2017 18:33:30 GMT+0800 (China Standard Time)

The result is OK so the select_by_key function do the job and exit properly. What do you do after the call of select_by_key ?

fafa13 · Answer 3 · Mon Nov 13 2017 19:16:30 GMT+0800 (China Standard Time)

I suspect that the problem is in the free_gtf_data function, when freeing the indexes related to the gtf_data. I have disabled this in the last push. Can you test it and tell me what happens ?

Denis Puthier · Answer 4 · Tue Nov 14 2017 16:17:00 GMT+0800 (China Standard Time)

This is clearly better. You are digging in the right direction.

Denis Puthier · Answer 5 · Tue Nov 14 2017 16:34:34 GMT+0800 (China Standard Time)

You should even probably pass all tests if you fix this issue in free_gtf_data call.

fafa13 · Answer 6 · Tue Nov 14 2017 19:55:53 GMT+0800 (China Standard Time)

I have pushed a new version that free indexes but in a different way. Can you test it ?

Denis Puthier · Answer 7 · Tue Nov 14 2017 22:35:49 GMT+0800 (China Standard Time)

Tested. This is unstable. It sometimes ends with a segfault... 2017-11-14 12:55 GMT+01:00 fafa13 <notifications@github.com>:

…

I have pushed a new version that free indexes but in a different way. Can you test it ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvxHjc4RcA3sRHwjN8ZPCmfCA66b51Gks5s2X_JgaJpZM4QaD4x> .

-- ==================================================================== Denis Puthier laboratoire INSERM TAGC/ERM206 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: puthier@tagc.univ-mrs.fr Tel: (National) 04 91 82 87 11 / (International) 33 4 91 82 87 11 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01 Web: http://tagc.univ-mrs.fr/puthier http://biologie.univ-mrs.fr/view-data.php?id=245 http://tagc.univ-mrs.fr/tbrowser ====================================================================

fafa13 · Answer 8 · Tue Nov 14 2017 22:52:30 GMT+0800 (China Standard Time)

OK. I have added some messages in the last push to help me to find the bug. Try it and send me the results please.

Denis Puthier · Answer 9 · Tue Nov 14 2017 22:55:42 GMT+0800 (China Standard Time)

$ gtftk short_long -i gtftk/data/simple_03/simple_short_long.gtf  -g 
  col = seqid 0
Updating index table ... OK
  col = source 0
Updating index table ... OK
  col = feature 0
Updating index table ... OK
  col = start 0
Updating index table ... OK
  col = end 0
Updating index table ... OK
  col = score 0
Updating index table ... OK
  col = strand 0
Updating index table ... OK
  col = phase 0
Updating index table ... OK
  col = attributes 2
    freeing index gene_id ... OK
    freeing index transcript_id ... OK
Updating index table ... OK
chr1	gtftk	gene	125	138	.	+	.	gene_id "G0001";
chr1	gtftk	transcript	125	138	.	+	.	gene_id "G0001"; transcript_id "G0001T002";
chr1	gtftk	exon	125	138	.	+	.	gene_id "G0001"; transcript_id "G0001T002"; exon_id "G0001T002E001";
chr1	gtftk	CDS	125	130	.	+	.	gene_id "G0001"; transcript_id "G0001T002"; ccds_id "CDS_G0001T002";
chr1	gtftk	gene	180	189	.	+	.	gene_id "G0002";
chr1	gtftk	transcript	185	189	.	+	.	gene_id "G0002"; transcript_id "G0002T002";
chr1	gtftk	exon	185	189	.	+	.	gene_id "G0002"; transcript_id "G0002T002"; exon_id "G0002T002E002";
chr1	gtftk	CDS	185	186	.	+	.	gene_id "G0002"; transcript_id "G0002T002"; ccds_id "CDS_G0002T002";
chr1	gtftk	gene	50	61	.	-	.	gene_id "G0003";
chr1	gtftk	transcript	54	61	.	-	.	gene_id "G0003"; transcript_id "G0003T002";
chr1	gtftk	exon	54	61	.	-	.	gene_id "G0003"; transcript_id "G0003T002"; exon_id "G0003T002E001";
chr1	gtftk	CDS	54	55	.	-	.	gene_id "G0003"; transcript_id "G0003T002"; ccds_id "CDS_G0003T002";
chr1	gtftk	gene	65	76	.	+	.	gene_id "G0004";
chr1	gtftk	transcript	65	76	.	+	.	gene_id "G0004"; transcript_id "G0004T002";
chr1	gtftk	exon	65	68	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; exon_id "G0004T002E001";
chr1	gtftk	exon	71	71	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; exon_id "G0004T002E002";
chr1	gtftk	exon	74	76	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; exon_id "G0004T002E003";
chr1	gtftk	CDS	66	68	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; ccds_id "CDS_G0004T002";
chr1	gtftk	CDS	71	71	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; ccds_id "CDS_G0004T002";
chr1	gtftk	CDS	74	75	.	+	.	gene_id "G0004"; transcript_id "G0004T002"; ccds_id "CDS_G0004T002";
chr1	gtftk	gene	33	47	.	-	.	gene_id "G0005";
chr1	gtftk	transcript	33	47	.	-	.	gene_id "G0005"; transcript_id "G0005T001";
chr1	gtftk	exon	33	35	.	-	.	gene_id "G0005"; transcript_id "G0005T001"; exon_id "G0005T001E001";
chr1	gtftk	exon	42	47	.	-	.	gene_id "G0005"; transcript_id "G0005T001"; exon_id "G0005T001E002";
chr1	gtftk	CDS	43	45	.	-	.	gene_id "G0005"; transcript_id "G0005T001"; ccds_id "CDS_G0005T001";
chr1	gtftk	gene	22	35	.	-	.	gene_id "G0006";
chr1	gtftk	transcript	28	35	.	-	.	gene_id "G0006"; transcript_id "G0006T002";
chr1	gtftk	exon	28	30	.	-	.	gene_id "G0006"; transcript_id "G0006T002"; exon_id "G0006T002E001";
chr1	gtftk	exon	33	35	.	-	.	gene_id "G0006"; transcript_id "G0006T002"; exon_id "G0006T002E002";
chr1	gtftk	CDS	29	30	.	-	.	gene_id "G0006"; transcript_id "G0006T002"; ccds_id "CDS_G0006T002";
chr1	gtftk	CDS	33	33	.	-	.	gene_id "G0006"; transcript_id "G0006T002"; ccds_id "CDS_G0006T002";
chr1	gtftk	gene	107	116	.	+	.	gene_id "G0007";
chr1	gtftk	transcript	107	116	.	+	.	gene_id "G0007"; transcript_id "G0007T001";
chr1	gtftk	exon	107	116	.	+	.	gene_id "G0007"; transcript_id "G0007T001"; exon_id "G0007T001E001";
chr1	gtftk	CDS	112	114	.	+	.	gene_id "G0007"; transcript_id "G0007T001"; ccds_id "CDS_G0007T001";
chr1	gtftk	gene	210	250	.	-	.	gene_id "G0008";
chr1	gtftk	transcript	220	250	.	-	.	gene_id "G0008"; transcript_id "G0008T002";
chr1	gtftk	exon	220	234	.	-	.	gene_id "G0008"; transcript_id "G0008T002"; exon_id "G0008T002E001";
chr1	gtftk	exon	240	250	.	-	.	gene_id "G0008"; transcript_id "G0008T002"; exon_id "G0008T002E002";
chr1	gtftk	CDS	220	221	.	-	.	gene_id "G0008"; transcript_id "G0008T002"; ccds_id "CDS_G0008T002";
chr1	gtftk	gene	3	14	.	-	.	gene_id "G0009";
chr1	gtftk	transcript	3	14	.	-	.	gene_id "G0009"; transcript_id "G0009T002";
chr1	gtftk	exon	3	14	.	-	.	gene_id "G0009"; transcript_id "G0009T002"; exon_id "G0009T002E001";
chr1	gtftk	CDS	5	10	.	-	.	gene_id "G0009"; transcript_id "G0009T002"; ccds_id "CDS_G0009T002";
chr1	gtftk	gene	176	186	.	+	.	gene_id "G0010";
chr1	gtftk	transcript	176	186	.	+	.	gene_id "G0010"; transcript_id "G0010T001";
chr1	gtftk	exon	176	186	.	+	.	gene_id "G0010"; transcript_id "G0010T001"; exon_id "G0010T001E001";
chr1	gtftk	CDS	184	186	.	+	.	gene_id "G0010"; transcript_id "G0010T001"; ccds_id "CDS_G0010T001";
chr1	gtftk	gene	250	280	.	-	.	gene_id "G0011";
chr1	gtftk	transcript	250	267	.	-	.	gene_id "G0011"; transcript_id "G0011T001";
chr1	gtftk	exon	250	259	.	-	.	gene_id "G0011"; transcript_id "G0011T001"; exon_id "G0011T001E001";
chr1	gtftk	exon	263	267	.	-	.	gene_id "G0011"; transcript_id "G0011T001"; exon_id "G0011T001E002";
chr1	gtftk	CDS	250	251	.	-	.	gene_id "G0011"; transcript_id "G0011T001"; ccds_id "CDS_G0011T001";
  col = seqid 0
Updating index table ... OK
  col = source 0
Updating index table ... OK
  col = feature 0
Updating index table ... OK
  col = start 0
Updating index table ... OK
  col = end 0
Updating index table ... OK
  col = score 0
Updating index table ... OK
  col = strand 0
Updating index table ... OK
  col = phase 0
Updating index table ... OK
  col = attributes 0
Segmentation fault: 11

fafa13 · Answer 10 · Tue Nov 14 2017 23:09:06 GMT+0800 (China Standard Time)

One more push ... I wait for the results ...

fafa13 · Answer 11 · Tue Nov 14 2017 23:09:24 GMT+0800 (China Standard Time)

Maybe it should be good for me to have a mac ...

Denis Puthier · Answer 12 · Tue Nov 14 2017 23:10:48 GMT+0800 (China Standard Time)

You have one. I will send you (again :)), the login. Bye 2017-11-14 16:09 GMT+01:00 fafa13 <notifications@github.com>:

…

Maybe it should be good for me to have a mac ... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvxHtw4c5-I7Wpv-L0r87eV51eCBNEMks5s2a0kgaJpZM4QaD4x> .

-- ==================================================================== Denis Puthier laboratoire INSERM TAGC/ERM206 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: puthier@tagc.univ-mrs.fr Tel: (National) 04 91 82 87 11 / (International) 33 4 91 82 87 11 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01 Web: http://tagc.univ-mrs.fr/puthier http://biologie.univ-mrs.fr/view-data.php?id=245 http://tagc.univ-mrs.fr/tbrowser ====================================================================

fafa13 · Answer 13 · Tue Nov 14 2017 23:53:54 GMT+0800 (China Standard Time)

I am on the mac and for me, there is no error. Can you confirm ?

Denis Puthier · Answer 14 · Wed Nov 15 2017 00:13:20 GMT+0800 (China Standard Time)

I don't confirm: gtftk short_long -i gtftk/data/simple_03/simple_short_long.gtf # raise a segfault. Be sure to work with the latest version # You may need to add a pub ssh-key on github in : upper-right corner

settings > SSH and GPG key

git clone git@github.com:dputhier/gtftk.git gtftk cd gtftk conda env create -n gtftk_dev -f conda/env.yaml source activate gtftk_dev make install 2017-11-14 16:53 GMT+01:00 fafa13 <notifications@github.com>:

…

I am on the mac and for me, there is no error. Can you confirm ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvxHqqEQvupmc5VEjhw87YeIknN3qBbks5s2beSgaJpZM4QaD4x> .

-- ==================================================================== Denis Puthier laboratoire INSERM TAGC/ERM206 Parc Scientifique de Luminy case 928 163, avenue de Luminy 13288 MARSEILLE cedex 09 FRANCE Mail: puthier@tagc.univ-mrs.fr Tel: (National) 04 91 82 87 11 / (International) 33 4 91 82 87 11 Fax: (National) 04 91 82 87 01 / (International) 33 4 91 82 87 01 Web: http://tagc.univ-mrs.fr/puthier http://biologie.univ-mrs.fr/view-data.php?id=245 http://tagc.univ-mrs.fr/tbrowser ====================================================================

Denis Puthier · Answer 15 · Thu Nov 30 2017 05:38:07 GMT+0800 (China Standard Time)

This one is fixed in f52ee30