Overzealous shutdown proceedure.
tirohia opened this issue · comments
If one hits ctrl-C to stop a pipeline running, it shuts down everything. As in all user owned processes, including the shell.
I'm running things in screen so I can leave pipelines running after logging out. If I hit ctrl-C to terminate the pipeline it asks whether to terminate it's own process. Which is nice. It then proceeds to terminate the screen session, all other screen sessions I have going in other windows, and boots me out of both this session and all other sessions on that machine that I have open. Not just a one off, first time I wasn't sure what happened, 2nd time I figured out what had happened and the 3rd time I did it just to test it.
This surely can't be the anticipated behaviour?
Ben.
=================================================================================| Starting Pipeline at 2018-10-15 11:12 |
============================================ Stage trim ============================================
=========================================== Stage align ============================================
====================================== Stage removeDuplicates ======================================
^C
Pipeline job running as process 46813. Terminate? (y/n): y
Terminating process 46813 ...
Running stop command ...
Shared connection to login.mahuika.nesi.org.nz closed.
ben@blueAnt:~/workspace$ ssh mahuika
First Factor:
Second Factor (optional):
First Factor:
Second Factor (optional):
Last login: Sun Oct 14 22:58:34 2018 from 192.168.30.14
[bcur001@mahuika02 ~]$ screen -ls
No Sockets found in /var/run/screen/S-bcur001.
[bcur001@mahuika02 ~]$
Similar problem to #190
Yes, might be related - @tirohia, can you clarify what OS and shell you are using?
It does indeed, sound similar. This is on a node running CentOS Linux 7, Kernel is Linux 3.10.0-693.2.2.el7.x86_64, using a bash shell.
On this line here I am not sure if removing the pid file in the background is such a good idea when you are creating it just shortly after... What happens on slower storage when you remove a file in the background is that it might wait too long and then clobber the new one you just created? Not sure if this would solve your issue but worth looking into I think. Sorry I think I got my wires crossed but I have a pull request that will check that the pid is at least a java process before attempting to kill it.