logging, output for a specific input for a specific stage?

[Gabriel A. Devenyi]

I may have a pipeline shutdown and show the following error at the end:

---------------------------- n4correct  ( nih_chp_09291_t1.n4correct )  ----------------------------

Command in stage n4correct failed with exit status = 143 :

/data/chamal/projects/saashi/ASD_heterogeneity/Datasets/NIMH_ASD/bpipe/bpipe_itn4_4.5-8.5_priors/iterativeN4_multispectral_nimh.sh nih_chp_09291_t1.n4correct.mnc ../input/nih_chp_09291_t1.mnc

I know that this is the first step in the pipeline so any stdout is very much buried.

The "bpipe log" doesn't seem to provide the ability to get a specific stage/input file.

Is this possible? If not, could it be added?

[Simon Sadedin]

In this case bpipe errors is the intended mechanism to see the output from the failed command(s)(along with all the other gory details). Did you give that a shot and it didn't work? In some cases - when there is an internal error (queuing system down, etc) you might not get any result from it.

There is another option but it requires knowing the bpipe internal command id or the stage name:

bpipe log -c <command id>

All the command ids get saved in .bpipe/commandtmp but that's probably a bit frustrating to dig into.

Hope this helps!

[Gabriel A. Devenyi]

Thanks.