Parallel setup doesn't work
jeromekelleher opened this issue · comments
Using the parallel option for setup on program_comparison or tsinfer_performance results in an exception. This is probably because it needs to pickle the main object (which would be very tricky to make work). This is the reason subprocess workers should be standalone functions.
Does this work for any of the datasets? Might be simplest just to remove this option.
It worked when I tried it for the other datasets (e.g. metrics_by_mutation_rate), but you are right that these are not standalone functions, so I was slightly surprised that it did work. Might this be some python version thing that has been sorted in a recent python3 version and which might differ between your tsinfer setup and mine?
I will try it again on holly when I'm back in Oxford, to try to figure out why it works for me when it probably shouldn't.
Ah, that could be it. Was this on your laptop? Holly is using a pretty old version of Python now. It's not a crucial feature in any case, so not to worry.
I think it worked for me both on my laptop and on holly, but I could be running a more recent Python3 on holly in a virtualenv. Will debug when I return, but as you say, this is not a showstopper.