Allow multi-node scheduling
owlas opened this issue · comments
Embarrassing parallelism is currently implemented with joblib but this does not work if the job should be distributed across multiple nodes of a cluster (i.e. distributed memory). It would be really good to allow this for massive horizontal scaling of computational resources.
A good option could be MPI for python