Giters

esa / pygmo2

A Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.

Home Page:https://esa.github.io/pygmo2/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

speeding up ipyparallel evolutions [FEATURE]

dalbabur opened this issue · comments

commented

Is your feature request related to a problem? Please describe.
Sending the problem and population to every ipyparallel engine with every evaluation is fairly slow, specially when using many (>100) engines. If the problem does not change during the evolution, it should be enough to send it only once. Additionally, it may be worth having the option to read/write the population to disk, completely avoiding the scatter step.

Describe the solution you'd like
I think I'm looking for a new bfe, that does not pickle everything every time, and that can use other ipyparallel views (such as BroadcastView). Would other aspects of pygmo have to change to enable this?