CMandAS3 errors and warnings depending on number of workers and parameters
nhansendev opened this issue · comments
Steps to reproduce
Versions:
Python 3.10.7
nevergrad 0.6.0
Using the code block below the following combinations produce either normal results, InjectionWarnings, or RuntimeErrors:
- Normal results: ~200 parameters or fewer, any number of workers
- InjectionWarnings: greater than 200 parameters and two or more workers
- RuntimeErrors: greater than 200 parameters and exactly one worker
Replacing the return value of the _loss function with zero or another constant prevents the issues from appearing, though this is not a particularly useful method.
The messages accompanying the warnings and errors both mention the ask function being called out-of-order.
Observed Results
InjectionWarnings:
/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 33, 'index': 1, 'counter': 3}
This could be a bug in the calling order/logics or due to
a too small popsize used in `ask()` or when only using
`ask(1)` repeatedly. Please check carefully.
In case this is desired, the warning can be surpressed with
``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``
warnings.warn("""orphanated injected solution %s
/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 35, 'index': 0, 'counter': 2}
This could be a bug in the calling order/logics or due to
a too small popsize used in `ask()` or when only using
`ask(1)` repeatedly. Please check carefully.
In case this is desired, the warning can be surpressed with
``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``
warnings.warn("""orphanated injected solution %s
/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py:2891: InjectionWarning: orphanated injected solution {'iteration': 35, 'index': 1, 'counter': 3}
This could be a bug in the calling order/logics or due to
a too small popsize used in `ask()` or when only using
`ask(1)` repeatedly. Please check carefully.
In case this is desired, the warning can be surpressed with
``warnings.simplefilter("ignore", cma.evolution_strategy.InjectionWarning)``
warnings.warn("""orphanated injected solution %s
RuntimeError:
Traceback (most recent call last):
File "/home/nate/Desktop/Atom/testing.py", line 840, in <module>
main()
File "/home/nate/Desktop/Atom/testing.py", line 832, in main
optimizer.minimize(_loss)
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 694, in minimize
args = self.ask()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
candidate = self._internal_ask_candidate()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 1660, in _internal_ask_candidate
candidate = opt.ask()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
candidate = self._internal_ask_candidate()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 2279, in _internal_ask_candidate
return opt.ask()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
candidate = self._internal_ask_candidate()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 737, in _internal_ask_candidate
return self.optim.ask()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/base.py", line 496, in ask
candidate = self._internal_ask_candidate()
File "/home/nate/.local/lib/python3.10/site-packages/nevergrad/optimization/optimizerlib.py", line 580, in _internal_ask_candidate
self._to_be_asked.extend(self.es.ask())
File "/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py", line 1922, in ask
pop_geno = self.ask_geno(number, xmean, sigma_fac)
File "/home/nate/.local/lib/python3.10/site-packages/cma/evolution_strategy.py", line 2165, in ask_geno
raise RuntimeError(
RuntimeError: Mean shift samples are expected but missing.
This happens if, for example, `ask` is called more than once, without calling `tell`
(because the first call removes the samples from the injection list).
`cma.sigma_adaptation.CMAAdaptSigmaTPA` step-size adaptation generates mean shift
samples and relies on them.
Using ``ask(1)`` for any subsequent calls of `ask` works OK and TPA works if the
first two samples from the first call are retained as first samples when calling `tell`.
EXAMPLE:
X = es.ask()
X.append(es.ask(1)[0])
...
es.tell(X, ...)
Expected Results
It runs without errors/warnings
Relevant Code
import nevergrad as ng
import random
def _loss(*args, **kwargs):
return random.random()
def main():
params = 300
workers = 2
parametrization = ng.p.Array(shape=(params,), lower=-3, upper=3)
optimizer = ng.optimizers.CMandAS3(
parametrization=parametrization, budget=1000, num_workers=workers
)
optimizer.minimize(_loss)
if __name__ == "__main__":
main()
Thanks for pointing this out.
I've run the code without error on the current Nevergrad. I'll double check with other configurations in case a Pypi package number might have an impact.
In case someone reproduces the error with nevergrad >= 0.8.0, could you << conv env export >> so that I can see which package versions are responsible ?
Thanks for pointing the issue, there were related bugs some time ago, I hope the current version fixes all bugs.
Everything runs smoothly at least in my environment and after the recent fixes.
Thank for your work @Obliman and others who commented related issues, this helps a lot.