run-python fails

[Tea Tušar]

On Windows, the python do.py run-python command produces the following error:

AML     ['code-experiments/src/coco_random.c', 'code-experiments/src/coco_suite.c', 'code-experiments/src/coco_observer.c', 'code-experiments/src/coco_runtime_c.c'] -> code-experiments/build/python/cython/coco.c
COPY    code-experiments/src/coco.h -> code-experiments/build/python/cython/coco.h
COPY    code-experiments/src/bbob2009_testcases.txt -> code-experiments/build/python/bbob2009_testcases.txt
COPY    code-experiments/build/python/README.md -> code-experiments/build/python/README.txt
EXPAND  code-experiments/build/python/setup.py.in to code-experiments/build/python/setup.py
PYTHON  setup.py install --user in code-experiments/build/python
RUN     python coco_test.py in code-experiments\build\python
ERROR: return value=1
Traceback (most recent call last):
  File "coco_test.py", line 13, in <module>
    from cocoex import Suite
ImportError: cannot import name Suite

[Nikolaus Hansen]

Very strange, these are lines 12 and 13 of coco_test.py:

import cocoex as ex
from cocoex import Suite

One general reason I can figure is that PYTHON setup.py install ... and python coco_test.py use different Pythons on the computer. The former uses os.environ.get('PYTHON').

You might test this by changing line 115 of cocoutils.py

    if os.environ.get('PYTHON') is not None:

to

    if False and os.environ.get('PYTHON') is not None:

and see whether this works (and/or print os.environ.get('PYTHON')).

[Nikolaus Hansen]

Otherwise, I would

import cocoex as ex

in IPython and investigate the module manually (TAB is your friend).

[Tea Tušar]

This is what I get in IPython:

In [1]: import cocoex as ex
C:\...\AppData\Roaming\Python\Python27\site-packages\cocoex-0.0_5605_g24beb26-py2.7-win-amd64.egg\cocoex\_interface.py:3: 
UserWarning: Module cocoex was already imported from 
C:\...\AppData\Roaming\Python\Python27\site-packages\cocoex-0.0_5605_g24beb26-py2.7-win-amd64.egg\cocoex\__init__.pyc, 
but c:\...\documents\...\code-experiments\build\python is being added to sys.path

In [2]: ex.
  File "<ipython-input-2-f38d02068a8d>", line 1
    ex.
       ^
SyntaxError: invalid syntax

Any ideas?

[Nikolaus Hansen]

Sorry, ex. was in reality ex.<TAB>, to see the available completions of the statement. How about

import cocoex
dir(cocoex)
cocoex.<TAB>

FTR, I see

['Observer',
 'Suite',
 '__builtins__',
 '__cached__',
 '__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__',
 'exceptions',
 'interface',
 'known_suite_names',
 'log_level',
 'utilities']

on dir(cocoex).

[Nikolaus Hansen]

Another comment: I believe you import an old version, namely some v0.0... (so a very old version) not sure though. I also don't exactly understand why you get this warning...

Re: version number, I see cocoex-0.7b0-...:

>>> cocoex.__file__
'/Users/hansen/.local/lib/python3.4/site-packages/cocoex-0.7b0-py3.4-macosx-10.5-x86_64.egg/cocoex/__init__.py'

So I guess your problem is that you don't see the most recently installed module, or that it hasn't been installed.

[Tea Tušar]

I checked and you are right, I do have an ancient version (cocoex-0.0_5605_...) that doesn't even have Suite. And this is what was causing problems.

However, shouldn't version update be kind of automatic?

[Tea Tušar]

Some more information. It turned out that I had four versions of cocoex installed:

C:\...>pip uninstall cocoex
Uninstalling CoCOex-0.0-5577-gab8ae8c:
  c:\...\appdata\roaming\python\python27\site-packages\cocoex-0.0_5577_gab
8ae8c-py2.7-win-amd64.egg
Proceed (y/n)? y
  Successfully uninstalled CoCOex-0.0-5577-gab8ae8c

C:\...>pip uninstall cocoex
Uninstalling CoCOex-0.0-818-gc6b9595:
  c:\...\appdata\roaming\python\python27\site-packages\cocoex-0.0_818_gc6b
9595-py2.7-win-amd64.egg
Proceed (y/n)? y
  Successfully uninstalled CoCOex-0.0-818-gc6b9595

C:\...>pip uninstall cocoex
Uninstalling CoCOex-0.0-811-g2ec2ae0:
  c:\...\appdata\roaming\python\python27\site-packages\cocoex-0.0_811_g2ec
2ae0-py2.7-win-amd64.egg
Proceed (y/n)? y
  Successfully uninstalled CoCOex-0.0-811-g2ec2ae0

C:\...>pip uninstall cocoex
Uninstalling CoCOex-0.0-810-g6b49540:
  c:\...\appdata\roaming\python\python27\site-packages\cocoex-0.0_810_g6b4
9540-py2.7-win-amd64.egg
Proceed (y/n)? y
  Successfully uninstalled CoCOex-0.0-810-g6b49540

C:\...>pip uninstall cocoex
Cannot uninstall requirement cocoex, not installed

When I afterwards call python do.py run-python, I get the same message as before:

PYTHON  setup.py install --user in code-experiments/build/python
RUN     python coco_test.py in code-experiments\build\python
ERROR: return value=1
Traceback (most recent call last):
  File "coco_test.py", line 13, in <module>
    from cocoex import Suite
ImportError: cannot import name Suite

If I try to uninstall cocoex now, I can't, because pip says I don't have one, while in IPython I again get the old one:

In [1]: import cocoex

In [2]: cocoex.
cocoex.Benchmark  cocoex.benchmarks cocoex.utilities
cocoex.Problem    cocoex.exceptions

In [2]: cocoex.__file__
Out[2]: 'C:\\...\\AppData\\Roaming\\Python\\Python27\\site-packages\\cocoex-0.0_5605_g24beb26-py2.7-win-amd64.egg\\cocoex\\__init__.pyc'

[Nikolaus Hansen]

Out of interest, can you show the result of

>>> import os
>>> print(os.environ.get('PYTHON'))

in IPython?

If the resulting value is not None, then python do.py run-python might work now, after pulling the current development branch.

Only if it works, I suggest to remove all old installed package(s) by hand: just delete all cocoex-0.*.egg folders you can find and re-install the current cocoex again. Like this you prevent strange random errors in future.

[Tea Tušar]

The result is None.

[Nikolaus Hansen]

Manually removing all previously installed versions of cocoex, namely all cocoex-*.egg folders, from the system worked to address the problem. Finding all versions can be non-trivial though.

The definite cause of the problem remains unclear. A possible reason might have been that the module has been installed previously without the --user option or that several versions of Python do not interact well with each other. This might be a useful entry point to dig deeper and

python -m site

prints some useful information.

On a Mac, removing the installation(s) found under ~/.local/lib e.g. with

find ~/.local/lib -name cocoex

has worked in some cases. mkdir old; find ~/.local/lib -name "cocoex-*" -exec mv \{\} old \; moves all these installations to an old folder.