Installation problem when using as a dependency with setuptools

Question

Installation problem when using as a dependency with setuptools

postlund opened this issue 7 years ago · comments

I'm using srptools (great project btw) as a dependency in my library (pyatv) but currently I'm experiencing some issues during installation. The stack trace looks like this:

pyatv$ python setup.py develop
running develop
...
Searching for srptools>=0.1.1
Reading https://pypi.python.org/simple/srptools/
Downloading https://pypi.python.org/packages/d2/19/4d9beec46e82d17249517291d0564c9eb6773c51015c278fef97312e63e3/srptools-0.1.1.tar.gz#md5=811c38f3e45039a8973cc6820d3ce94f
Best match: srptools 0.1.1
Processing srptools-0.1.1.tar.gz
Writing /tmp/easy_install-7pdx2q5r/srptools-0.1.1/setup.cfg
Running srptools-0.1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-7pdx2q5r/srptools-0.1.1/egg-dist-tmp-dzq732f7
Traceback (most recent call last):
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 158, in save_modules
    yield saved
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
    yield
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 254, in run_setup
    _execfile(setup_script, ns)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 48, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/setup.py", line 6, in <module>
    # Read in version without importing pyatv
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/srptools/__init__.py", line 1, in <module>
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/srptools/context.py", line 4, in <module>
ImportError: No module named 'six'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 48, in <module>
    'Topic :: Home Automation',
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/develop.py", line 36, in run
    self.install_for_development()
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/develop.py", line 150, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 746, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 826, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1092, in best_match
    return self.obtain(req, installer)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1104, in obtain
    return installer(requirement)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 699, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 880, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1119, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1105, in run_setup
    run_setup(setup_script, args)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 257, in run_setup
    raise
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
    yield
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 170, in save_modules
    saved_exc.resume()
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 145, in resume
    six.reraise(type, exc, self._tb)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 158, in save_modules
    yield saved
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
    yield
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 254, in run_setup
    _execfile(setup_script, ns)
  File "/home/pista/git/pyatv/lib/python3.5/site-packages/setuptools/sandbox.py", line 48, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/setup.py", line 6, in <module>
    # Read in version without importing pyatv
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/srptools/__init__.py", line 1, in <module>
  File "/tmp/easy_install-7pdx2q5r/srptools-0.1.1/srptools/context.py", line 4, in <module>
ImportError: No module named 'six'

The problem itself is quite simple. Since you are importing VERSION from srptools in setup.py, that will basically evaluate everything in that module. So context will be pulled in which in turn requires "six". Since this package is not part of setup_requires, it will not yet have been installed at this stage. A suggestion to work-around this is to place version in a separate file (that does not depend on anything) and just pull that one in. I have done a similar thing in my library:
https://github.com/postlund/pyatv/blob/master/setup.py
https://github.com/postlund/pyatv/blob/master/pyatv/const.py
I assume adding six to setup_requires might work as well, but I have not tried that myself. Is this anything that can be fixed soon? It's a bit of a usage blocker.

Igor Starikov · Answer 1 · Wed Jun 14 2017 16:35:37 GMT+0800 (China Standard Time)

Thank you for the report.
Six is added into setup_requires.

Pierre Ståhl · Answer 2 · Wed Jun 14 2017 18:07:01 GMT+0800 (China Standard Time)

Thanks for the quick reply! As I said, I'm not 100% sure that setup_requires works that way as I have not used it. I suspect it is for other things. Some logically reasoning about the issue concludes that the import-line that causes an issue here must still be evaluated before setuptools is configured. So I do not think it will work and AFAIK it's only possible to test it out once it's on pypi. It must be downloadable by pip.

Pierre Ståhl · Answer 3 · Wed Jun 14 2017 18:10:41 GMT+0800 (China Standard Time)

Just tried it out in a clean venv and it doesn't work (setup.py cannot be evaluated):

$ git clone https://github.com/idlesign/srptools/
$ cd srptools
$virtualenv  -p python3.5 .
Running virtualenv with interpreter /usr/bin/python3.5
Using base prefix '/usr'
New python executable in /home/pista/git/srptools/bin/python3.5
Also creating executable in /home/pista/git/srptools/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ source bin/activate
$ python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    from srptools import VERSION
  File "/home/pista/git/srptools/srptools/__init__.py", line 1, in <module>
    from .context import SRPContext
  File "/home/pista/git/srptools/srptools/context.py", line 4, in <module>
    from six import integer_types, PY3
ImportError: No module named 'six'

So unless a system wide installation of python is used and six is installed there, it gives this issue.

Igor Starikov · Answer 4 · Wed Jun 14 2017 22:39:35 GMT+0800 (China Standard Time)

I see. Let's make it a separate module.

Pierre Ståhl · Answer 5 · Thu Jun 15 2017 14:00:26 GMT+0800 (China Standard Time)

It's still the same issue I'm afraid:

$ python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    from srptools.version import VERSION
  File "/home/pista/git/srptools/srptools/__init__.py", line 1, in <module>
    from .context import SRPContext
  File "/home/pista/git/srptools/srptools/context.py", line 4, in <module>
    from six import integer_types, PY3
ImportError: No module named 'six'

When you import anything from your library, even a specific module, the __init__.py file will be loaded as well. So it's basically the same situation as before. That's why I do this hack:

# Read in version without importing pyatv
# http://stackoverflow.com/questions/6357361/alternative-to-execfile-in-python-3
exec(compile(open('pyatv/const.py', "rb").read(), 'pyatv/const.py', 'exec'))

The details are on SO.

Igor Starikov · Answer 6 · Thu Jun 15 2017 17:17:00 GMT+0800 (China Standard Time)

Sorry, I'm kind of in a rush for last three days.
I shall see what exactly is going on there in a day or two.

Pierre Ståhl · Answer 7 · Thu Jun 15 2017 17:18:40 GMT+0800 (China Standard Time)

No worries, take the time you need! Personal life comes first!

Igor Starikov · Answer 8 · Sat Jun 17 2017 12:27:36 GMT+0800 (China Standard Time)

We'll do it in a slightly different manner with re, since:

exec seems to be an overkill;
I'd like to keep compatibility with release command of makeapp.

Should work now.

Pierre Ståhl · Answer 9 · Sun Jun 18 2017 17:14:06 GMT+0800 (China Standard Time)

Sure, you do it as you please 😄 I can confirm that it works now! Would be really good if you could make a release to pypi as well, so it's possible to use srptools as a dependency in other projects.

Igor Starikov · Answer 10 · Sun Jun 18 2017 20:11:00 GMT+0800 (China Standard Time)

0.2.0 is out.
Thank you. Closing this.

Pierre Ståhl · Answer 11 · Sun Jun 18 2017 20:44:51 GMT+0800 (China Standard Time)

Top notch, thank you very much!