Corrigir erro build #2556 travis: Getting distribution for 'lazy_object_proxy' / assert newdist is not None / AssertionError

[IDG SERPRO]

Erro começou em https://travis-ci.org/plonegovbr/brasil.gov.portal/builds/535408347.

Primeiro, efetue um git clone do projeto:

git clone https://github.com/plonegovbr/brasil.gov.portal.git
cd brasil.gov.portal

Para simular, usamos a idéia apresentada em https://community.plone.org/t/buildout-not-working-couldnt-find-index-page-for-couldnt-find-a-distribution-disabling-non-https-access-to-apis-on-pypi/5069/21?u=idgserpro que usa um diretório de eggs completamente limpo:

mkdir -p eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs"

Traceback:

Got astroid 1.6.6.
Getting distribution for 'wrapt'.
zip_safe flag not set; analyzing archive contents...
Got wrapt 1.11.1.
Getting distribution for 'lazy_object_proxy'.
/home/user/.pythonz/pythons/CPython-2.7.9/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
  warnings.warn(msg)
/home/user/.pythonz/pythons/CPython-2.7.9/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_scm_version'
  warnings.warn(msg)
While:
  Installing pylint.
  Getting distribution for 'lazy_object_proxy'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 2123, in main
    getattr(buildout, command)(args)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 796, in install
    installed_files = self[part]._call(recipe.install)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 1553, in _call
    return f()
  File "/tmp/brasil.gov.portal/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", line 126, in install
    reqs, ws = self.working_set()
  File "/tmp/brasil.gov.portal/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 913, in install
    return installer.install(specs, working_set)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 714, in install
    for dist in self._get_dist(req, ws):
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 563, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 1730, in _move_to_eggs_dir_and_compile
    assert newdist is not None  # newloc above is missing our dist?!
AssertionError

Ao rodar novamente:

mkdir -p eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs"

Traceback:

Develop: '/tmp/brasil.gov.portal/.'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Unused options for buildout: 'buildout-user' 'effective-user' 'package-min-coverage' 'unzip'.
Updating instance.
Updating test.
Updating code-analysis.
Installed Git pre-commit hook.
Uninstalled Git pre-push hook.
Updating i18ndude.
Updating dependencychecker.
Updating node.
Updating omelette.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/extras -> /tmp/brasil.gov.portal/parts/omelette/rdflib/extras).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyMicrodata -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyMicrodata).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyRdfa -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyRdfa).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyRdfa/extras -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyRdfa/extras).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyRdfa/host -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyRdfa/host).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyRdfa/rdfs -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyRdfa/rdfs).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/parsers/pyRdfa/transform -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/parsers/pyRdfa/transform).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/serializers -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/serializers).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/sparql -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/sparql).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/sparql/results -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/sparql/results).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/plugins/stores -> /tmp/brasil.gov.portal/parts/omelette/rdflib/plugins/stores).  Skipping.
omelette: (While processing egg rdflib) Link already exists (/tmp/brasil.gov.portal/eggs/rdflib-4.1.2-py2.7.egg/rdflib/tools -> /tmp/brasil.gov.portal/parts/omelette/rdflib/tools).  Skipping.
omelette: (While processing egg ZServer) Link already exists (/tmp/brasil.gov.portal/eggs/ZServer-3.0-py2.7.egg/Products -> /tmp/brasil.gov.portal/parts/omelette/Products).  Skipping.
Updating precompile.
Compiling Python files.
Compiling locale files.
Installing pylint.
Getting distribution for 'lazy_object_proxy'.
/home/user/.pythonz/pythons/CPython-2.7.9/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
  warnings.warn(msg)
/home/user/.pythonz/pythons/CPython-2.7.9/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_scm_version'
  warnings.warn(msg)
While:
  Installing pylint.
  Getting distribution for 'lazy_object_proxy'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 2123, in main
    getattr(buildout, command)(args)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 796, in install
    installed_files = self[part]._call(recipe.install)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/buildout.py", line 1553, in _call
    return f()
  File "/tmp/brasil.gov.portal/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", line 126, in install
    reqs, ws = self.working_set()
  File "/tmp/brasil.gov.portal/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 913, in install
    return installer.install(specs, working_set)
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 714, in install
    for dist in self._get_dist(req, ws):
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 563, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "/tmp/brasil.gov.portal/eggs/zc.buildout-2.9.6-py2.7.egg/zc/buildout/easy_install.py", line 1704, in _move_to_eggs_dir_and_compile
    os.rename(tmp_loc, newloc)
OSError: [Errno 39] Directory not empty

Por algum motivo, ele descompacta a dependência lazy-object-proxy como lazy_object_proxy-0.0.0-py2.7-linux-x86_64.egg, ou seja, com versão "0.0.0".

Se rodar:

mkdir -p eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs" versions:zc.buildout= versions:setuptools=

Não ocorre o erro, mas ao testar isso no ambiente do Travis, não funcionou. Então deduzimos que foi um problema introduzido pelo lazy-object-proxy 1.4.x, tanto que, ao pinar para 1.3.x, o erro para de ocorrer.

Esse relato foi aberto para documentar todo esse processo e, caso ocorra novamente com alguma nova dependência, decidir se a dependência que será pinada ou se serão as versões de zc.buildout e de setuptools.