Raise "error: could not create 'build': File exists" while installing

Question

Raise "error: could not create 'build': File exists" while installing

ruifengma opened this issue 5 years ago · comments

When I firstly install trfl, it raised error almost at the end of installation,
Failed building wheel for trfl Running setup.py clean for trfl Failed to build trfl Installing collected packages: trfl Running setup.py install for trfl ... error
The further issue is like

running install running build running build_py creating build error: could not create 'build': File exists

Kaleigh Clary commented 5 years ago

MacOS

Matteo Hessel · Answer 1 · Mon Mar 04 2019 22:28:57 GMT+0800 (China Standard Time)

What command did you use to install trfl?

Kaleigh Clary · Answer 2 · Tue Mar 12 2019 04:25:36 GMT+0800 (China Standard Time)

I also had this problem using pip install git+git://github.com/deepmind/trfl.git. Here is my full error:

  Cloning git://github.com/deepmind/trfl.git to /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45
Requirement already satisfied: six in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.11.0)
Requirement already satisfied: absl-py in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (0.5.0)
Requirement already satisfied: numpy in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.14.5)
Requirement already satisfied: dm-sonnet in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.29)
Requirement already satisfied: wrapt in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.11.1)
Requirement already satisfied: semantic-version in /anaconda/lib/python3.5/site-packages (from dm-sonnet->trfl==1.0) (2.6.0)
Requirement already satisfied: contextlib2 in /anaconda/lib/python3.5/site-packages (from dm-sonnet->trfl==1.0) (0.5.5)
Building wheels for collected packages: trfl
  Building wheel for trfl (setup.py) ... error
  Complete output from command //anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-wheel-4y1mqadi --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  error: could not create 'build': File exists

  ----------------------------------------
  Failed building wheel for trfl
  Running setup.py clean for trfl
Failed to build trfl
Installing collected packages: trfl
  Running setup.py install for trfl ... error
    Complete output from command //anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-record-2pmg5v_c/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    error: could not create 'build': File exists

    ----------------------------------------
Command "//anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-record-2pmg5v_c/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/```

rui · Answer 3 · Tue Mar 12 2019 22:36:42 GMT+0800 (China Standard Time)

What command did you use to install trfl?

I used pip install git+git://github.com/deepmind/trfl.git for installation since using 'pip install trfl', it even cannot find the distribution.

rui · Answer 4 · Tue Mar 12 2019 22:37:24 GMT+0800 (China Standard Time)

I also had this problem using pip install git+git://github.com/deepmind/trfl.git. Here is my full error:

  Cloning git://github.com/deepmind/trfl.git to /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45
Requirement already satisfied: six in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.11.0)
Requirement already satisfied: absl-py in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (0.5.0)
Requirement already satisfied: numpy in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.14.5)
Requirement already satisfied: dm-sonnet in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.29)
Requirement already satisfied: wrapt in /anaconda/lib/python3.5/site-packages (from trfl==1.0) (1.11.1)
Requirement already satisfied: semantic-version in /anaconda/lib/python3.5/site-packages (from dm-sonnet->trfl==1.0) (2.6.0)
Requirement already satisfied: contextlib2 in /anaconda/lib/python3.5/site-packages (from dm-sonnet->trfl==1.0) (0.5.5)
Building wheels for collected packages: trfl
  Building wheel for trfl (setup.py) ... error
  Complete output from command //anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-wheel-4y1mqadi --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  error: could not create 'build': File exists

  ----------------------------------------
  Failed building wheel for trfl
  Running setup.py clean for trfl
Failed to build trfl
Installing collected packages: trfl
  Running setup.py install for trfl ... error
    Complete output from command //anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-record-2pmg5v_c/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    error: could not create 'build': File exists

    ----------------------------------------
Command "//anaconda/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-record-2pmg5v_c/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/hm/yd8cl6vd1bjbv3sjvd_nrrcc0000gr/T/pip-req-build-83bozo45/```

same with mine

Matteo Hessel · Answer 5 · Wed Mar 13 2019 00:31:21 GMT+0800 (China Standard Time)

what platform are you using? Mac-os or Linux?

Matteo Hessel · Answer 6 · Wed Mar 13 2019 02:30:45 GMT+0800 (China Standard Time)

Hi, thanks for the information.

We will be releasing pre-built binaries for MacOS very soon so that you will be able to install using pip install trfl.

In the meantime there is an open pull request (#12) that will allow you to build locally for MacOS (this will also be merged soon).

Matteo Hessel · Answer 7 · Thu Mar 14 2019 23:56:29 GMT+0800 (China Standard Time)

We now have pre-built binaries for both MacOS and Ubuntu, uploaded to PyPi
you can install them with pip install trfl

Ubuntu: make sure you have TensorFlow 1.12.0 and TensorFlow Probability 0.5
MacOS: make sure you have TensorFlow 1.13.1 and TensorFlow Probability 0.6

rui · Answer 8 · Fri Mar 15 2019 17:39:09 GMT+0800 (China Standard Time)

Thanks @mtthss , trfl could be installed now, but couldn't import it

Traceback (most recent call last): File "/Users/ruifengma/Code/truffle/first.py", line 2, in <module> import trfl File "/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/__init__.py", line 31, in <module> from trfl.dist_value_ops import categorical_dist_double_qlearning File "/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/dist_value_ops.py", line 33, in <module> from trfl import distribution_ops File "/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/distribution_ops.py", line 30, in <module> from trfl import gen_distribution_ops File "/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/gen_distribution_ops.py", line 2, in <module> _op_lib = tf.load_op_library(tf.resource_loader.get_path_to_datafile("_gen_distribution_ops.so")) File "/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/_gen_distribution_ops.so, 6): image not found

Matteo Hessel · Answer 9 · Fri Mar 15 2019 17:47:02 GMT+0800 (China Standard Time)

Do you have TensorFlow 1.13.1 and TensorFlow Probability 0.6 installed?

rui · Answer 10 · Fri Mar 15 2019 22:10:52 GMT+0800 (China Standard Time)

Do you have TensorFlow 1.13.1 and TensorFlow Probability 0.6 installed?

Yes, I do. followed by your previous guidance

Sergei Lebedev · Answer 11 · Fri Mar 15 2019 22:55:56 GMT+0800 (China Standard Time)

Hi @ruifengma, could you ls /Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/ please and post the output here?

abdel · Answer 12 · Mon Mar 18 2019 09:07:14 GMT+0800 (China Standard Time)

I had the same issue with the latest macOS binary, posted the outputs and info here: #9 (comment)

I guess can probably consolidate all 3 current issues.

Matteo Hessel · Answer 13 · Mon Mar 18 2019 17:01:42 GMT+0800 (China Standard Time)

Hi @abdel, good idea. The third issue was closed from the author since he could install with pip, so let's consolidate the remaining two here.

Could you ls /Users/ruifengma/.virtualenvs/truffle/lib/python3.6/site-packages/trfl/ please and post the output here?

abdel · Answer 14 · Mon Mar 18 2019 17:22:24 GMT+0800 (China Standard Time)

Hi @mtthss - see the contents of trfl in the site-packages directory below. As I mentioned in #9 (comment) previously, the gen_distribution_ops.so is there if I build from source, but not there when I install directly from pip.

I also noted the difference in the output wheels (between the pre-built wheels and one built locally), could be relevant here?

in site-packages/
› ls trfl -l
total 180
-rw-r--r--  1 Abdel staff  2693 Mar 18 10:29 __init__.py
drwxr-xr-x 20 Abdel staff   640 Mar 18 10:29 __pycache__/
-rw-r--r--  1 Abdel staff 17004 Mar 18 10:29 action_value_ops.py
-rw-r--r--  1 Abdel staff  3771 Mar 18 10:29 base_ops.py
-rw-r--r--  1 Abdel staff  2531 Mar 18 10:29 clipping_ops.py
-rw-r--r--  1 Abdel staff 14956 Mar 18 10:29 discrete_policy_gradient_ops.py
-rw-r--r--  1 Abdel staff 11383 Mar 18 10:29 dist_value_ops.py
-rw-r--r--  1 Abdel staff  6547 Mar 18 10:29 distribution_ops.py
-rw-r--r--  1 Abdel staff  3695 Mar 18 10:29 dpg_ops.py
-rw-r--r--  1 Abdel staff   190 Mar 18 10:29 gen_distribution_ops.py
-rw-r--r--  1 Abdel staff  4079 Mar 18 10:29 indexing_ops.py
-rw-r--r--  1 Abdel staff  2955 Mar 18 10:29 periodic_ops.py
-rw-r--r--  1 Abdel staff 10388 Mar 18 10:29 pixel_control_ops.py
-rw-r--r--  1 Abdel staff 16800 Mar 18 10:29 policy_gradient_ops.py
-rw-r--r--  1 Abdel staff 18763 Mar 18 10:29 retrace_ops.py
-rw-r--r--  1 Abdel staff  9577 Mar 18 10:29 sequence_ops.py
-rw-r--r--  1 Abdel staff  5106 Mar 18 10:29 target_update_ops.py
-rw-r--r--  1 Abdel staff 11075 Mar 18 10:29 value_ops.py
-rw-r--r--  1 Abdel staff 12052 Mar 18 10:29 vtrace_ops.py

Matteo Hessel · Answer 15 · Mon Mar 18 2019 22:12:08 GMT+0800 (China Standard Time)

Hi @abdel,

About the difference between your wheel and our wheel:

Your wheel: trfl-1.0-cp36-cp36m-macosx_10_7_x86_64.whl
Our wheel: trfl-1.0-cp36-cp36m-macosx_10_6_intel.whl

intel is used for a universal build with the i386 and x86_64 architectures
(from https://docs.python.org/2/distutils/apiref.html#module-distutils.util)
so I think that this shouldn't be an issue.

However, when I install from pip on my personal MacOS laptop I can see _gen_distribution_ops.so.

Just to double check, in your message you mention you can't find gen_distribution_ops.so;
it should be _gen_distribution_ops.so, with an underscore, both when you build locally as well as when you install from pip. Can you verify that there is no _gen_distribution_ops.so?

JRequenaC · Answer 16 · Mon Mar 18 2019 22:44:38 GMT+0800 (China Standard Time)

Hi @mtthss, same issue here:

MacOS, TF 1.13.1, TF probability 0.6, installing from pip: no _gen_distribution_ops.so image.

site-packages/trfl has the same contents as shown by @abdel before.

Akanksha Atrey · Answer 17 · Mon Mar 18 2019 23:42:03 GMT+0800 (China Standard Time)

Same issue as @JRequenaC.

Matteo Hessel · Answer 18 · Tue Mar 19 2019 03:24:34 GMT+0800 (China Standard Time)

We managed to reproduce the issue, which seems to only affect the python3 prebuilt binaries for MacOS. We will investigate this further (note that, in the meanwhile, building for MacOS locally is still possible for both python2 and python3 according to the instructions in INSTALL.md).

abdel · Answer 19 · Tue Mar 19 2019 07:28:25 GMT+0800 (China Standard Time)

intel is used for a universal build with the i386 and x86_64 architectures
(from https://docs.python.org/2/distutils/apiref.html#module-distutils.util)
so I think that this shouldn't be an issue.
That's good to know, thanks for the information!

Just to double check, in your message you mention you can't find gen_distribution_ops.so;
it should be _gen_distribution_ops.so, with an underscore, both when you build locally as well as when you install from pip. Can you verify that there is no _gen_distribution_ops.so?

Yeah, I just mistyped that 😄 Just to confirm was no _gen_distribution_ops.so. But looks like you guys found the issue anyway, I'll use the my locally built version in the meantime. Thanks!

Matteo Hessel · Answer 20 · Sat Apr 06 2019 01:00:29 GMT+0800 (China Standard Time)

TRFL version 1.0.1 is now out fixing the issue with the previous MacOS binaries.
We have now included new pre-built binaries for both Ubuntu and MacOS.
In both cases we require TensorFlow 1.13.1 and TensorFlow Probability 0.6.