[BUG] Unable to install `datar` in `Python 3.11 & Windows 10` due to `ERROR: Failed building wheel for rtoml`

Question

[BUG] Unable to install `datar` in `Python 3.11 & Windows 10` due to `ERROR: Failed building wheel for rtoml`

GitHunter0 opened this issue a year ago · comments

datar version checks

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of datar and its backends.

Issue Description

Hey @pwwang ,
I cannot install datar in Python 3.11 due to ERROR: Failed building wheel for rtoml.
The complete message is below:

Building wheels for collected packages: rtoml
  Building wheel for rtoml (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for rtoml (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\rtoml
      copying rtoml\__init__.py -> build\lib.win-amd64-cpython-311\rtoml
      copying rtoml\py.typed -> build\lib.win-amd64-cpython-311\rtoml
      running build_ext
      running build_rust
      error: can't find Rust compiler
     
      If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from 
it. Installing from the wheel would avoid the need for a Rust compiler.
     
      To update pip, run:
     
          pip install --upgrade pip      If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler 
toolchain.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for rtoml
Failed to build rtoml
ERROR: Could not build wheels for rtoml, which is required to install pyproject.toml-based projects

Expected Behavior

I expected pip install datar or pip install datar[pandas] to work.

Installed Versions

OS: Windows 10 Pro

Name Version Build Channel

asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
bzip2 1.0.8 he774522_0
ca-certificates 2023.5.7 h56e8100_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
debugpy 1.5.1 py311hd77b12b_0
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.7.0 pyha770c72_0 conda-forge
importlib_metadata 6.7.0 hd8ed1ab_0 conda-forge
ipykernel 6.15.0 pyh025b116_0 conda-forge
ipython 8.14.0 pyh08f2357_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
jupyter_client 8.2.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.1 py311h1ea47a8_0 conda-forge
libffi 3.4.4 hd77b12b_0
libsodium 1.0.18 h8d14728_1 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
numpy 1.25.0 pypi_0 pypi
openssl 3.0.9 h2bbff1b_0
packaging 23.1 pyhd8ed1ab_0 conda-forge
pandas 2.0.2 pypi_0 pypi
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pip 23.1.2 py311haa95532_0
platformdirs 3.6.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.38 pyha770c72_0 conda-forge
prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge
psutil 5.9.0 py311h2bbff1b_0
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pygments 2.15.1 pyhd8ed1ab_0 conda-forge
python 3.11.3 he1021f5_1
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.11 2_cp311 conda-forge
pytz 2023.3 pypi_0 pypi
pywin32 305 py311h2bbff1b_0
pyzmq 25.1.0 py311hd77b12b_0
rtoml 0.9.0 pypi_0 pypi
setuptools 67.8.0 py311haa95532_0
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.41.2 h2bbff1b_0
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h2bbff1b_0
tornado 6.2 py311h2bbff1b_0
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.6.3 hd8ed1ab_0 conda-forge
typing_extensions 4.6.3 pyha770c72_0 conda-forge
tzdata 2023.3 pypi_0 pypi
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
wheel 0.38.4 py311haa95532_0
xz 5.4.2 h8cc25b3_0
zeromq 4.3.4 h0e60522_1 conda-forge
zipp 3.15.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h8cc25b3_0

pwwang · Answer 1 · Tue Jun 27 2023 05:22:04 GMT+0800 (China Standard Time)

Is upgrading pip solving it?

$ pip install --upgrade pip

Related: docker/compose#8105

GitHunter0 · Answer 2 · Tue Jun 27 2023 05:39:04 GMT+0800 (China Standard Time)

Is upgrading pip solving it?

No man, I had already tried, my pip is up to date.

pwwang · Answer 3 · Tue Jun 27 2023 05:41:51 GMT+0800 (China Standard Time)

How about:

$ pip install -U setuptools_rust

?

GitHunter0 · Answer 4 · Tue Jun 27 2023 06:07:03 GMT+0800 (China Standard Time)

$ pip install -U setuptools_rust

Also didn't work...

pwwang · Answer 5 · Tue Jun 27 2023 06:09:16 GMT+0800 (China Standard Time)

Can you export your environment and attach it here:

$ conda env export > environment.yml

Let me see if I can reproduce it.

GitHunter0 · Answer 6 · Tue Jun 27 2023 06:19:23 GMT+0800 (China Standard Time)

Sure:

name: py311
channels:
  - conda-forge
  - defaults
dependencies:
  - asttokens=2.2.1=pyhd8ed1ab_0
  - backcall=0.2.0=pyh9f0ad1d_0
  - backports=1.0=pyhd8ed1ab_3
  - backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
  - bzip2=1.0.8=he774522_0
  - ca-certificates=2023.05.30=haa95532_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - debugpy=1.5.1=py311hd77b12b_0
  - decorator=5.1.1=pyhd8ed1ab_0
  - executing=1.2.0=pyhd8ed1ab_0
  - importlib-metadata=6.7.0=pyha770c72_0
  - importlib_metadata=6.7.0=hd8ed1ab_0
  - ipykernel=6.15.0=pyh025b116_0
  - ipython=8.14.0=pyh08f2357_0
  - jedi=0.18.2=pyhd8ed1ab_0
  - jupyter_client=8.2.0=pyhd8ed1ab_0
  - jupyter_core=5.3.1=py311h1ea47a8_0
  - libffi=3.4.4=hd77b12b_0
  - libsodium=1.0.18=h8d14728_1
  - matplotlib-inline=0.1.6=pyhd8ed1ab_0
  - nest-asyncio=1.5.6=pyhd8ed1ab_0
  - openssl=3.0.9=h2bbff1b_0
  - packaging=23.1=pyhd8ed1ab_0
  - parso=0.8.3=pyhd8ed1ab_0
  - pickleshare=0.7.5=py_1003
  - pip=23.1.2=py311haa95532_0
  - platformdirs=3.6.0=pyhd8ed1ab_0
  - prompt-toolkit=3.0.38=pyha770c72_0
  - prompt_toolkit=3.0.38=hd8ed1ab_0
  - psutil=5.9.0=py311h2bbff1b_0
  - pure_eval=0.2.2=pyhd8ed1ab_0
  - pygments=2.15.1=pyhd8ed1ab_0
  - python=3.11.3=he1021f5_1
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - python_abi=3.11=2_cp311
  - pywin32=305=py311h2bbff1b_0
  - pyzmq=25.1.0=py311hd77b12b_0
  - setuptools=67.8.0=py311haa95532_0
  - six=1.16.0=pyh6c4a22f_0
  - sqlite=3.41.2=h2bbff1b_0
  - stack_data=0.6.2=pyhd8ed1ab_0
  - tk=8.6.12=h2bbff1b_0
  - tornado=6.2=py311h2bbff1b_0
  - traitlets=5.9.0=pyhd8ed1ab_0
  - typing-extensions=4.6.3=hd8ed1ab_0
  - typing_extensions=4.6.3=pyha770c72_0
  - vc=14.2=h21ff451_1
  - vs2015_runtime=14.27.29016=h5e58377_2
  - wcwidth=0.2.6=pyhd8ed1ab_0
  - wheel=0.38.4=py311haa95532_0
  - xz=5.4.2=h8cc25b3_0
  - zeromq=4.3.4=h0e60522_1
  - zipp=3.15.0=pyhd8ed1ab_0
  - zlib=1.2.13=h8cc25b3_0
  - pip:
      - numpy==1.25.0
      - pandas==2.0.2
      - pytz==2023.3
      - rtoml==0.9.0
      - semantic-version==2.10.0
      - setuptools-rust==1.6.0
      - tzdata==2023.3

pwwang · Answer 7 · Tue Jun 27 2023 06:58:28 GMT+0800 (China Standard Time)

Looks like the culprit is that rtoml doesn't have a binary/wheel built under Windows:

To build it on Windows, you need rust (conda install -c conda-forge rust) and the Microsoft C++ Build Tools (https://learn.microsoft.com/en-us/windows/dev-environment/rust/setup)

 ~/github/datar   dev  📦 v0.12.1  🐍 v3.10.9  ~1s 
15:51:02 ❯ poetry show -t
datar-arrow 0.0.0 The pyarrow backend for datar
├── datar >=0.12,<0.13
└── pyarrow >=11,<12
    └── numpy >=1.16.6 
datar-numpy 0.2.0 The numpy backend for datar
├── datar >=0.12,<0.13
└── numpy >=1.20,<2.0
datar-pandas 0.3.0 The pandas backend for datar
├── datar >=0.12,<0.13
├── datar-numpy >=0.2,<0.3
│   ├── datar >=0.12,<0.13 
│   └── numpy >=1.20,<2.0 
└── pdtypes >=0.0.4,<0.0.5
    └── pandas >=1.2,<2.0 
        ├── numpy >=1.20.3 
        ├── numpy >=1.21.0 (circular dependency aborted here)
        ├── numpy >=1.23.2 (circular dependency aborted here)
        ├── python-dateutil >=2.8.1 
        │   └── six >=1.5 
        └── pytz >=2020.1 
numpy 1.24.3 Fundamental package for array computing in Python
pipda 0.12.0 A framework for data piping in python
└── executing >1.1.0,<2.0
pytest 7.3.1 pytest: simple powerful testing with Python
├── colorama *
├── exceptiongroup >=1.0.0rc8
├── iniconfig *
├── packaging *
├── pluggy >=0.12,<2.0
└── tomli >=1.0.0
pytest-cov 4.0.0 Pytest plugin for measuring coverage.
├── coverage >=5.2.1
│   └── tomli *  
└── pytest >=4.6
    ├── colorama * 
    ├── exceptiongroup >=1.0.0rc8 
    ├── iniconfig * 
    ├── packaging * 
    ├── pluggy >=0.12,<2.0 
    └── tomli >=1.0.0 
python-simpleconf 0.5.7 Simple configuration management with python.
├── diot >=0.1,<0.2
│   └── inflection >=0.5,<0.6 
└── rtoml >=0.8,<0.9     <<<<<<<<<<<<<<<<<<<<<
python-slugify 8.0.1 A Python slugify application that also handles Unicode
└── text-unidecode >=1.3
simplug 0.3.0 A simple plugin system for python with async hooks supported
└── diot >=0.1,<0.2
    └── inflection >=0.5,<0.6 
six 1.16.0 Python 2 and 3 compatibility utilities

Looks like python-simpleconf is the only dependency that relies on rtoml. Let's see if we could fall back to tomli or tomllib(introduced since python311) if rtoml is not available.

I will also submit an issue to the author of rtoml to see if there is a chance to build wheel for Window while publishing it.

But the repo hasn't been updated for quite a while, my PR has also been pending for a long time.

GitHunter0 · Answer 8 · Tue Jun 27 2023 08:35:56 GMT+0800 (China Standard Time)

Thank you @pwwang , now I was able to install datar after conda install -c conda-forge rust. I already had C++ build tools.

However, rust is large and takes a long time to install, so I hope this 3rd-party issue gest solved.

pwwang · Answer 9 · Tue Jun 27 2023 09:53:29 GMT+0800 (China Standard Time)

With v0.12.2, rtoml is skipped for python3.11, instead, tomllib (builtin with python 3.11) is used.

GitHunter0 · Answer 10 · Tue Jun 27 2023 10:06:49 GMT+0800 (China Standard Time)

@pwwang , your efficiency is second to none, my friend.