[BUG] ValueError: invalid variable 'py_version_nodot_plat'

Question

[BUG] ValueError: invalid variable 'py_version_nodot_plat'

Delgan opened this issue 3 years ago · comments

Delgan commented 3 years ago

setuptools version

setuptools==60.2.0

Python version

Python 3.10.1

OS

Windows

Additional environment information

No response

Description

Hi.

I have just faced an error appearing when setuptools.setup() is called. This error did not appear a month ago (code is unchanged). It works fine with Python < 3.10.

This is a CI Github action you can find here (setuptools==60.2.0 and Python 3.10.1): https://github.com/Delgan/win32-setctime/runs/4673761053?check_suite_focus=true

Compared to a month ago (setuptools==59.3.0 and Python 3.10.0): https://github.com/Delgan/win32-setctime/runs/4345756128?check_suite_focus=true

This Stackoverflow question is probably related: https://stackoverflow.com/questions/70472466/python3-valueerror-invalid-variable-py-version-nodot-plat
Workaround suggested is to install wheel (I didn't tested it) but I'm not sure why it's now required. I'm afraid it could break library installation for people not explicitly installing the wheel dependency.

Expected behavior

Installing the library shouldn't fail.

How to Reproduce

Fork https://github.com/Delgan/win32-setctime
Start Github Action tests workflow
- It will run first python -m pip install --user --upgrade pip setuptools
- It will then install the library with python -m pip install --user -e .[dev] based on this setup.py file
- The installation will fail with setup() raising KeyError and ValueError exceptions

Output

Requirement already satisfied: pip in c:\hostedtoolcache\windows\python\3.10.1\x64\lib\site-packages (21.3.1)
Requirement already satisfied: setuptools in c:\hostedtoolcache\windows\python\3.10.1\x64\lib\site-packages (58.1.0)
Collecting setuptools
  Downloading setuptools-60.2.0-py3-none-any.whl (953 kB)
Installing collected packages: setuptools
Successfully installed setuptools-60.2.0
Obtaining file:///D:/a/win32-setctime/win32-setctime
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pytest>=4.6.2
  Downloading pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting black>=19.3b0
  Downloading black-21.12b0-py3-none-any.whl (156 kB)
Collecting pathspec<1,>=0.9.0
  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting tomli<2.0.0,>=0.2.6
  Downloading tomli-1.2.3-py3-none-any.whl (12 kB)
Collecting mypy-extensions>=0.4.3
  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typing-extensions!=3.10.0.1
  Downloading typing_extensions-4.0.1-py3-none-any.whl (22 kB)
Collecting platformdirs>=2
  Downloading platformdirs-2.4.1-py3-none-any.whl (14 kB)
Collecting click>=7.1.2
  Downloading click-8.0.3-py3-none-any.whl (97 kB)
Collecting atomicwrites>=1.0
  Downloading atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB)
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting toml
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting packaging
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
Collecting colorama
  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting attrs>=19.2.0
  Downloading attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting py>=1.8.2
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.6-py3-none-any.whl (97 kB)
Installing collected packages: pyparsing, colorama, typing-extensions, tomli, toml, py, pluggy, platformdirs, pathspec, packaging, mypy-extensions, iniconfig, click, attrs, atomicwrites, win32-setctime, pytest, black
  Running setup.py develop for win32-setctime
    ERROR: Command errored out with exit status 1:
     command: 'C:\hostedtoolcache\windows\Python\3.10.1\x64\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'D:\\a\\win32-setctime\\win32-setctime\\setup.py'"'"'; __file__='"'"'D:\\a\\win32-setctime\\win32-setctime\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
         cwd: D:\a\win32-setctime\win32-setctime\
    Complete output (37 lines):
    running develop
    C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\command\easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    Traceback (most recent call last):
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\util.py", line 258, in subst_vars
        return _subst_compat(s).format_map(lookup)
    KeyError: 'py_version_nodot_plat'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "D:\a\win32-setctime\win32-setctime\setup.py", line 16, in <module>
        setup(
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\__init__.py", line 155, in setup
        return distutils.core.setup(**attrs)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\core.py", line 148, in setup
        return run_commands(dist)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\core.py", line 163, in run_commands
        dist.run_commands()
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
        self.run_command(cmd)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\dist.py", line 985, in run_command
        cmd_obj.ensure_finalized()
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\command\develop.py", line 52, in finalize_options
        easy_install.finalize_options(self)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\command\easy_install.py", line 273, in finalize_options
        self.expand_dirs()
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\command\easy_install.py", line 408, in expand_dirs
        self._expand_attrs(dirs)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\command\easy_install.py", line 390, in _expand_attrs
        val = subst_vars(val, self.config_vars)
      File "C:\Users\runneradmin\AppData\Roaming\Python\Python310\site-packages\setuptools\_distutils\util.py", line 260, in subst_vars
        raise ValueError(f"invalid variable {var}")
    ValueError: invalid variable 'py_version_nodot_plat'
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\hostedtoolcache\windows\Python\3.10.1\x64\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'D:\\a\\win32-setctime\\win32-setctime\\setup.py'"'"'; __file__='"'"'D:\\a\\win32-setctime\\win32-setctime\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

Code of Conduct

I agree to follow the PSF Code of Conduct

Jason R. Coombs · Answer 1 · Tue Jan 04 2022 00:00:15 GMT+0800 (China Standard Time)

Thanks for the report.

The reason the issue emerged with Setuptools 60 is because Setuptools 60 intentionally adopts its own local copy of distutils. The easiest quick fix is to disable that behavior with the escape hatch by setting SETUPTOOLS_USE_DISTUTILS=stdlib. However, that is only a temporary solution as distutils in stdlib is eventually going away.

The reported error message is indicating that one of the INSTALL_SCHEMES in that environment includes a {py_version_nondot_plat} substitution, but that variable is not indicated in any of the config vars.

We need to figure out where that value is defined and how it's meant to be satisfied.

Jason R. Coombs · Answer 2 · Tue Jan 04 2022 00:11:39 GMT+0800 (China Standard Time)

Aha, so it looks like that value was added to the install schemes in python/cpython@dd18001 for Python 3.10. Probably distutils needs to at the very least honor any config vars present in sysconfig but not in distutils. I'm transferring the issue there.

Delgan · Answer 3 · Wed Jan 12 2022 03:44:46 GMT+0800 (China Standard Time)

Hey @jaraco, thanks for the quick fix. Unfortunately, I noticed the error persists with setuptools==60.5.0: https://github.com/Delgan/win32-setctime/runs/4750681694?check_suite_focus=true

How is that possible?

Jason R. Coombs · Answer 4 · Wed Jan 12 2022 06:31:46 GMT+0800 (China Standard Time)

I think it's possible I haven't merged these changes into Setuptools. The easiest way to check is to click on the commit, then replace distutils with setuptools (pypa/setuptools@1560a1f) and then look at the tags, which I see includes 60.3.0, so it is included in the releases.

I see now that the issue you're seeing is in easy_install, which in setuptools-specific and has its own copy of config_vars. :(. I guess I'll open a new issue with Setuptools.

There are just so many forks of distutils that even Setuptools has more than one.