use_scm_version sometimes used, sometimes not

[Danek Duvall]

I've been struggling for a day to figure out why in some cases, a custom setup.py with a call to setup(use_scm_version...), the custom version function I have is called, and others, it's ignored. I'm sure it'll be obvious, but I'm stumped.

If I run python -m setuptools_scm, it looks like setup.py is never even loaded, so I've only been able to get it to work from an installer (I'm using uv, but pip seems to have the same behavior). Unfortunately, the installer creates an ephemeral isolated virtual environment, so I can't tweak the code in either setuptools or setuptools_scm to help debug.

It works in my primary development environment on my laptop, as well as in a freshly cloned repo on the same machine, even in a different shell. I also created a fresh repo with just the bare minimum of things in it for the installer to run, and it works there as well. Presumably this means there's something installed on this machine that's making it work.

It works in GitLab CI, in a job using nikolaik/python-nodejs:python3.10-nodejs20-bullseye as the image (Debian based), as well in this image in a container on my laptop (the container is x86_64, the laptop is apple silicon).

It fails in GitLab CI, in a job using ubuntu:22.04, as well as a VM of my own running the same.

I've tried to eliminate all the variables between different experiments, so I'm reasonably confident that the difference lies in what's installed on the machine, but that's pretty big and I don't know where to start to narrow that down.

Can anyone suggest what my next step should be?

[Danek Duvall]

Because I'm sure I'll be asked, here's the pyproject.toml for the simplified test case:

[build-system]
requires = ["setuptools>=64.0", "setuptools_scm>=8"]
build-backend = "setuptools.build_meta"

[project]
name = "ss-test"
dynamic = ["version"]

[tool.setuptools_scm]

And here's setup.py:

with open("/tmp/debug-output", "w") as out:
    print("entered setup.py", file=out)

from traceback import print_stack

from setuptools import setup
from setuptools_scm import ScmVersion

def myversion_func(version: ScmVersion) -> str:
    out = open("/tmp/debug-output", "a")
    print("v" + "=" * 70 + "v", file=out)
    print_stack(file=out)
    print("^" + "=" * 70 + "^", file=out)

    return "15"

with open("/tmp/debug-output", "a") as out:
    print("about to call setup", file=out)
    print("v" + "=" * 70 + "v", file=out)
    print_stack(file=out)
    print("^" + "=" * 70 + "^", file=out)

setup(use_scm_version={"version_scheme": myversion_func, "local_scheme": lambda v: ""})

with open("/tmp/debug-output", "a") as out:
    print("called setup", file=out)

I also meant to note that git on my laptop is 2.39.3, on the image where it works it's 2.30.2, and 2.34.1 on my VM where it doesn't.

[Ronny Pfannschmidt]

This is likely a bug in older setuptools-scm

Those would override the version in the hook for the toml configuration

If you don't set extra values in the toml configuration I recommend using only the setup.py configuration

[Ronny Pfannschmidt]

Also output using SETUPTOOLS_SCM_DEBUG=True would help

[Danek Duvall]

This is likely a bug in older setuptools-scm

It's 8.0.4 in both cases.

Those would override the version in the hook for the toml configuration

If you don't set extra values in the toml configuration I recommend using only the setup.py configuration

I don't know what that would look like. Just get rid of the empty [tool.setuptools_scm] section?

Also output using SETUPTOOLS_SCM_DEBUG=True would help

I'd neglected to discover that while uv won't allow stdout and stderr through, pip will, with -v. I've created a gist with both the working run and the non-working run. The working run is from my Mac, and has been lightly processed to change Users to home.

Looks like the working version logs this line first:

DEBUG setuptools_scm._integration.setuptools version_keyword { ... }

and later logs

DEBUG setuptools_scm._integration.setuptools infer_version { ... }

while the non-working version logs do the reverse.

Also, the Configuration object in the working version sets version_scheme to my function, while in the non-working version it's set to guess-next-dev.

[Ronny Pfannschmidt]

This may relate to setuptools then, I wasn't aware the order of those hooks can change, yay new bug

Drop of the empty section should ensure only the version keyword hook happens

I'll investigate a fix later

Thanks for the details

[Danek Duvall]

Thanks; that's working great for me for now!

(Also, I finally discovered python setup.py --version as a test; much easier and quicker than running through an installer, and always displays stdout and stderr.)

[Mathieu Lafon]

Same issue here where infer_version is sometimes executed before version_keyword.

setuptools supports an order attribute on hook functions to influence the order of execution. As a quick workaround, adding infer_version.order=1 at the end of setuptools_scm/_integration/setuptools.py can be used to avoid the issue.

[Ronny Pfannschmidt]

The order depends on legacy call vs build backend and setuptools version

I have a rough idea for solving it but I'll have to work with the setuptools team