--extra-index-url

[Sergey Rybakov]

Is it possible to build source distributions that depend on at least one package, which is not in public PyPI?

For example, I have a requirements.txt

numpy
requests

--extra-index-url https://test.pypi.org/simple/
some-package

When I'm doing python -m build --sdist I get

setuptools.extern.packaging.requirements.InvalidRequirement: Expected package name at the start of dependency specifier
    --extra-index-url https://test.pypi.org/simple/
    ^

I tried to set pip-args in envs and pass it directly as --extra-index-url=https://test.pypi.org/simple/ --trusted-host=https://test.pypi.org/simple/ but no result

Is there any way to do it?

[gl-yziquel]

You do not need such a flag. I build python code with locally deployed dependencies. The trick is to having correct configurations globally. For instance, in the ~/.pypirc file, you can have declarations like:

[distutils]
index-servers =
    local

[local]
repository = http://127.0.0.1:3141/root/local/

This is kind of the default setup you'd get with a locally installed devpi server.

Adapt to your specific personal needs.

[Agriya Khetarpal]

As a workaround, it should be possible to install (into a virtual environment) the private-indexed dependency first, and then run python -m build (inside the virtual environment) with the --no-isolation command-line flag (while pre-installing any other dependencies required at build-time).

i.e., this might work:

python3.X -m virtualenv venv
source venv/bin/activate
pip install cmake dependencyA dependencyB  # and so on
pip install private_dependency --extra-index-url "https://path.to.your/pvt/index/"
pip install build
python -m build --sdist --no-isolation --outdir PATH

you might need to make adjustments if you have a custom build_sdist command in setup.py (assuming that you're using setuptools as a build-backend).

P.S. It looks like PDM and Poetry directly support adding extra index URLs in pyproject.toml, according to this discussion), but setuptools has removed it in the past.

---

P.P.S. I tested this locally with the nightly wheels for SciPy:

This is what I modified my build-system table to look like:

[build-system]
requires = [
  "setuptools>=64",
  "pooch",
  "tqdm",
  "wheel==0.42.0",
  "scipy==1.13.0.dev0",
]

and these commands

pip install "setuptools>=64" pooch tqdm "wheel==0.42.0"
pip install "scipy==1.13.0.dev0" --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
pip install build
python -m build --sdist --no-isolation

work, as intended. Please note that I am suggesting the use of a virtual environment to counter the loss of an isolated build that you incur with the use of the --no-isolation flag; and that --index-url is preferable to --extra-index-url for security reasons (unless you explicitly require scourging through both indexes, that is)

[matsjoyce-refeyn]

Another workaround, setting the PIP_INDEX_URL (from https://pip.pypa.io/en/stable/cli/pip_list/#cmdoption-extra-index-url) also seems to work.