Unable to download packages from offline machine with configured proxy
toiletsandpaper opened this issue · comments
Checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pixi, using
pixi --version
.
Reproducible example
PS C:\Users\user\Desktop\pixi_test> pixi add mypy
WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
Added mypy >=1.11.1,<2
PS C:\Users\user\Desktop\pixi_test> pixi add pytorch
WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
x failed to solve the conda requirements of 'default' 'win-64'
`-> Cannot solve the request because of: No candidates were found for pytorch *.
PS C:\Users\user\Desktop\pixi_test> pixi add --pypi torch
WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
x failed to download pypi name mapping
|-> Middleware error: Request error: error sending request for url (https://conda-mapping.prefix.dev/hash-
| v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)
|-> Request error: error sending request for url (https://conda-mapping.prefix.dev/hash-
| v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)
|-> error sending request for url (https://conda-mapping.prefix.dev/hash-
| v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)
|-> client error (Connect)
|-> dns error: No such host is known. (os error 11001)
`-> No such host is known. (os error 11001)
PS C:\Users\user\Desktop\pixi_test> pixi config list
WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
tls-no-verify = true
detached-environments = false
[mirrors] "https://conda.anaconda.org/conda-forge" = [ "http://MY_MIRROR/repository/conda-forge-proxy", "http://MY_MIRROR/repository/anaconda-pytorch", "http://MY_MIRROR/repository/anaconda-nvidia",
]
[pypi-config]
index-url = "http://MY_MIRROR/repository/pypi-all/simple"
keyring-provider = "subprocess"
Issue description
So I have several local machines, that cutted out from internet, but have access to already cached packages with Sonatype Nexus.
When I configure my pdm/poerty/pip/conda configs with exact cacher urls that in example above - everything works right with every tool, but pixi rejects to work properly.
I followed step-by-step with this guide https://pixi.sh/v0.18.0/advanced/global_configuration/
BTW, WARN Ignoring 'repodata_options'
is very strange message, because you have that in your example.
For now, the only thing that working is installing packages from the first specified mirror for conda-forge
(ex. conda-forge-proxy
), like it worked for installing mypy
in the above example
Expected behavior
I expected that pixi will:
- fetch all specified mirrors in order, not only the first one
- fetch pypi packages even without
compressed_mappings
(btw, pip haveindex
andindex-url
in config)
The error is caused by the requirement to find our custom conda-pypi-mapping
This makes sure pixi won't override conda packages with pypi
packages when the same packages come from the solve.
You can selfhost or hardcode the mapping in your project. Documentation here: https://pixi.sh/dev/reference/project_configuration/#conda-pypi-map-optional
The mapping is located here: https://github.com/prefix-dev/parselmouth/blob/main/files/compressed_mapping.json
The WARN Ignoring
was a typo in the documentation it should have been repodata_options
or repodata-options
in the newer version. This is fixed in the latest documentation.
I don't really get it. How to config properly? I've downloaded mapping, then tried to add conda-pypi-mapping = {"conda-forge"}
and even with replacing "conda-forge"
with full link to "https://conda.anaconda.org/conda-forge"
, but still have this warn:
WARN Defined custom mapping channel https://conda.anaconda.org/conda-forge/ is missing from project channels
Can we just have config flag like disable-conda = true
for environments, when we don't use conda at all? Maybe then you don't even will be needed to check mappings... 👉👈
You can disable mapping with giving it an empty table:
conda-pypi-map = { }
The mapping was indeed broken. Not sure for how long but I fixed that in #1820
I've changed config to
conda-pypi-map = { }
and now I can install packages only from conda (btw, looks like not only from the first one mirror in list, but even from nvidia conda channel). Also pixi now panicking when I do add
/remove
some packages - but package itself is installed/uninstalled after this error as normal anyways.
But --pypi
packages now failing... all info in .txt below
Any info about how to make pypi work properly?
We'll have to dive a little deeper. I'm sorry for the inconvenience. You could manually make a pip install
task to to the pypi work untill we figure this out properly.