Unable to download packages from offline machine with configured proxy

[Ilya Lubenets]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

PS C:\Users\user\Desktop\pixi_test> pixi add mypy
 WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
 WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
Added mypy >=1.11.1,<2

PS C:\Users\user\Desktop\pixi_test> pixi add pytorch
 WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
 WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
  x failed to solve the conda requirements of 'default' 'win-64'
  `-> Cannot solve the request because of: No candidates were found for pytorch *.

PS C:\Users\user\Desktop\pixi_test> pixi add --pypi torch
 WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
 WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
  x failed to download pypi name mapping
  |-> Middleware error: Request error: error sending request for url (https://conda-mapping.prefix.dev/hash-
  |   v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)                                                
  |-> Request error: error sending request for url (https://conda-mapping.prefix.dev/hash-                                
  |   v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)                                                
  |-> error sending request for url (https://conda-mapping.prefix.dev/hash-                                               
  |   v0/0fce54f8ec3e59f5ef3bb7641863be4e1bf1279623e5af3d3fa726e8f7628ddb)                                                
  |-> client error (Connect)                                                                                              
  |-> dns error: No such host is known. (os error 11001)                                                                  
  `-> No such host is known. (os error 11001)                                                                                                                                                                                                   
PS C:\Users\user\Desktop\pixi_test> pixi config list
 WARN Ignoring 'repodata_options' in at C:\Users\user\Desktop\pixi_test\.pixi\config.toml
tls-no-verify = true
detached-environments = false
[mirrors]                                                                                                                   "https://conda.anaconda.org/conda-forge" = [                                                                                    "http://MY_MIRROR/repository/conda-forge-proxy",                                                                                 "http://MY_MIRROR/repository/anaconda-pytorch",                                                                                  "http://MY_MIRROR/repository/anaconda-nvidia",                                                                           
]                                                                                                                                                                                                                                               
[pypi-config]
index-url = "http://MY_MIRROR/repository/pypi-all/simple"
keyring-provider = "subprocess"

Issue description

So I have several local machines, that cutted out from internet, but have access to already cached packages with Sonatype Nexus.
When I configure my pdm/poerty/pip/conda configs with exact cacher urls that in example above - everything works right with every tool, but pixi rejects to work properly.

I followed step-by-step with this guide https://pixi.sh/v0.18.0/advanced/global_configuration/
BTW, WARN Ignoring 'repodata_options' is very strange message, because you have that in your example.

For now, the only thing that working is installing packages from the first specified mirror for conda-forge (ex. conda-forge-proxy), like it worked for installing mypy in the above example

Expected behavior

I expected that pixi will:

1. fetch all specified mirrors in order, not only the first one
2. fetch pypi packages even without compressed_mappings (btw, pip have index and index-url in config)

[Ruben Arts]

The error is caused by the requirement to find our custom conda-pypi-mapping This makes sure pixi won't override conda packages with pypi packages when the same packages come from the solve.

You can selfhost or hardcode the mapping in your project. Documentation here: https://pixi.sh/dev/reference/project_configuration/#conda-pypi-map-optional

The mapping is located here: https://github.com/prefix-dev/parselmouth/blob/main/files/compressed_mapping.json

The WARN Ignoring was a typo in the documentation it should have been repodata_options or repodata-options in the newer version. This is fixed in the latest documentation.

[Ilya Lubenets]

I don't really get it. How to config properly? I've downloaded mapping, then tried to add conda-pypi-mapping = {"conda-forge"} and even with replacing "conda-forge" with full link to "https://conda.anaconda.org/conda-forge", but still have this warn:

WARN Defined custom mapping channel https://conda.anaconda.org/conda-forge/ is missing from project channels

pixi_config_list_cat_piped.txt
pixi.txt

[Ilya Lubenets]

Can we just have config flag like disable-conda = true for environments, when we don't use conda at all? Maybe then you don't even will be needed to check mappings... 👉👈

[Ruben Arts]

You can disable mapping with giving it an empty table:

conda-pypi-map = { }

The mapping was indeed broken. Not sure for how long but I fixed that in #1820

[Ilya Lubenets]

I've changed config to

conda-pypi-map = { }

and now I can install packages only from conda (btw, looks like not only from the first one mirror in list, but even from nvidia conda channel). Also pixi now panicking when I do add/remove some packages - but package itself is installed/uninstalled after this error as normal anyways.

But --pypi packages now failing... all info in .txt below

pixi.txt

[Ilya Lubenets]

Any info about how to make pypi work properly?

[Ruben Arts]

We'll have to dive a little deeper. I'm sorry for the inconvenience. You could manually make a pip install task to to the pypi work untill we figure this out properly.