magic-lantern / offline-conda-mirror

Tool for downloading conda files for offline use on a different system

Offline Conda Mirror

Tool for downloading conda packages and all required dependencies for offline use on a different system. While this may work on Windows, it has only been tested on macOS and Linux.

Additionally, this requires Python 3.

Prerequisites

Install full conda environment on both source and destination systems

How to use

Suppose you want to setup a Conda environment on a new host with PyTorch, but the host doesn't have Internet access. Normally you would just run: conda install pytorch torchvision cudatoolkit=10.1 -c pytorch but that won't work due to outbound Internet not being available.

On machine with Internet access

Download this repository to a machine that has Internet access. Then run localmirror.py and provide appropriate arguments:

python ./localmirror.py --help

Usage: localmirror.py [OPTIONS] PACKAGES...

Options:
  --channel <CHANNEL NAME>        Note: multiple --channel options is
                                  supported
  --platform [linux-64|linux-32|osx-64|win-64|win-32]
                                  [default: linux-64]
  --target-directory <TARGET_DIR>
                                  [default: localmirror]
  --help                          Show this message and exit.

If you want to download for offline installation PyTorch, you would run the following: python ./localmirror.py --channel=pytorch pytorch torchvision cudatoolkit=10.1 "python >=3.7,<3.8.0a0" Note: Providing Python version may be helpful but is not always required.

On machine without Internet access

1. Copy all folders and files downloaded above to new host
2. Add this line to your /etc/hosts file (or modify existing localhost line)

   127.0.0.1    localhost repo.anaconda.com conda.anaconda.org
   

3. Add this line to your ~/.condarc file (create it if it doesn't already exist) - ssl_verify set to false as we are using a self-signed SSL certificate.

   ssl_verify: false
   

4. cd localmirror
5. which python # substitute your appropriate python including path in next line
6. sudo python ../https-server.py # this is required so can listen on port 443 as expected
7. conda create -n mynewenv python=3.7
8. conda activate mynewenv
9. conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

Notes about Offline pip Mirror

Pip is much easier to use offline. It has a built in command for downloading a package and all dependencies. You can either run:

pip download somepackage

Or you can run:

pip download -r requirements.txt

By default the download command will save to the current directory without installing the packages. If you move the download folder to an offline machine, you can then install with:

pip install --no-index --find-links /path/to/pipfiles/ somepackage

Or:

pip install --no-index --find-links /path/to/pipfiles/ -r requirements.txt

based on https://stackoverflow.com/questions/11091623/python-packages-offline-installation#14447068