This is a dockerized environment for running LLM experiments w/ the OpenAI API and locally w/ an Nvidia CUDA GPU. It mounts a single GPU and the following folders:
/data/docker/llm-experiments/data/ai/models
See this related LLM Experiments worksheet for related data: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit#gid=741531996
- List of accessible foundational models
- List of fine-tunes
- List of Datasets
lm-evalresults- Some ChatGPT bencmark tests
Also, here are some other prior work:
- Practical LLMs - results from poking around w/ self-hosted LLMs
- AI Safety - my research on AI Ethics and Safety (including AGI xrisk)
- Docker and Docker Compose
Setup:
git clone https://github.com/AUGMXNT/llm-experiments.git
cd llm-experiments
# Modify the volume paths, UID/GUIDs in the Dockerfile and docker-compose.yml
docker-compose build
cp env.sample .env
# and update w/ your keys
# This script finds the instance and runs `docker-compose up -d` if necessary
./connect_docker_instance.sh
You should now be inside the Docker container with access to your mounted directories, necessary tools, and environment variables. conda and mamba are pre-installed and you should be in the (base) venv.
We do this on the initial build, but to run the code, you'll probably want to run as necessary:
pip install -r requirements.txt
If you make any changes to the Dockerfile or docker-compose.yml, you can update the Docker image by running:
docker-compose build
docker-compose up -d