![Type a message to begin the conversation…/$ Hey, my name is Reimu. I maintain the barrier to Gensokyo./Ah, the legendary Reimu. I'm sure your barrier is the most powerful in Gensokyo. That's a lot of responsibility for someone your age!/$ it's not that much really /Anyway, I was thinking that I might try to visit Gensokyo someday. Do you think it will be okay if I just show up one day?/$ Probably fine./But there could be consequences to my actions. I don't know exactly how the barrier works./$ The barrier only affects living things and their creations./I'm glad to hear that. I'll try to plan my visit carefully then.](https://private-user-images.githubusercontent.com/6141784/241089462-da9cbf86-bc1c-499d-9d9e-7eff3a210696.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUwMzY4MDMsIm5iZiI6MTcyNTAzNjUwMywicGF0aCI6Ii82MTQxNzg0LzI0MTA4OTQ2Mi1kYTljYmY4Ni1iYzFjLTQ5OWQtOWQ5ZS03ZWZmM2EyMTA2OTYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgzMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MzBUMTY0ODIzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MjEyNzBjZDk5YmRmNzVjNzNlY2VkMmYwOGFjZGUyMjZhMDJhYmVmNDgwZTBmZmM1NzdiMWNmZWUxNzQxYzgxZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.MjKF30n8hVh7yJX0I-0d0JxsS8_5UYwCrKD9ODrt9lE)
Python script to demonstrate how to invoke models such as MPT-7B-Chat from the command-line.
All instructions are written assuming your command-line shell is bash.
Clone repository:
git clone https://github.com/Birch-san/mpt-play.git
cd mpt-play
This is to avoid interfering with your current Python environment (other Python scripts on your computer might not appreciate it if you update a bunch of packages they were relying on).
Follow the instructions for virtualenv, or conda, or neither (if you don't care what happens to other Python scripts on your computer).
Create environment:
. ./venv/bin/activate
pip install --upgrade pip
Activate environment:
. ./venv/bin/activate
(First-time) update environment's pip
:
pip install --upgrade pip
Download conda.
Skip this step if you already have conda.
Install conda:
Skip this step if you already have conda.
Assuming you're using a bash
shell:
# Linux installs Anaconda via this shell script. Mac installs by running a .pkg installer.
bash Anaconda-latest-Linux-x86_64.sh
# this step probably works on both Linux and Mac.
eval "$(~/anaconda3/bin/conda shell.bash hook)"
conda config --set auto_activate_base false
conda init
Create environment:
conda create -n p311-mpt python=3.11
Activate environment:
conda activate p311-mpt
Ensure you have activated the environment you created above.
(Optional) treat yourself to latest nightly of PyTorch, with support for Python 3.11 and CUDA 12.1:
# CUDA
pip install --upgrade --pre torch --extra-index-url https://download.pytorch.org/whl/nightly/cu121
# Mac
pip install --upgrade --pre torch --extra-index-url https://download.pytorch.org/whl/nightly/cpu
Install dependencies:
pip install -r requirements.txt
From root of repository:
python -m scripts.chat_play --trust_remote_code --bf16
On Mac you'll need to disable bfloat16. PYTORCH_ENABLE_MPS_FALLBACK=1
is not necessary in current PyTorch nightly, but there's no harm keeping it (and it may help on older PyTorch):
PYTORCH_ENABLE_MPS_FALLBACK=1 python -m scripts.chat_play --trust_remote_code
This repository is itself MIT-licensed.
Includes MIT-licensed code copied from Artidoro Pagnoni's qlora, and Apache-licensed code copied from MosaicML's mpt-7b-chat Huggingface Space.