An API to interact with the GPT-J language model and variants! You can use and test the model in two different ways:
- Streamlit web app at http://api.vicgalle.net:8000/
- The proper API, documented at http://api.vicgalle.net:5000/docs
These are the endpoints of the public API and require no authentication. Click on each to see the parameters!
- generate :
POST /generate/
- classify :
POST /classify/
- Python:
import requests
context = "In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English."
payload = {
"context": context,
"token_max_length": 512,
"temperature": 1.0,
"top_p": 0.9,
}
response = requests.post("http://api.vicgalle.net:5000/generate", params=payload).json()
print(response)- Python (zero-shot classification):
import requests
payload = {
"sequence" : "The movie started slow, but in the end was absolutely amazing!",
"labels" : "positive,neutral,negative"}
response = requests.post("http://api.vicgalle.net:5000/classify", params=payload).json()
print(response)- Bash:
curl -X 'POST' \
'http://api.vicgalle.net:5000/generate?context=In%20a%20shocking%20finding%2C%20scientists%20discovered%20a%20herd%20of%20unicorns%20living%20in%20a%20remote%2C%20previously%20unexplored%20valley%2C%20in%20the%20Andes%20Mountains.%20Even%20more%20surprising%20to%20the%20researchers%20was%20the%20fact%20that%20the%20unicorns%20spoke%20perfect%20English.&token_max_length=512&temperature=1&top_p=0.9' \
-H 'accept: application/json' \
-d ''Just SSH into a TPU VM. This code was tested on both the v2-8 and v3-8 variants.
First, install the requirements and get the weights:
python3 -m pip install -r requirements.txt
wget https://the-eye.eu/public/AI/GPT-J-6B/step_383500_slim.tar.zstd
sudo apt install zstd
tar -I zstd -xf step_383500_slim.tar.zstd
rm step_383500_slim.tar.zstd
And just run
python3 serve.py
Then, you can go to http://localhost:5000/docs and use the API!
Just run
python3 -m streamlit run streamlit_app.py --server.port 8000
If you have a request, I'll be happy to help you at vgallegoalcala at gmail dot com
Special thanks to the following people, who sponsor this project! <3
Many thanks to the support of the TPU Research Cloud, https://sites.research.google/trc/