kunaldeo / vertexai-palm-vscode-endpoint-server

vertexai palm server for huggingface-vscdoe custom endpoint

Vertex AI Palm VSCode Endpoint Server

Vertex AI Palm VSCode server for huggingface-vscdoe custom endpoint.

Usage

pip install -r requirements.txt
python main.py --project_id <gcp project name>

Fill http://localhost:8000/api/generate/ into Hugging Face Code > Model ID or Endpoint in VSCode.

If you are using Palm api you may want to change the following settings as well in VSCode's settings.json file:

"HuggingFaceCode.endToken": "",
"HuggingFaceCode.middleToken": "",
"HuggingFaceCode.stopToken": "",
"HuggingFaceCode.startToken": "",

API

curl -X POST http://localhost:8000/api/generate/ -d '{"inputs": "", "parameters": {"maxOutputTokens": 64}}'
# response = {"generated_text": ""}