openai.error.APIError while running client API example

[googlebleh]

Issue Description

openai.error.APIError while trying to run the client API example

Expected Result

similar output to the client API example

How to Reproduce

$ python
Python 3.10.10 (main, Mar  5 2023, 22:26:53) [GCC 12.2.1 20230201] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import openai
>>> openai.api_key = 'dummy'
>>> openai.api_base = "http://127.0.0.1:5000/v1"
>>> result = openai.Completion.create(model='codegen', prompt='def hello', max_tokens=16, temperature=0.1, stop=["\n\n"])
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/openai/api_requestor.py", line 331, in handle_error_response
    error_data = resp["error"]
KeyError: 'error'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/openai/api_resources/completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/lib/python3.10/site-packages/openai/api_requestor.py", line 226, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/lib/python3.10/site-packages/openai/api_requestor.py", line 619, in _interpret_response
    self._interpret_response_line(
  File "/usr/lib/python3.10/site-packages/openai/api_requestor.py", line 682, in _interpret_response_line
    raise self.handle_error_response(
  File "/usr/lib/python3.10/site-packages/openai/api_requestor.py", line 333, in handle_error_response
    raise error.APIError(
openai.error.APIError: Invalid response object from API: '{"detail":[{"loc":["body","model"],"msg":"string does not match regex \\"^(fastertransformer|py-model)$\\"","type":"value_error.str.regex","ctx":{"pattern":"^(fastertransformer|py-model)$"}}]}' (HTTP response code was 422)
>>>

Further Information

python-openai version 0.27.2

$ cat .env 
NUM_GPUS=1
GPUS=0
API_EXTERNAL_PORT=5000
TRITON_HOST=127.0.0.1
TRITON_PORT=8001
MODEL=py-codegen-2B-multi
MODEL_DIR=/home/user/repos/fauxpilot/models/py-Salesforce-codegen-2B-multi
HF_CACHE_DIR=/home/user/repos/fauxpilot/huggingface_cache

[github-actions]

Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!

[zhanglistar]

I had the same issue, try this: curl -s -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{"prompt":"def hello","max_tokens":100,"temperature":0.1,"stop":["\n\n"], "model":"py-model"}' http://localhost:5000/v1/engines/codegen/completions

[Sergio Paganoni]

Hello, I'm experiencing the exact same issue.
I tested openai from version 0.27.2 down to 0.25.0 thinking that was maybe an API change causing the issue.
(curl example worked)

[Kirill Rd]

I get the same apiError error 422, but other issues also come up. If I run this curl command, I get :

{"id": "cmpl-Oge64DTOCTupyIgi5rEYbjH78WoP2", "choices": []}

When I check fauxpilot log, I get:

fauxpilot-triton-1         | I0328 04:48:30.340278 94 grpc_server.cc:4587] Started GRPCInferenceService at 0.0.0.0:8001
fauxpilot-triton-1         | I0328 04:48:30.340495 94 http_server.cc:3303] Started HTTPService at 0.0.0.0:8000
fauxpilot-triton-1         | I0328 04:48:30.381505 94 http_server.cc:178] Started Metrics Service at 0.0.0.0:8002
fauxpilot-copilot_proxy-1  | INFO:     2023-03-28 04:48:45,661 :: 172.20.0.1:37538 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity
fauxpilot-copilot_proxy-1  | [StatusCode.UNAVAILABLE] failed to connect to all addresses
fauxpilot-copilot_proxy-1  | WARNING: Model 'py-model' is not available. Please ensure that `model` is set to either 'fastertransformer' or 'py-model' depending on your installation
fauxpilot-copilot_proxy-1  | Returned completion in 2.7740001678466797 ms
fauxpilot-copilot_proxy-1  | INFO:     2023-03-28 04:49:14,155 :: 172.20.0.1:37540 - "POST /v1/engines/codegen/completions HTTP/1.1" 200 OK

I've made only one modification to the docker-compose.yml, changing external mapping of 8000 port to something else, since my 8000 port is already taken.

[Boris Brangeon]

Hi,
I test model="fastertransformer" 👍

prompt="def hello"
response = openai.Completion.create(
        model="fastertransformer",
        prompt=prompt,
        temperature=0.1,
        max_tokens=256,
        top_p=1,
        stop=["\n\n"]
    )
choices = response.choices

[CodingmanJC]

I get the same apiError error 422, but other issues also come up. If I run this curl command, I get :

{"id": "cmpl-Oge64DTOCTupyIgi5rEYbjH78WoP2", "choices": []}

When I check fauxpilot log, I get:

fauxpilot-triton-1         | I0328 04:48:30.340278 94 grpc_server.cc:4587] Started GRPCInferenceService at 0.0.0.0:8001
fauxpilot-triton-1         | I0328 04:48:30.340495 94 http_server.cc:3303] Started HTTPService at 0.0.0.0:8000
fauxpilot-triton-1         | I0328 04:48:30.381505 94 http_server.cc:178] Started Metrics Service at 0.0.0.0:8002
fauxpilot-copilot_proxy-1  | INFO:     2023-03-28 04:48:45,661 :: 172.20.0.1:37538 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity
fauxpilot-copilot_proxy-1  | [StatusCode.UNAVAILABLE] failed to connect to all addresses
fauxpilot-copilot_proxy-1  | WARNING: Model 'py-model' is not available. Please ensure that `model` is set to either 'fastertransformer' or 'py-model' depending on your installation
fauxpilot-copilot_proxy-1  | Returned completion in 2.7740001678466797 ms
fauxpilot-copilot_proxy-1  | INFO:     2023-03-28 04:49:14,155 :: 172.20.0.1:37540 - "POST /v1/engines/codegen/completions HTTP/1.1" 200 OK

I've made only one modification to the docker-compose.yml, changing external mapping of 8000 port to something else, since my 8000 port is already taken.

I also find this problem。Did you find some solution to solve it？

[Kirill Rd]

Nope