Unable to use custom certs with OpenAI client
rnag opened this issue · comments
I'm unable to get custom cert bundle path (client side) working with OpenAI. For background, I'm on a work laptop which is behind a VPN, so we use a custom SSL cert (.pem file) for any outbound requests. The Python requests library seems to honor this with the use of the REQUESTS_CA_BUNDLE env variable, but was unable to get it working with the OpenAI client.
Here's what I tried so far:
import os
import openai
print(os.environ['REQUESTS_CA_BUNDLE']) # confirm that it prints out correct path to .pem file
openai.verify_ssl_certs = False # disable verify but this does nothing, only logs a warning
openai.api_key = 'my-api-key'
file = '/path/to/my/file'
with open(file, 'rb') as in_file:
openai.File.create(
file=in_file, purpose='search')
This results in the below error being printed to console:
/Users/rnag/path/to/pem/file
/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py:47: UserWarning: verify_ssl_certs is ignored; openai always verifies.
warnings.warn("verify_ssl_certs is ignored; openai always verifies.")
Traceback (most recent call last):
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connection.py", line 416, in connect
self.sock = ssl_wrap_socket(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 1070, in _create
self.do_handshake()
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 783, in urlopen
return self.urlopen(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 783, in urlopen
return self.urlopen(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/files (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 255, in request_raw
result = _thread_context.session.request(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/files (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/path/to/py/file", line 11, in <module>
openai.File.create(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_resources/file.py", line 43, in create
response, _, api_key = requestor.request("post", url, files=files)
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 98, in request
result = self.request_raw(
File "/Users/rnag/.pyenv/versions/3.10.0/lib/python3.10/site-packages/openai/api_requestor.py", line 266, in request_raw
raise error.APIConnectionError("Error communicating with OpenAI") from e
openai.error.APIConnectionError: Error communicating with OpenAI
My workaround
Its not ideal but current workaround I'm using is to manually update the api_requestor.py script to pass in verify=False to request method:
timeout=TIMEOUT_SECS,
verify=False,
Hi @rnag! Thanks for filing this issue.
As a general security rule we'd very much prefer to verify all requests which is why we don't allow setting verify=False.
It should be possible to connect to our API with just a curl command (https://beta.openai.com/docs/api-reference/making-requests) or just a standalone requests.post. Can you try either of those with your certs and see if the issue persists?
I'm going to close this for now. Feel free to reopen!
So after further testing, it looks like I was running into the same issue with requests library, but not with curl. I realized that the issue, for whatever reason, was that I was exporting REQUESTS_CA_BUNDLE in my env to point to the path of my client cert (pem) file. I am exporting the same path with other env variables like AWS_CA_BUNDLE which is needed for the CLI, and have had no issues with it so far.
For some reason requests and openai client in python didn't seem to like it when the REQUESTS_CA_BUNDLE was set. Once I unset this variable, everything started magically working, and the cert errors disappeared on my end. So I would agree that this issue should (hopefully) be resolved now.