allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

Home Page:https://clear.ml/docs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SSLError with file uploads

abfshaal opened this issue · comments

Describe the bug

When trying to upload files to clearml I am getting this error
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))
even tho I have the api.verify set to false

The clearml deployment I have is using the helm chart deployment, with the version of clearml 1.10.4 for server, and the source code for launching the data upload on version 1.10.3.

If I add a s3 storage, some part of the data upload works, and another starts showing the retrying error message
Retrying (Retry(total=2, connect=5, read=5, redirect=5, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))'

If I export the environment variable CURL_CA_BUNDLE to an empty string, it works, but is there any other better solution than this workaround ?

Hi,
having CURL_CA_BUNDLE set to empty is just a workaround to avoid certificate validation. Best practice is to use valid certificates, if in your case they are produced internally, you need to have CURL_CA_BUNDLE set to file containing CA bundle so system is able to trust it.