Session.verify ignored if REQUEST_CA_BUNDLES is set; behaviour not documented.
StefanKopieczek opened this issue · comments
Summary
Session-level CA overrides are ignored if either REQUEST_CA_BUNDLES
or CURL_CA_BUNDLES
is set.
This is unintuitive as you'd expect a per-session override to take precedence over global state. It's also not mentioned in the documentation.
Repro
The following script normally outputs '200', but instead fails with SSL verification errors if either of the above environment variables are set (because session.verify = False
gets ignored).
import requests
session = requests.Session()
session.verify = False
r = session.get('https://self-signed.badssl.com')
print(r.status_code)
Expected Result
I'd intuitively expect the repro above to return 200 regardless of the state of the environment variables.
Actual Result
If REQUESTS_CA_BUNDLE
or CURL_CA_BUNDLE
is set, the script above fails with verification errors, even though session.verify = False
.
Reproduction Steps
- Create a file
test.py
containing the script from the Summary section above. - Run
export REQUESTS_CA_BUNDLE=$(python3 -c "import certifi; print(certifi.where())")
- This sets
REQUESTS_CA_BUNDLE
to the system default truststore. - Any other valid value for
REQUESTS_CA_BUNDLE
would work here too.
- This sets
- Run
python3 test.py
and observe SSL validation errors.
System Information
$ python -m requests.help
{
"chardet": {
"version": "4.0.0"
},
"cryptography": {
"version": "3.4.8"
},
"idna": {
"version": "3.3"
},
"implementation": {
"name": "CPython",
"version": "3.10.12"
},
"platform": {
"release": "5.15.0-102-generic",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "30000020",
"version": "21.0.0"
},
"requests": {
"version": "2.25.1"
},
"system_ssl": {
"version": "30000020"
},
"urllib3": {
"version": "1.26.5"
},
"using_pyopenssl": true
}
I have also reproduced this on requests 2.31.0.
Other Impact
This behaviour also impacts other uses of session.verify
, such as session.verify = 'my_custom_ca_bundle.pem'
– if the environment variables are present, the custom CA bundle will not be used.
Proposed Fix
I'm happy to submit a docs PR to clarify this behaviour. In an ideal world I think session.verify
would just take precedence over the environment variables, but making that change might break consumers that are inadvertently relying on the current semantics – so I think a docs change is the best we can do.
If you do want the behavioural change, I'm also happy to submit a PR for that instead of the docs fix.
Thanks
I wanted to take this opportunity to say thanks for a fantastic library – I've been a happy user of requests
for over a decade and I really appreciate the hard work of all involved :)
Additional note for anyone affected: setting session.trust_env = False
works around this issue by disregarding the environment variables. However, this will also disable detection of any proxies declared in the environment.
Duplicate of #3829
In the future, please search closed and open issues before creating new ones that are duplicates.
Thanks @sigmavirus24 – I did search but didn't find it. I appreciate the redirect.