ResourceWarning: unclosed <ssl.SSLSocket...
jedie opened this issue · comments
I get some warnings about unclosed SSL socket, looks like:
...
.../utilities/test_utils.py:110: in __exit__
self.vcr_mock.__exit__(exc_type, exc_val, exc_tb)
site-packages/vcr/cassette.py:105: in __exit__
next(self.__finish, None)
site-packages/vcr/cassette.py:65: in _patch_generator
with contextlib.ExitStack() as exit_stack:
/usr/local/lib/python3.10/contextlib.py:561: in __exit__
if cb(*exc_details):
site-packages/vcr/patch.py:402: in __exit__
connection = pool.pool.get()
/usr/local/lib/python3.10/warnings.py:109: in _showwarnmsg
sw(msg.message, msg.category, msg.filename, msg.lineno,
.../settings/warning_handler.py:47: in __call__
frames = traceback.extract_stack()
WARNING from "test_sdk_client.py" in test_get_category_by_key():
"/usr/local/lib/python3.10/contextlib.py ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=...>"
Any advise to fix that?
Hi @jedie there are many details missing from your report. Python version seems to be 3.10, version of vcrpy is unclear, user level code doing the requests is unclear, could be requests, urllib3 v1 or v2 etc, is this behavior new or does it happen across all versions of vcrpy etc. Please add more details.
Sorry, here Version information:
- Python 3.10
- VCR.py v4.3.1
- requests v2.31.0
- urllib3 v1.26.16
Thanks! Is this behavior new or does it happen across all versions of vcrpy?
File test_sdk_client.py
was making requests using requests
directly or with some layer in between? If so which one?
I think vcrpy does not open SSL connections by itself so I would assume it's either a bug in vcrpy's connection pool code or the library outside of vcrpy that does open SSL connections. I'm guessing here, any help with debugging is appreciated.
Three weeks without a reproducer, closing as "cannot reproduce". Happy to re-open if someone can provide a minimal producible example. Thanks!
Sorry, i have currently no time to investigate this further... I currently just ignore the warning ;)
Hi @hartwork,
This repo created by @mezhaka has a minimal reproducible example with pytest and here is an even simpler one without pytest:
import warnings
import requests
import vcr
def main():
session = requests.Session()
with vcr.use_cassette(path="/dev/null"):
session.get("http://neverssl.com")
# session.close()
if __name__ == "__main__":
warnings.filterwarnings("always", category=ResourceWarning)
main()
In Python 3.8, with vcrpy 5.1.0 I get a sys:1: ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.1.114', 64833), raddr=('34.223.124.45', 80)>
. Even closing the session explicitly with session.close()
will not stop the ResourceWarning
.
here is an even simpler one without pytest:
I've been trying to make this one fail locally to try and fix this (and also provide a new unit test for vcrpy) but can't seem to do so. Is there anything else that needs installing? It just seems to work with no failure for me.
I haven't seen this error for a while. So i assume it's fixed. But i have no more details, sorry.
EDIT: I still use vcrpy only with normal unittests (with Django) and not via pytest. Works fine...
I'm able to reproduce this reliably with current vcrpy master. Adding the connection.close()
line from #758 resolves the issue.
Now trying to isolate a test case that isn't my entire closed-source app...
Reproduction steps
- Checkout current vcrpy master
pip install pytest-vcr
- Dump the following into "test.py"
import warnings
import pytest
import requests
@pytest.mark.vcr
def test_main():
warnings.filterwarnings("error", category=ResourceWarning)
session = requests.Session()
session.get("http://neverssl.com")
rm -Rf cassettes && pytest test.py -vvv -s
That last step reliably gets the following
platform linux -- Python 3.8.14, pytest-7.4.0, pluggy-1.2.0 -- /home/palfrey/.virtualenvs/vcrpy/bin/python
cachedir: .pytest_cache
rootdir: /home/palfrey/src/vcrpy
configfile: pyproject.toml
plugins: vcr-1.0.2, cov-4.1.0, httpbin-2.0.0
collected 1 item
test.py::test_main PASSED
================================================================ warnings summary =================================================================
test.py::test_main
/home/palfrey/.virtualenvs/vcrpy/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <socket.socket fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
Traceback (most recent call last):
File "/home/palfrey/.pyenv/versions/3.8.14/lib/python3.8/contextlib.py", line 510, in __exit__
if cb(*exc_details):
ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.25', 52936), raddr=('34.223.124.45', 80)>
warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
The pytest-vcr
could be the issue, but given the connection.close()
patch seems to fix this, I doubt it!
here is an even simpler one without pytest:
I've been trying to make this one fail locally to try and fix this (and also provide a new unit test for vcrpy) but can't seem to do so. Is there anything else that needs installing? It just seems to work with no failure for me.
Here are the steps to reproduce the problem in a container to make sure you have exactly the same environment:
$ docker run --rm -it python:3.11 bash
root@container:/# pip install vcrpy==5.1.0 requests==2.31.0
root@container:/# cat > test.py
# copy/paste the code from my previous comment (https://github.com/kevin1024/vcrpy/issues/710#issuecomment-1667240075) and press ctrl-d
root@container:/# python test.py
sys:1: ResourceWarning: unclosed <socket.socket fd=3, family=2, type=1, proto=6, laddr=('172.17.0.2', 50746), raddr=('34.223.124.45', 80)>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Looks like this was reproduced and there's a PR to fix it. I'm also having the same issue on my test suite