Use of `MmapedDict` results in fd leaks (resource leaks)
mgorny opened this issue · comments
When running the test suite with -Wdefault, I get multiple reports of resource leaks, plus one of the tests is flaky and sometimes fails because of ResourceWarning being emitted where UserWarning was expected first:
$ python -m pytest -Wdefault
========================================================= test session starts =========================================================
platform linux -- Python 3.11.8, pytest-8.0.0, pluggy-1.4.0
rootdir: /tmp/client_python
collected 309 items
tests/openmetrics/test_exposition.py .................... [ 6%]
tests/openmetrics/test_parser.py ............................................. [ 21%]
tests/test_asgi.py ssssssss [ 23%]
tests/test_core.py .................................................................................................... [ 55%]
tests/test_exposition.py .......................................................... [ 74%]
tests/test_gc_collector.py .. [ 75%]
tests/test_graphite_bridge.py ....... [ 77%]
tests/test_multiprocess.py .....................F....... [ 87%]
tests/test_parser.py ........................ [ 94%]
tests/test_platform_collector.py .. [ 95%]
tests/test_process_collector.py .... [ 96%]
tests/test_samples.py .. [ 97%]
tests/test_twisted.py s [ 97%]
tests/test_wsgi.py ....... [100%]
============================================================== FAILURES ===============================================================
_____________________________________________ TestMultiProcess.test_remove_clear_warning ______________________________________________
self = <tests.test_multiprocess.TestMultiProcess testMethod=test_remove_clear_warning>
def test_remove_clear_warning(self):
os.environ['PROMETHEUS_MULTIPROC_DIR'] = self.tempdir
with warnings.catch_warnings(record=True) as w:
values.ValueClass = get_value_class()
registry = CollectorRegistry()
collector = MultiProcessCollector(registry)
counter = Counter('c', 'help', labelnames=['label'], registry=None)
counter.labels('label').inc()
counter.remove('label')
counter.clear()
assert os.environ['PROMETHEUS_MULTIPROC_DIR'] == self.tempdir
> assert issubclass(w[0].category, UserWarning)
E AssertionError: assert False
E + where False = issubclass(<class 'ResourceWarning'>, UserWarning)
E + where <class 'ResourceWarning'> = <warnings.WarningMessage object at 0x7f22956b82d0>.category
tests/test_multiprocess.py:395: AssertionError
========================================================== warnings summary ===========================================================
tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
/usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=37, family=2, type=1, proto=0, laddr=('127.0.0.1', 40145)>
for b in _UNSAFE_URL_BYTES_TO_REMOVE:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
/usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=38, family=2, type=1, proto=0, laddr=('127.0.0.1', 46167)>
for b in _UNSAFE_URL_BYTES_TO_REMOVE:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_collect
<frozen os>:676: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp9on9bz25/counter_1334413.db' mode='ab+' closefd=True>
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/gauge_all_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/histogram_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/gauge_all_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/histogram_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpx3sjgfk6/counter_1.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_123.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
/usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_456.db' mode='ab+' closefd=True>
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_123.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
/tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_456.db' mode='ab+' closefd=True>
with open(filename, 'rb') as infp:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_expansion
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmptd9lerf6'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_multi_expansion
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmp4ho56w3t'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_process_restart
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpp3siphvy'>
self._obj = None
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_multiprocess.py::TestMmapedDict::test_process_restart
/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:546: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpqi5kklpc'>
fin()
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_parser.py::TestParse::test_untyped
/tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_123.db' mode='ab+' closefd=True>
for sa, sb in zip(a.samples, b.samples):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_parser.py::TestParse::test_untyped
/tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_456.db' mode='ab+' closefd=True>
for sa, sb in zip(a.samples, b.samples):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= short test summary info =======================================================
FAILED tests/test_multiprocess.py::TestMultiProcess::test_remove_clear_warning - AssertionError: assert False
======================================== 1 failed, 299 passed, 9 skipped, 32 warnings in 4.10s ========================================An example tracemalloc output:
$ PYTHONTRACEMALLOC=20 python -m pytest -Wdefault
[…]
tests/test_parser.py::TestParse::test_untyped
/usr/lib/python3.11/typing.py:362: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpyaxyl9i6/summary_456.db' mode='ab+' closefd=True>
return cached(*args, **kwds)
Object allocated at:
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 114
runtestprotocol(item, nextitem=nextitem)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 133
reports.append(call_and_report(item, "call", log))
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 226
call = call_runtest_hook(item, when, **kwds)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 265
return CallInfo.from_call(
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 345
result: Optional[TResult] = func()
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 266
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 501
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 119
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 102
res = hook_impl.function(*args)
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 173
item.runtest()
File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py", line 333
self._testcase(result=self) # type: ignore[arg-type]
File "/usr/lib/python3.11/unittest/case.py", line 678
return self.run(*args, **kwds)
File "/usr/lib/python3.11/unittest/case.py", line 623
self._callTestMethod(testMethod)
File "/usr/lib/python3.11/unittest/case.py", line 579
if method() is not None:
File "/tmp/client_python/tests/test_multiprocess.py", line 79
s2 = Summary('s', 'help', registry=None)
File "/tmp/client_python/prometheus_client/metrics.py", line 151
self._metric_init()
File "/tmp/client_python/prometheus_client/metrics.py", line 513
self._count = values.ValueClass(self._type, self._name, self._name + '_count', self._labelnames,
File "/tmp/client_python/prometheus_client/values.py", line 68
self.__reset()
File "/tmp/client_python/prometheus_client/values.py", line 82
files[file_prefix] = MmapedDict(filename)
File "/tmp/client_python/prometheus_client/mmap_dict.py", line 64
self._f = open(filename, 'rb' if read_mode else 'a+b')This is with 0.20.0 (7a80f00).