Memory corruption when using the library

[Andrew Svetlov]

Hi.
I have a bug report for aiohttp.
Looks like aiodns has a subtle bug: aio-libs/aiohttp#3573
I cannot reproduce the problem but the reporter provided a stack trace which is weird:

/aiohttp-1.py:38: DeprecationWarning: The object should be created from async function
  resolver = AsyncResolver(nameservers=["114.114.114.114"])
/aiohttp-1.py:41: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
  async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) assession:
Traceback (most recent call last):
  File "/aiohttp-1.py", line 51, in <module>
    asyncio.run(dnstesting())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/aiohttp-1.py", line 42, in dnstesting
    r = await session.get("http://google.com")
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/client.py", line 476, in _request
    timeout=real_timeout
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 522, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 854, in _create_connection
    req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 955, in _create_d/aiohttp-1.py:38: DeprecationWarning: The object should be created from async function
  resolver = AsyncResolver(nameservers=["114.114.114.114"])
/aiohttp-1.py:41: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
  async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) assession:
Traceback (most recent call last):
  File "/aiohttp-1.py", line 51, in <module>
    asyncio.run(dnstesting())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/aiohttp-1.py", line 42, in dnstesting
    r = await session.get("http://google.com")
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/client.py", line 476, in _request
    timeout=real_timeout
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 522, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 854, in _create_connection
    req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 955, in _create_direct_connection
    traces=traces), loop=self._loop)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 788, in _resolve_host
    host, port, family=self._family))
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/resolver.py", line 64, in resolve
    resp = await self._resolver.gethostbyname(host, family)
RuntimeError: Task <Task pending coro=<TCPConnector._resolve_host() running at /pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py:788> cb=[shield.<locals>._done_callback() at /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py:776]> got Future <Future pending> attached to a different loop
python3(38437,0x107b185c0) malloc: *** error for object 0x9000000000000000: pointer being freed was not allocated
python3(38437,0x107b185c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6irect_connection
    traces=traces), loop=self._loop)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 788, in _resolve_host
    host, port, family=self._family))
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/resolver.py", line 64, in resolve
    resp = await self._resolver.gethostbyname(host, family)
RuntimeError: Task <Task pending coro=<TCPConnector._resolve_host() running at /pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py:788> cb=[shield.<locals>._done_callback() at /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py:776]> got Future <Future pending> attached to a different loop
python3(38437,0x107b185c0) malloc: *** error for object 0x9000000000000000: pointer being freed was not allocated
python3(38437,0x107b185c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Do you have any idea?

[Saúl Ibarra Corretgé]

Thanks @asvetlov! I'll try to repro and let you know how it goes.

[Saúl Ibarra Corretgé]

At a first glance I don't see how it can fail, since we don't do any special allocations on gethostbyname. I'll dig deeper.

[Saúl Ibarra Corretgé]

FTR: the original issue (aio-libs/aiohttp#3573) was a misuse of aiohttp, but I'm still uncertain about the memory error.

[Saúl Ibarra Corretgé]

Fixed in fe57a61

After the change in pycares the ares channel will be automatically destroyed later.