aio-libs / aiomcache

Minimal asyncio memcached client

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Client pool connections not re-used

fgasperino opened this issue · comments

I'm experiencing a resource leak when using this library. Here is an example of how to reproduce the condition.

import asyncio
import aiomcache

loop = asyncio.get_event_loop()
client = aiomcache.Client('127.0.0.1', 11211, pool_size=2)

@asyncio.coroutine
def get(c, k):
    yield from c.get(k)

for key in (b'non-existent-1', b'non-existent-2', b'non-existent-3', b'non-existent-4'):
    loop.run_until_complete(get(client,key))

Here is sample memcached output using the above test. You can see the client initiates a pool of 2 connections, and then proceeds to open a new connection for each get() call, instead of reusing the connections in the pool.

<17 server listening (auto-negotiate)
<18 server listening (auto-negotiate)
<19 send buffer was 9216, now 7456540
<20 send buffer was 9216, now 7456540
<19 server listening (udp)
<19 server listening (udp)
<19 server listening (udp)
<20 server listening (udp)
<19 server listening (udp)
<20 server listening (udp)
<20 server listening (udp)
<20 server listening (udp)
<21 new auto-negotiating client connection
<22 new auto-negotiating client connection
<23 new auto-negotiating client connection
23: Client using the ascii protocol
<23 get non-existent-1
>23 END
<24 new auto-negotiating client connection
24: Client using the ascii protocol
<24 get non-existent-2
>24 END
<25 new auto-negotiating client connection
25: Client using the ascii protocol
<25 get non-existent-3
>25 END
<26 new auto-negotiating client connection
26: Client using the ascii protocol
<26 get non-existent-4
>26 END

The following seems to clear the issue.

diff --git a/aiomcache/pool.py b/aiomcache/pool.py
index 98295a7..332bc7f 100644
--- a/aiomcache/pool.py
+++ b/aiomcache/pool.py
@@ -47,6 +47,8 @@ class MemcachePool:
                 if _conn.reader.at_eof() or _conn.reader.exception():
                     self._do_close(_conn)
                     conn = None
+                else:
+                    conn = _conn

             if conn is None:
                 conn = yield from self._create_new_conn()