Example on using aiomcache for a cluster
duxing opened this issue · comments
Thanks for building this amazing lib!
I was wondering if you can share an example for using aiomcache
to connect to a memcached cluster with more than one host?
Seems like this is how connection are established and I don't see any other source code for connection management for multi-node cluster. Given how widely memcached clusters are used, I think an example would be really helpful
@duxing I was looking at the source code, as I'm still wondering if I should use aiomcache, emcache or pymemcache (probably will end up with emcache as pymemcache provides most features, but emcache is async, and aiomcache doesn't appear to have cluster support). I don't see any hashing code. I doubt this library provides clustering support at the time of writing this.
@takeda I did the same assessment and I prefer the features and performance of aiomcache
more than other options. I briefly went through the source code and didn't see the client-side of hashing code and I haven't continued to figure out the set up for a multi-node cluster. Not having that support would be a deal breaker for production adoption and I really wish it can be supported
But, if anyone wants to work on it, we can certainly accept new PRs.
The hash computation can be expensive, especially with large number of nodes, because (as I understand from looking at emcache code) the hashing is done for every node, and then node with highest score is picked up. That could be the reason why it is slower.
I do like aiomcache interface, get
, and multi_get
feels clearer than get
and get_multi
.
same here, looking for hashing - cluster - hashring implementation through aiomcache
import uhashring
from aiocache import caches, cached, AioCache
# Define your memcached server list
MEMCACHED_SERVERS = [
{"host": "server1", "port": 11211},
{"host": "server2", "port": 11211},
# Add more servers as needed
]
# Create a hashring instance
hash_ring = uhashring.HashRing(initial_servers=MEMCACHED_SERVERS)
# Define a custom cache backend class with hashring
class MemcachedHashRingBackend(AioCache):
async def get(self, key):
server = hash_ring.get_node(key) # Get server using hashring
async with caches[server["host"]].get(key) as value:
return value
async def set(self, key, value, ttl=None):
server = hash_ring.get_node(key)
await caches[server["host"]].set(key, value, ttl)
async def delete(self, key):
server = hash_ring.get_node(key)
await caches[server["host"]].delete(key)
# Configure aiocache with the custom backend
cache = caches.get("my_cache", backend=MemcachedHashRingBackend)
# Example usage with caching
@cached(ttl=60, cache=cache)
async def fetch_data(key):
# Simulate data fetching
data = f"Data for key {key}"
return data
async def main():
data = await fetch_data("my_key")
print(data)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
facebook/mcrouter#368 (comment) mcrouter has raft / consensus while replicating as seen from this reply