Doesn't work with Redis Cluster
shaharmor opened this issue · comments
When trying to run against a Redis Cluster node:
Traceback (most recent call last):
File "/usr/local/bin/rma", line 9, in <module>
load_entry_point('rma==0.1.6', 'console_scripts', 'rma')()
File "/usr/local/lib/python3.4/dist-packages/rma/cli/rma_cli.py", line 95, in main
app.run()
File "/usr/local/lib/python3.4/dist-packages/rma/application.py", line 109, in run
for v in scanner.scan(limit=self.limit):
File "/usr/local/lib/python3.4/dist-packages/rma/scanner.py", line 61, in scan
for key_tuple in self.batch_scan():
File "/usr/local/lib/python3.4/dist-packages/rma/scanner.py", line 44, in batch_scan
yield from self.resolve_types(ret)
File "/usr/local/lib/python3.4/dist-packages/rma/scanner.py", line 50, in resolve_types
key_with_types = msgpack.unpackb(self.resolve_types_script(ret))
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 2694, in __call__
return client.evalsha(self.sha, len(keys), *args)
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 1944, in evalsha
return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.4/dist-packages/redis/connection.py", line 582, in read_response
raise response
redis.exceptions.ResponseError: CROSSSLOT Keys in request don't hash to the same slot
Maybe you should put a note about it in the README.. would have saved me some time 😄
We do not use redis cluster right now, so i do not know this not working with it. From stacktrace i see this can be fast fixed with some perfomance breakdown. So i tnink i fix it during this week.
Ok thnx, let me know when you have and i'll give it a shot
@shaharmor how do you see cluster report? It is per node report or full cluster? Now i fix original issue, but as i see redis-py can`t work with MOVE responses. So i want to understang how to it in right way. If you need node stat i can just skip keys from node which actually not on this node.
@misterion - I tried running it right now with latest version.
Its running and looks like its going over all keys (In the specified node i connected it to), but the report doesn't show anything other than the global stats. (Just empty tables).
I don't expect the report to be of the full cluster (Although that would be awesome), but just for the node i connected to.
When you use SCAN on the node itself, its going only over the keys in that node (Not in the entire cluster), so it shouldn't be a change from what you do today.
But for some reason there is no output for the keys it scanned.
I still working under it. From time to time node ask to MOVE to other node to execute command for key returned by SCAN. Now my devops moves one of out redis test server to cluster to test all of this in live cluster server. Playing now with redis-py-cluster library - looks like this should work fine as in cluster as in single mode.
Now waiting resposne with Grokzen/redis-py-cluster#123. Looks like this library has a bug with MOVED response :(
The issue from redis-py-cluster was fixed in unstable branch according to the contributors comments. I check it with my dev cluster and still have MOVE fails with it. So want to check why this so and help to improve redis-py-cluster to fix this. I have working workaround to hack fix this but this patch could be used in production becouse based on reACK on each MOVE error. In this case scanning takes absolutely unreal time.