Akka.Persistence.Redis v1.4.16 - the "we removed Akka.Persistence.Query support" thread

[Aaron Stannard]

In the course of investigating #109, we came to some conclusions about the use of Akka.Persistence.Query in Akka.Persitsence.Redis:

1. It's not all that straightforward to implement in clustered Redis scenarios (including Twemproxy) - as individual entities need to be persisted into a single Redis node inside the cluster in order for routing to function correctly, and tags will need to be stored into a single node as well and there's a very good chance these two nodes will not be the same. Therefore, we can't guarantee that the IPersistentRepresentation and the Tagged referencing it can be written in the same atomic transaction. That's a bit of a problem.
2. It's non-performant - the Akka.Persistence.Redis tests that leverage Akka.Persistence.Query took half an hour to run on a single node. Once we removed Akka.Persistence.Query functionality from the RedisJournal and removed the tests that covered it, the entire single node test suite ran in under 60 seconds. You can see some benchmark values here: #114 (comment)
3. Akka.Persistence.Query isn't a great fit for Redis - if you want flexible read / write queries, use a RDBMS or MongoDb. If you want blazing fast reads / writes for rapid-fire event sourcing, use this plugin.

We're opening this issue so we can get data from users this change adversely affected - if you were previously using Akka.Persistence.Query with Akka.Persistence.Redis and need that functionality back, please let us know and tell us a bit about your use case here!

[Andy Furnival]

@Aaronontheweb I was trying to get your Cluster Sharding Viewer example working with Redis, as I wanted to understand why I'm getting occasional issues with an issue that was seen a long time, which causes the message "System.ArgumentException: Shard 98 is already allocated (Parameter 'e')" on Persistence Recovery.

I was hoping to interrogate Redis when the system is in a bad state, and try to deserialise the data and see what gets thrown up (and avoid all the boilerplate trying to go direct to redis to pull the raw data and try to deserialise it myself)

Given this notice above, am I right in thinking that any use of Persistent Query will simply not work for Redis? I'm not able to switch to ddata just yet, so its either Redis or I may need to look at an alternative plugin

[Aaron Stannard]

I was trying to get your Cluster Sharding Viewer example working with Redis

Which sample is this?

[Andy Furnival]

https://github.com/Aaronontheweb/Cluster.Sharding.Viewer this one.

[Aaron Stannard]

Ah I see - good news though, this query:

https://github.com/Aaronontheweb/Cluster.Sharding.Viewer/blob/d19012d23ac1664608e8fbe27fddbebf5e549d14/src/Cluster.Sharding.Viewer/Program.cs#L22-L23

Is supported by Akka.Persistence.Redis.Query since it doesn't depend on tags.

Akka.Persistence.Redis/src/Akka.Persistence.Redis/Query/RedisReadJournal.cs

Lines 103 to 116 in 4aa8cd9

 public Source<EventEnvelope, NotUsed> CurrentEventsByPersistenceId( 
     string persistenceId, 
     long fromSequenceNr = 0L,  
     long toSequenceNr = long.MaxValue) 
     => Source.ActorPublisher<EventEnvelope>( 
             EventsByPersistenceIdPublisher.Props( 
                 persistenceId,  
                 fromSequenceNr,  
                 toSequenceNr,  
                 null,  
                 _maxBufferSize,  
                 _writeJournalPluginId)) 
             .MapMaterializedValue(_ => NotUsed.Instance) 
             .Named("CurrentEventsByPersistenceId-" + persistenceId); 

So does that viewer not run correctly with Redis right now?

[Andy Furnival]

No, I can see query whilst monitoring redis which should return data (it does when called in redis-cli), but nothing ever returns in the console app. Might be serialisers. I’ll keep at it, now I know it should work