Multiple projections per endpoint

[Charles Solar]

To reduce the need to replace the entire endpoint stream every upgrade - lets create projections for each namespace value in [Versioned] events and subscribe to each.

Then, a replay would only be needed for the specific projection if an event version is upgraded inside a projection or a new event added.

thoughts:

• should this only be for events? Or can we do commands too?
• what happens if a command is updated but only 1 handler is updated to use the new version?
• should handler projection names include a hash of handled event versions to prevent a user from updating the handler and not incrementing version?
• should we support event upgrading/downgrading into a specific handler?

[Charles Solar]

Thinking about this issue - while it was a good idea at the time there is a big issue.

Given an endpoint that has 10 different event namespaces and maintains read models in a single database. If we update one event and replay only that namespace - how would we know which objects in the database to rebuild. The endpoint would receive all the events over again pertaining to "Customers" but since its already processed those events it would be overwriting the customers it already created.

It may be a better feature to support updating projections when a new event is added or upgraded.

[Galen]

• At the moment the total"projection" is implied based on all the eventhandlers available in the endpoint. If there is a new event added then the whole endpoint is automatically replayed when the version number is bumped as the new event registration changes the subscription in ES.

What I would like is to have greater control on each projection within the endpoint. So group the event subscriptions by event handler class and register at event store multiple subscriptions I guess.

Then use subscription version number based on a decorator on each handler instead of the assembly version number.

The challenge is whether ES let's an endpoint essentially listen/own multiple projections explicitly I guess.

Galen

[Charles Solar]

@galenp breaking up the projection into many pieces is pretty easy to do - but while I was working on it I thought about what would happen if - say a new event was added and only the "Customer" projection was replayed.

Basically because the endpoint -> database is meant to be a 1-1 relationship, the now-updated endpoint would get all the "customer" events over again. Only they've already seen all those events and the database has all those models. There would be no way to "delete" all the customer read models for the rebuild ahead of time.

[Charles Solar]

I was thinking it would be better to just update the projection when starting - and leave it to the user to increment their endpoint's version if they want to replay everything

[Galen]

So the endpoint has 3 projections:

CustomerDetails
CustomerOrders
OrderDetails

You add a new event and it's used in 2 of 3 projections.

This triggers a rebuild in 2 of 3 projections.

It would be nice to have a hook here before the rebuild to delete the existing readmodel before the rebuild occurs.

Some/most readmodels shouldn't require a delete as a replay over the top should yield the same result as a replay on a blank db. The hook option would be nice tho and suit cases where the projection reads the existing data. Something I try to avoid doing.

I think you can have it so it replays only at the start of the endpoint. Or maybe have if so we can update the version by sending a system message and triggering a replay when that occurrs.

[Charles Solar]

After a chat - the way I was thinking about splitting was wrong.

We'd not be splitting on the event's Versioned namespace - we'd be splitting based on the Handler namespace.

And a rule would be to not change other handler's models (good practice in any case)

A model would be a self-contained unit inside a handler - and a projection would encompass all the events the 1 handler handles.

We can add a special Handler.Rebuilding hook method to allow the user to delete all the models they've built before starting the rebuild

[Charles Solar]

Upon startup the endpoint would map out which events are handled by which handlers and create a single projection which links events to one of multiple handlers like so

'CustomerCreated': function(event, state) {
   linkTo('CustomerDetailsHandler.0.1');
   linkTo('CustomerAnalyticsHandler.0.1');
}

Specific details on versioning the handler streams and the projection itself will need to be thought about more deeply