Reconfiguring exchanges with DynamicConfig
mfw78 opened this issue · comments
Is your feature request related to a problem? Please describe.
From my understanding, dynamic configuration in cryptostore conducts a hard stop / start when re-configuring exchanges. This seems likely to result in data loss for any clients that may be relying upon pass_through support, as well as leaving gaps in the data store.
Describe the solution you'd like
- Process to monitor
config.yaml - If exchange is already configured, supply delta of feeds to subscribe / unsubscribe.
- Already running process executes subscribe / unsubscribe as required.
Describe alternatives you've considered
To bypass this problem on K8S I would attempt to store the config.yaml in a ConfigMap, then redeploy the app with a minimum availability requirement, running two pods in the deployment so one was always present, providing data.
The problem here is that due to the config.yaml being updated, as the deployment rolls the update, the pod being preserved for the minimum availability requirement will detect the change to the config.yaml due to the ConfigMap update, so the "reserve" pod data will gap.
if only new subscriptions need to be added, that would be an easy thing to add (cryptofeed supports adding new feeds to a running feedhandler), but currently no exchange in cryptofeed supports unsubscribes, so while its do-able, its a lot of work, and I have other higher priority things i'm working on. I'll take a PR if its something you (or someone else) wants to implement