feat: robust DA service
apoorvsadana opened this issue · comments
Currently we've off chain workers in Madara that listen to new block changes and storage changes and accordingly submit DA jobs. We want to move this work away from the Madara sequencer itself and have a different service which polls on for new blocks and does the same thing. We want this service to be robust in the way that
- It can retry in case of submission failures
- Understand reorgs of the DA layer and resubmit blocks
- Batch DA submissions across multiple blocks
Maybe https://github.com/paritytech/orchestra would be useful. This is what we use in the Polkadot node to coordinate all off-chain work instead of off-chain workers.
Hey @rphmeier, thanks for the suggesting! Can you point me to the places where this is used if possible?
https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/node/overseer/src/lib.rs
Each of the subsystems here is in the adjacent node/core or node/network directories. I've found this architecture is reasonably good for separating concerns and coordinating many moving pieces around chain updates.
There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a 👍
Because this issue is marked as stale, it will be closed and locked in 7 days if no further activity occurs.
Thank you for your contributions!
da is not part of madara anymore since #1580