Onyx-Protocol / Onyx

Onyx

Home Page:https://Onyx.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

net/raft: formalize initial state

jbowens opened this issue · comments

There's an undocumented expectation in the net/raft package that the State passed in to Start is completely empty except that its nextNodeID counter must already be set to the second node's ID. (see database/sinkdb.newState) It's also important that all nodes in a cluster have the same initial state, otherwise their states could diverge despite applying the same raft log.

This is a pretty weird expectation and at the very least should be documented.

Also related, our current initial state accidentally sets the nextNodeID to 50 instead of 2 due to an encoding/decoding mismatch. If it's possible to fix this for new clusters without breaking old clusters, maybe we should.

Eventually, maybe we should refactor net/raft's interface to look something like:

// New constructs a new, inactive raft service.
New() (*Service, error)

// Init creates a new raft cluster and begins the raft algorithm.
(*Service).Init(initialState State) error

// Join joins the raft cluster addressed by the provided boot address
// and begins the raft algorithm.
(*Service).Join(initialState State, bootAddress string) error

// Recover restarts an existing raft node.
(*Service).Recover(existingState State) error

That might need to come after sinkdb's state implementation implements raft.Storage.