net/raft: formalize initial state
jbowens opened this issue · comments
There's an undocumented expectation in the net/raft package that the State passed in to Start is completely empty except that its nextNodeID counter must already be set to the second node's ID. (see database/sinkdb.newState) It's also important that all nodes in a cluster have the same initial state, otherwise their states could diverge despite applying the same raft log.
This is a pretty weird expectation and at the very least should be documented.
Also related, our current initial state accidentally sets the nextNodeID to 50 instead of 2 due to an encoding/decoding mismatch. If it's possible to fix this for new clusters without breaking old clusters, maybe we should.
Eventually, maybe we should refactor net/raft's interface to look something like:
// New constructs a new, inactive raft service.
New() (*Service, error)
// Init creates a new raft cluster and begins the raft algorithm.
(*Service).Init(initialState State) error
// Join joins the raft cluster addressed by the provided boot address
// and begins the raft algorithm.
(*Service).Join(initialState State, bootAddress string) error
// Recover restarts an existing raft node.
(*Service).Recover(existingState State) error
That might need to come after sinkdb's state implementation implements raft.Storage.