SDK to handle server failure when selecting a replica to connect to
tock-ibm opened this issue · comments
The commonTxContext selects a replica when it does the GET and POST inside every transaction context.
func (t *commonTxContext) selectReplica() *url.URL
Currently this always selects the first replica in the set of replicas.
- We have to make the selection process adaptive to server availability.
- For example, one server out of 3 may be down; such that a request to it fails. the TxContext must try the rest of them until it finds the available servers.
- It should order the servers or label them according to availability such that in the next round it will hit an available server, in case the status of the cluster stays the same.
- If none of the servers are available, then after a few tries (define a policy on how much time to spend on retries and what is the retry interval), return an error to the client.
Do a simple implementation first; improve when the server exposes an API that describes the cluster status.
The session, when it creates a tx context, could give it a replicaSet with availability info already baked in, which is a result of all previous TXs. This way a new TX will not have to rediscover the cluster availability which was already known to previous TXs. This could be achieved by the session polling the cluster status of the server.