Giters

cybozu-go / moco

MySQL operator on Kubernetes using GTID-based semi-synchronous replication.

Home Page:https://cybozu-go.github.io/moco/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The clustering process will be delayed by exceeding the rate limit for requests to the api-server.

kmdkuk opened this issue · comments

commented

What

If there are a large number of MySQLCluster resources, the clustering process will be delayed by exceeding the rate limit for requests to the api-server.
It has been observed that when the clustering process is delayed, operations such as switchover are also delayed.

The default rate limit (QPS) is 20.
https://github.com/kubernetes-sigs/controller-runtime/blob/a26de2d610c3cf4b2a02688534aaf5a65749c743/pkg/client/config/config.go#L84-L85

We have observed a delay of about 20 seconds when we created about 400 Clusters.
The high frequency of calls to MySQLCluster reconcile causes processing delays even when MySQLCluster resources are less than 400.

How

Add QPS to moco-controller's options.

Checklist

  • Finish implementation of the issue
  • Test all functions
  • Have enough logs to trace activities
  • Notify developers of necessary actions