The replica from old revision terminated before the new revision is fully ready
yuzisun opened this issue · comments
Both old and new revision has 4 replicas, during rollout we noticed that the replica from old revision terminated before the new revision is fully ready. In this case we end up serving the requests with reduced capacity and less expected number of replicas.
tests-predictor-00003-deployment-7644f8cbb5-cdgt4 1/2 Terminating 0 54m
tests-predictor-00003-deployment-7644f8cbb5-jwfgh 1/2 Terminating 0 55m
tests-predictor-00003-deployment-7644f8cbb5-mxmms 1/2 Terminating 0 54m
tests-predictor-00003-deployment-7644f8cbb5-wlwt2 2/2 Terminating 0 56m
tests-predictor-00005-deployment-65fcd445d-9z2kk 2/2 Running 0 3m27s
tests-predictor-00005-deployment-65fcd445d-ddcdr 1/2 Running 0 46s
tests-predictor-00005-deployment-65fcd445d-q9s4m 2/2 Running 0 2m6s
tests-predictor-00005-deployment-65fcd445d-qhnrj 2/2 Running 0 4m49s