Maximum parallel replications for pull
Joseph94m opened this issue · comments
Is your feature request related to a problem? Please describe.
It's annoying and causes an avalanche effect when my pull based replications for the same Job start before the previous one is over.
It starts off by taking 2 minutes, then it slowly starts piling up until it reaches hours...
Describe the solution you'd like
The ability to specify, in the replication configuration:
- Maximum of replications waiting for an inProgress replication to finish
- Maximum replications that can be inProgress
Describe the main design/architecture of your solution
What do you guys think before we discuss implementation
The cron string is 0 */2 * * * *, it means that the replication is start every 2 minutes, you should always keep the schedule interval longer than a single job complete time.
Hello, that's the fallback approach indeed.
But you cannot always guarantee a replication's time due to occasional bulk insertions in the source registry and network variability.
In your case, most of the InProgress replication job is actually Pending, you could check the job in the job service dashboard, they should in the job queue. you could adjust the job service worker count to control the max Parallel job.