rp cpu issue
anDoer opened this issue · comments
rp does not allocate the requested number of cpus properly. The runtime option
docker run --cpus= only specifies, how many CPU resources can be used, but it does not prevent different jobs from utilizing the same cpu. The docker documentation states the following:
| --cpus= | Specify how much of the available CPU resources a container can use. For instance, if the host machine has two CPUs and you set --cpus="1.5", the container is guaranteed at most one and a half of the CPUs. This is the equivalent of setting --cpu-period="100000" and --cpu-quota="150000". |
|---|
With increasing number of submitted jobs, this drastically impacts the overall runtime of every job.
A better option is the following docker runtime option
| --cpuset-cpus | Limit the specific CPUs or cores a container can use. A comma-separated list or hyphen-separated range of CPUs a container can use, if you have more than one CPU. The first CPU is numbered 0. A valid value might be 0-3 (to use the first, second, third, and fourth CPU) or 1,3 (to use the second and fourth CPU). |
|---|
We performed some tests and latter options seems to result in a constant runtime for all running jobs.
docker inspect seems to return all allocated cpus for a running container, so it might be a better option to keep track of allocated cpus?
Line 330 in 96d44db