Cores use for concurrent programs
j-m-nunes opened this issue · comments
When I run two or more copies of the same program, the subprocesses (say 4) for the different program copies always share the first 4 cores although there are other cores free. Is this the expected behaviour, that all programs request the first (4) cores?
Thanks in advance,
Very good remark! Indeed, the Parmap code uses the kernel-level interface
to lock each process to a different core, but does so in a very simple way:
if there are n cores and k processes, it locks the k processes to the first
k cores.
When you run two programs, all their processes will try to use the same
cores.
The simplest solution would be to have the client program accepting a
specification of a set of cores on the command line of the program,
something like
--coreset=0-3,5,8
and have parmap use only this set, with, of course, some sanity checks
(core 8 really exists, etc.), and defaulting to the set of all cores.
Should not be difficult to implement... if anybody wants to give a try, I
would add something like
(** {6 Setting and getting the default value for the core set } *)
val set_coreset : int list -> unit
val get_coreset : unit -> int list
and then change all the code in Parmap that does
(* spawn children *)
for i = 0 to ncores-1 do
...
done
to
List.iter (fun i ->
...) coreset
Roberto
2014-11-05 12:36 GMT+01:00 j-m-nunes notifications@github.com:
When I run two or more copies of the same program, the subprocesses (say
4) for the different program copies always share the first 4 cores although
there are other cores free. Is this the expected behaviour, that all
programs request the first (4) cores?
Thanks in advance,—
Reply to this email directly or view it on GitHub
#30.
Roberto Di Cosmo
Professeur En delegation a l'INRIA
PPS E-mail: roberto@dicosmo.org
Universite Paris Diderot WWW : http://www.dicosmo.org
Case 7014 Tel : ++33-(0)1-57 27 92 20
5, Rue Thomas Mann
F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo
FRANCE. Twitter: http://twitter.com/rdicosmo
Attachments:
MIME accepted, Word deprecated
http://www.gnu.org/philosophy/no-word-attachments.html
Office location:
Bureau 320 (3rd floor)
Batiment Sophie Germain
Avenue de France
Metro Bibliotheque Francois Mitterrand, ligne 14/RER C
GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3
Another solution might be to optionally disable process pinning to core.
It's funny, but I have never observed the behavior seen by this user.
@j-m-nunes are you on Linux?
@j-m-nunes just to check quickly if it solves your problem: remove calls to Setcore.setcore in parmap.ml
(there is only one call site).
then recompile and reinstall your modified version of parmap, then same for your test application.
Really recent linux kernels might shuffle processes less (natural CPU affinity), so disabling CPU affinity might not be that stupid for users who experience this problem (and who run a modern kernel).
Another possible simple fix would be to setcore the process to whatever core it is currently running (and hope that the operating system is not stupid in deciding where to start a new process).
The proper fix is not simple: it means parmap users need to be aware of where are current parmap processes running on their machine (even the ones launched by other users on the same computer...).
I can reproduce the user problem by running concurrently two copies of the same program.
I told each copy to use 8 cores, my machine has 16, but both htop and gkrellm show that only the first 8 cores are busy.
if you are interested, I can send other solutions to the problem: the default core pinning policy should be randomized and users should also be able to completely disable core pinning (in addition to users being able to be control freaks, that you already implemented). The current default policy is really bad, especially when you consider multi user systems (like servers).
related to #66
@coti Camille, if you know one way to do core pinning on a multi-user system by asking the OS (Linux at least and maybe OS X also), we might be interested.
From the command line or via some C code.
For example, is there an interface to do:
- request_pinnable_cores: the OS would tell us which cores we could pin to
- pin_cores: we try to reserve some cores for pinning
- unpin_cores: we stop using the cores we have reserved (should be called automatically
if the process exit).
So that several users and programs executing on the same system could do core-pinning without walking on each others' toes.
Currently, my solution is to disable core pinning, but the performance could be better.
Another way would be to have a system call to pin to core, which would fail if another process
is already pinned to that same core.
Thanks,
F.
maybe this is a missing interesting functionality in the kernel
Maybe we'll get some feedback in there:
https://unix.stackexchange.com/questions/425065/linux-how-to-know-which-processes-are-pinned-to-which-core
This was fixed, and this default behavior can be changed by the following call:
Parmap.disable_core_pinning ()