Measurement and threading with core affinity

[Shuwen Sun]

Hi Jon,

Thank you for creating and maintaining the crate, it comes handy for probably quite a lot of people. However I do have a few specific questions that I can't find the answers from the doc.

Q1: Is it possible to measure the time it takes while executing each job? I know that register can return something other than () wrapped within a Result but I don't think that is what c.run returns. Is this kind of measurement possible at all? I am doing logging right now but ideally the results can be stored and pull together at some point.

c.register("foobar", |job| -> io::Result<()> {
    println!("{:?}", job);
    Ok(())
});

Q2: I read the doc and see that this crate comes with a workers option where we can specify more than one thread. Is it possible to pin those threads to different cores to actually take advantage of all the cores I have? If not, I guess we will need to use thread + core affinity with more than one consumers, right?

Best,

[Jon Gjengset]

Hey!

Q1: Hmm, I think that may be a new capability of Faktory that it didn't have when I first implemented it. May be totally possible to implement now, so I'd be happy to look at a PR!

Q2: I'm not sure why you'd specifically like to pin threads to cores? The operating system will already spread threads around to ensure that it keeps the cores busy as more load arrives.

[Shuwen Jethro Sun]

Thanks for your reply!

Yeah Q1 is probably some feature we can add. Maybe we can mark that as help wanted or something. Hopefully I can think about it once my current project is done one day.

Re Q2: I was using faktory as a job queue where I want to enable different number of cores (more cores) running in parallel as I increase the number of jobs I submitted. Maybe the current implementation of configuring workers already satisfy this? I know I can do that with core affinity and threading but I am not sure whether configuring number of workers here is equivalent

[Jon Gjengset]

You don't actually need to set affinity to have multiple threads use multiple cores. The OS takes care of that for you :) But yes, increasing the number of workers corresponds exactly to running multiple threads.

[Shuwen Jethro Sun]

Ah, then this is probably solved now. I did try it once and i think the multi-worker mode is working fine. We probably can close this :)