[Question] If parallel iterations also themselves use parallel iteration will the work stealing "just work"?
Lokathor opened this issue · comments
If I have some parallel iteration that I want to do (eg: process X-many files in parallel) and each of those parallel tasks also has parallel work (eg: within a file, process each item parsed in parallel), do all the tasks at the different "levels" of parallel work still participate in the same work stealing system? Say there's one file that has just one item and so that top level task completes very quickly, but the second file has 1000 items, will all of the second file's item work be distributed into the thread pool evenly and it just "works out"?
It works out. From the perspective of the rayon-core pool, a parallel iterator is just a bunch of nested joins, so then nesting iterators just look like more of that.
cool beans