Scoped spawn API to spawn non-static futures

[Michael Hewson]

crossbeam, rayon, and scoped_threadpool all have an API that allows you to safely spawn threads with non-'static closures. It's kind of complicated and requires unsafe code to implement. I was wondering if it would be possible to do the same thing with runtime's spawn method, and how much interest there is in spawning non-'static futures. It could be implemented in a separate crate at first and only merged in at some point in the future if it makes sense to do so.

[Yosh]

Hi! Thanks for opening this issue. This seems quite similar to withoutboats/juliex#19. That issue was filed first, so I'll go ahead and continue the conversation on there (:

[rmanoka]

Hi! I did a little bit of prototyping along the lines mentioned by @yoshuawuyts in the other thread (where we don't directly spawn crossbeam threads and block). Would be really helpful to hear some expert opinions on this (esp. the unsafe-ties). Below, we summarize what we are looking to address, and a possible implementation

Motivation

Present executors (tokio, juliex, futures-rs) all support spawning 'static futures onto a thread-pool. owever, it is often useful to parallelly spawn a stream of futures. While the future combinators such as for_each_concurrent offer concurrency, they are bundled as a single Task structure by the executor, and hence are not driven parallelly. This can be seen when benchmarking a reasonable
number (> ~1K) of I/O futures, or a few CPU heavy futures.

Scope API

We propose an API similar to crossbeam::scope to allow controlled spawning of futures that are not 'static. The key function is:

pub fn scope<'a, T: Send + 'static,
             F: FnOnce(&mut Scope<'a, T>)>(f: F)
             -> impl Stream {
    // ...
}

This function is used as follows:

#[tokio::test]
async fn scoped_futures() {
    let not_copy = String::from("hello world!");
    let not_copy_ref = &not_copy;

    let mut stream = crate::scope(|s| {
        for _ in 0..10 {
            let proc = async || {
                assert_eq!(not_copy_ref, "hello world!");
            };
            s.spawn(proc());
        }
    });

    // Uncomment this for compile error
    // std::mem::drop(not_copy);

    use futures::StreamExt;
    let mut count = 0;
    while let Some(_) = stream.next().await {
        count += 1;
    }
    assert_eq!(count, 10);
}

Implementation

Our current implementation simply uses unsafe glue to actually spawn the futures. Then, it records the lifetime of the futures in the returned Stream object.

Currently, for soundness, we simply panic! if the stream is dropped before it is fully driven. Another option (not implemented here), may be to drive the stream using a current-thread executor inside the Drop impl. That might be right analogue of crossbeam waiting on the join of spawned threads. Would be great to hear any thoughts on the problem, and the safety of this implementation.

Please find the implementation here: https://github.com/rmanoka/tokio-scoped

[Yosh]

@rmanoka thanks for sharing; that looks very cool!