Parent future is too big in memory
timotree3 opened this issue · comments
Timo Carlin-Burns commented
If a parent future spawns a child future with TaskGroup::spawn
, the memory used by the child is included in the memory used by the parent.
Example
use std::mem::size_of_val;
use task_group::TaskGroup;
fn main() {
assert_eq!(size_of_val(&big_future()), 4098);
assert_eq!(size_of_val(&empty_future()), 1);
assert_eq!(
size_of_val(&spawns_big_future_using_tokio()),
size_of_val(&spawns_empty_future_using_tokio())
);
assert_eq!(
size_of_val(&spawns_big_future_using_task_group()),
size_of_val(&spawns_empty_future_using_task_group())
);
// ^ Fails
// thread 'main' panicked at 'assertion failed: `(left == right)`
// left: `4368`,
// right: `272`', src/main.rs:13:5
}
async fn spawns_big_future_using_task_group() {
let (task_group, task_manager) = TaskGroup::new();
task_group.spawn("big future", big_future()).await.unwrap();
drop(task_group);
task_manager.await.unwrap();
}
async fn spawns_empty_future_using_task_group() {
let (task_group, task_manager) = TaskGroup::new();
task_group
.spawn("empty future", empty_future())
.await
.unwrap();
drop(task_group);
task_manager.await.unwrap();
}
async fn spawns_big_future_using_tokio() {
tokio::spawn(big_future()).await.unwrap().unwrap();
}
async fn spawns_empty_future_using_tokio() {
tokio::spawn(empty_future()).await.unwrap().unwrap();
}
async fn big_future() -> Result<(), ()> {
let big_object = [0_u8; 4096];
// Hold _big_object across an await point
async { () }.await;
drop(big_object);
Ok(())
}
async fn empty_future() -> Result<(), ()> {
Ok(())
}
I believe this is caused by TaskGroup::spawn being an async function, so all the arguments are represented in the async function's initial state. This can be fixed by switching it to a normal function that returns impl Future
, and starting the async block after the task has been spawned.
PS: thanks for this awesome library!
Pat Hickey commented
Thanks for the detailed report!