Build Hangs Recursive Struct

[Devin Jean]

I've been using this crate extensively for a no-std language vm. Part of my interface is as follows (minimized):

#[derive(Collect)]
#[collect(no_drop)]
pub enum Value<'gc> {
    List(Gc<'gc, Vec<Value<'gc>>>),
    Closure(Gc<'gc, Closure<'gc>>),
}

#[derive(Collect)]
#[collect(no_drop)]
pub struct Closure<'gc> {
    pub vars: Vec<Value<'gc>>,
}

This has worked perfectly fine for months. However, I recently made one little change to add a custom user type S, and suddenly cargo build and cargo check hang and eat up memory until it runs out and crashes. I've managed to track down the issue to this dependency and have a minimal example:

#[derive(Collect)]
#[collect(no_drop)]
pub enum Value<'gc, S> { // note the param S, which seems to cause the problem
    List(Gc<'gc, Vec<Value<'gc, S>>>),
    Closure(Gc<'gc, Closure<'gc, S>>),
}

#[derive(Collect)]
#[collect(no_drop)]
pub struct Closure<'gc, S> {
    pub vars: Vec<Value<'gc, S>>,
}

Worse, there is no error message whatsoever, so it was hard to track down. But I saw #4 and decided to try it. In my minimal example, this now gives a recursion depth error (better), but it doesn't solve my actual code base issues, which are more like the following again hanging example:

use gc_arena::{Collect, Gc};

#[derive(Collect)]
#[collect(no_drop)]
pub enum Value<'gc, S> {
    List(Gc<'gc, Vec<Self>>),
    Closure(Gc<'gc, Closure<'gc, S>>),
    Closure2(Gc<'gc, Closure2<'gc, S>>),
}

#[derive(Collect)]
#[collect(no_drop)]
pub struct Closure<'gc, S> {
    pub vars: Vec<Value<'gc, S>>,
}

#[derive(Collect)]
#[collect(no_drop)]
pub struct Closure2<'gc, S> {
    pub vars: Vec<Value<'gc, S>>,
}

Because adding a second non-self branch to the enum reintroduces the problem, I'm assuming this crate does some kind of breadth-first search for checking that a type's fields/variants satisfy Collect requirements (with a special case for Self), but which doesn't protect from cycles? Is there not a better way to check for cycles than a special case for (only) Self?

I need to be able to expose Closure instances to the library user, but I could possibly fix this trivial example by inlining the fields of Closure in Value::Closure, replacing Value<'gc, S> with Self, and reconstructing the actual Closure instance from the fields when requested by the user. However this would not solve my actual issues because the real (full) definition of Closure has several layers of indirect typing, all of which need to be exposed to users.

So my main questions are: why does the code without S not cause a problem, and is there some workaround to make this work?

[Catherine West]

This isn't a good solution, but just as a temporary measure you can always unsafely implement Collect manually.

I don't know what's causing this error, but IF you have time and want to look into it, the first question I would want answered is whether it's breaking during the proc macro or during compilation. I assume like #4 it's happening during compilation? If so, if you can output whatever the proc macro is generating that would be helpful.

This crate is using synstructure to do pretty much everything complicated in its proc macro Collect derive, and there's not really much going on outside of that. I suspect that the where bounds on fields (which we tell syntstructure to generate here) are very weird and its the compilation that is hanging? It's gross but would it work to just not have bounds on the field types and just let the compilation fail later? Does this derive work if you change AddBounds::Fields in that line to AddBounds::None?

[Devin Jean]

Right, it looks like it's breaking at compilation, cause cargo expand terminates even when cargo check doesn't. I think I've found a fix in the gc-arena crate: just changing the T: 'gc + Collect trait bound to T: 'gc in the definition of the Gc and GcCell structs fixes the problem, and I can make a PR for this. I think this would be safe to allow, because all instances of Gc and GcCell are created by functions that require the full T: 'gc + Collect bound, so removing the Collect requirement from the struct definitions doesn't (I don't think) introduce any unsoundness. Indeed, because the struct itself required that trait bound, we already know all references to it for any operation already require the full bounds, so it shouldn't change anything, but fixes the infinite recursion error during trait bound evaluation.

[Catherine West]

That's a neat way to solve that problem! I was just coming here actually to also suggest that we might need something like this to cover all cases: https://serde.rs/container-attrs.html#bound

I'm pretty sure your PR is correct independent of this, but I still think you can probably tie the compiler in knots somehow. I think a more complete solution to this problem might be to copy what serde does, so add bounds based on the generic type parameters, not the fields, and provide a #[collect(bound = "")] escape hatch... if I'm understanding all this correctly which I'm not super confident about still.

In any case, I think that PR makes everything easier to deal with and prevents unnecessary bounds so it's maybe a good idea anyway?