Query syntax and extractor for syntax tree elements

[David Tolnay]

For cargo expand I need a library that can index into a syn::File using a string-based query syntax. Roughly:

pub fn select(file: syn::File, query: String) -> Vec<syn::Item>;

Let's consider the following File as input:

mod a {
    mod b {
        trait C {
            fn d() {
                struct E;
            }
            fn f(self) {}
        }
    }
    fn b() {}
}

1. If I query for "a::b::C" then I should get back:

trait C {
    fn d() {
        struct E;
    }
    fn f(self) {}
}

2. If I query for "a::b::C::d::E" I should get:

struct E;

3. If I query for "a::b::C::f" I should get the trait C filtered down to only function f. The trait needs to be included because fn f(self) {} by itself is not a valid top-level Item.

trait C {
    fn f(self) {}
}

4. If I query for "a::b" I should get back both the mod b and the function b since these are both a::b. One lives in the type namespace and one lives in the value namespace. In general there can be any number of responses to a query.

Also I would like to track and propagate cfg attributes intelligently. For example in the following input:

#[cfg(feature = "g")]
mod imp {
    pub struct H(u8);
}
#[cfg(not(feature = "g"))]
mod imp {
    pub struct H(u16);
}

5. If I query for "imp::H" I should get:

#[cfg(feature = "g")]
pub struct H(u8);

#[cfg(not(feature = "g"))]
pub struct H(u16);

6. As a stretch goal, consider supporting wildcard path segments "a::b::_::D" that stand in for any name, and multi wildcards "..::D" where the .. stands in for any number of path segments. These would be similar to * and ** respectively in the wildcard rules of gitignore.

A basic implementation of this exists here in cargo-expand but only supports stepping into modules -- behaviors 1 and 4. It does not support behaviors 2, 3, 5, 6.

This function will be useful outside of cargo-expand so the implementation should go in its own crate.

[Ted Driggs]

I have something that passes tests 1 through 4 of the listed set. I have some questions that will impact the final behavior of the crate; even if I'm not the one to build it these will probably need answers.

1. In example 3, I'm treating types, methods, and constants all the same; if they're the immediate child of the trait then I include the parent trait when responding. Is this behavior acceptable?
2. In example 5, if H were to have its own cfg attribute, is it acceptable to stack those onto the returned element?
3. Can _ and .. wildcards be split into separate tests? The current implementation would allow for singles, but not for unknown depth, and I suspect I'm not the only one who'd run into a problem there.

[David Tolnay]

Nice! Thanks for looking at this.

• Yep, that sounds fine to me.

• I think so. You would end up with something like:

  #[cfg(feature = "g")]
  #[cfg(whatever_is_on_H)]
  pub struct H(u8);

• Neither _ or .. is important to implement. I would happily use a library even without those, or with only one of them.

[Ted Driggs]

In that case I'll have something ready in the next couple days.

[David Tolnay]

Sweet! If you want, also send a PR to cargo-expand to depend on your implementation.

[Ted Driggs]

Are methods declared in intrinsic impl blocks relevant?

[David Tolnay]

Such as in the following, queried for S::f?

struct S;

impl S {
    fn f() {}
}

I could go either way. Will leave that up to your discretion. I don't want to give the impression that we are doing anything too intelligent with this filtering, as it is all purely syntax manipulation and not actually resolving anything. If instead of struct S the module contained use other_crate::S, some may find it strange that the function is specified by path::to::local::S::f rather than other_crate::S::f. But at the same time I can imagine the behavior being useful.

[Ted Driggs]

@dtolnay that's a good point. I'll leave it out for now; we can always add it later.

[Ted Driggs]

https://github.com/TedDriggs/syn-select Wildcarding isn't implemented yet, but I'm planning to add single-term wildcards.

[David Tolnay]

Nicely done. I'll get this integrated into cargo-expand and follow up with issues in your issue tracker if needed. Thanks!

I marked the idea off the list with a link to your crate.