Could bxl gain the ability to create targets on the fly?

[zjturner]

I was looking for the ability to create a target and then build it.

The idea came from the desire to write an analysis pass that iterated every target, and remove deps 1 by 1, building the target at each step along the way. If the build passes, you have an extraneous dep that you can indicate to the user so they can remove it.

I thought of lots of other use cases for this as well. For example, running clang-tidy over a build by modifying compiler flags and rebuilding.

has the ability to create targets from bxl been discussed? Or is it already there and I didn’t find it?

[Austin Seipp]

What kind of user-interface do you even imagine this having? I've had some similar thoughts, but I think it's pretty use case and policy specific. Trimming deps fields for example can be done a lot of ways, I suspect, and it's not clear what the workflow should really be (you're basically doing fuzzing but on the build graph, which strongly suggests to me it should be something very automated, and then has different constraints.)

For the clang-tidy case, you can just run actions in bxl so assuming you can analyze the targets and get their command lines, I think that one should be pretty easy.

[zjturner]

What kind of user-interface do you even imagine this having? I've had some similar thoughts, but I think it's pretty use case and policy specific. Trimming deps fields for example can be done a lot of ways, I suspect, and it's not clear what the workflow should really be (you're basically doing fuzzing but on the build graph, which strongly suggests to me it should be something very automated, and then has different constraints.)

Given the following graph for illustration purposes (and assuming these are actual cxx_library targets with real source files, include paths, etc, not just empty targets as below):

cxx_library(name = "a")
cxx_library(name = "b", deps=[":a"])
cxx_library(name = "c", deps=[":a"])
cxx_library(name = "d", deps=[":b", ":c"])
cxx_library(name = "e", deps=[":a", ":c"])
cxx_library(name = "f", deps=[":e", ":c"])

I would expect it to work like this:

$ buck2 bxl bxl/analyze-deps.bxl:extraneous //:f
Scanning deps recursively for target //:f    Target queue: [":a"]
Scanning target ":a": No extraneous deps found!   Target queue: [":b", ":c"]
Scanning target ":b": No extraneous deps found!   Target queue: [":c"]
Scanning target ":c": Found extraneous dep [":a"]!   Target queue: [":d", ":e"]
Scanning target ":d": Found extraneous dep [":c"]!   Target queue: [":e"]
Scanning target ":e": No extraneous deps found!   Target queue: [":f"]
Scanning target ":f": Found extraneous dep [":e"]!  Target queue []

Based on this output, the user knows they can now go back and change their buck file to:

cxx_library(name = "a")
cxx_library(name = "b", deps=[":a"])
cxx_library(name = "c")
cxx_library(name = "d", deps=[":b"])
cxx_library(name = "e", deps=[":a", ":c"])
cxx_library(name = "f", deps=[":c"])

For the clang-tidy case, you can just run actions in bxl so assuming you can analyze the targets and get their command lines, I think that one should be pretty easy.

It's not quite that simple, because clang-tidy does not produce an output. If you have a genrule() or similar that generates code, that needs to run before clang-tidy can succeed. You can of course run your entire build first via buck2 build //<target-path>, but that is roughly twice as slow because you're actually running the compiler twice now for every file. If you're clever you can "just" build all of the generated code, but even generating code is just one possible reason that manipulating command lines and running them might not work.

The build graph can be arbitrary. Maybe you use http_archive() for example to put some headers down on disk that the rest of the code won't compile without. If you can just manipulate the graph directly and then tell buck2 to build the graph, it's a lot more robust and it'll always be correct and maximally efficient.

[Neil Mitchell]

You can use anon targets from Bxl, which I think is the way to create targets on the fly from Bxl. CC @wendy728 who probably can point at examples.

[Wendy Yu]

Yeah I think anon targets could work here. Here is the documentation for it: https://buck2.build/docs/developers/anon_targets/

[zjturner]

Thanks, I’ll close this unless I find out it doesn’t work