Giters

facebookincubator / buck2-change-detector

Given a Buck2 built project and a set of changes (e.g. from source control) compute the targets that may have changed. Sometimes known as a target determinator, useful for optimizing a CI system.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Filtering by configuration

cbarrete opened this issue · comments

commented

Does this project somehow allow filtering by platform/applying different configurations to the targets, or is this something that should be done separately?

Concretely, using targets/btd can return targets that are not supported by the default platform, which causes our buck2 build/test invocations to fail with a check the target's compatibility attributes type error. The other side of that coin is that I do not know how to specifically get the targets for an alternative configuration.

Using -- --target-platforms //my/execution:platform in the targets call does not seem to filter out the targets, and I couldn't see anything in btd that would help either.

Any pointers would be appreciated!

commented

Great questions! Internally btd is step 1 in a pipeline, where the next step takes configuration information (from the PACKAGE files and the labels attribute) and comes up with a plan of where to test what things. As an approximate example, if you write set_package_value("ci.spec", ["linux", "mac"]) then we'll get your code on Mac and Linux (with various layers of macros to make that more pleasant). We also run all jobs with --build-report and --skip-incompatible-targets. We do want to open source the second step of the pipeline, eventually, but at the moment it is full of Meta-specific hacks as it has a long history of being multiple systems we are only now consolidating around Buck2.

If your CI is small enough, it may be that --skip-incompatible-targets and running all the targets on Windows/Mac/Linux is sufficient?

commented

Thanks for the quick reply, this all makes a ton of sense!
--skip-incompatible-targets may very well be enough for us for now, and --build-report could come in handy to get better build results.

While I'm here, more of a buck2 question than a change detector one, but can you confirm that there is no way to get multiple configurations in a single dag/instantiation? Or even multiple commands such as build/test?
Basically, I'm trying to figure out how to optimally run the matrix of build/test * number of configurations (especially when some configurations share most targets with no differences) * potential other axes in the future (?). A single buck2 invocation with a single DAG would be ideal of course, but as I understand, this is not supported.

commented

There is no easy way to do any of that today, as far as I know.

  1. Doing a build and a test simultaneously isn't hard in theory. I imagine we need a --build flag to test, and then it would do both. Not much work.
  2. Doing multiple execution platforms in one go is meant to be enabled by modifiers - https://buck2.build/docs/rfcs/cfg-modifiers/api/. I'm not sure how far along that work is though - CC @scottcao

But what you describe is definitely reasonable.

commented
  1. Is good to know, I might add that myself at some point then. It's more a theoretical question for us for now, but it would be a nice addition eventually, as I expect that "run my tests but also build the rest, which tests might not depend on" is a very common use case.
  2. That would be great, I will take some time to read that RFC, thanks for linking it