How do you read metadata from a bxl?

[zjturner]

I tried several incantations of this, but every attempt I have to read the metadata attribute of a target results in getting back an object of type opaque_metadata, and it has no methods to obtain the actual contents.

It's ironic, because if I print a target object (e.g. ctx.output.print(target.attrs_eager())), it prints just fine as part of the aggregate struct. But if I try to reach in to pull out the metadata, the value is of the metadata attribute is not available.

Since the entire point of metadata is to examine it in a bxl, I feel like I'm missing something obvious.

I've tried going through a configured target as well as an unconfigured target, and neither one lets me get the metadata dict out.

Is it currently possible to do this?

Here's a sample bxl with output:

    targets = ctx.configured_targets(target_filter)
    for target in targets:
        metadata = target.attrs_eager().metadata
        ctx.output.print(type(metadata))
        ctx.output.print(metadata)

        ctx.output.print("metadata type = {}".format(metadata.type))
        ctx.output.print("metadata value = {}".format(metadata.value()))

    targets = ctx.unconfigured_targets(target_filter)
    for target in targets:
        metadata = target.attrs.metadata
        ctx.output.print(type(metadata))
        ctx.output.print(metadata)

        ctx.output.print("metadata type = {}".format(metadata.type))
        ctx.output.print("metadata value = {}".format(metadata.value()))

Build ID: 2443058b-0f29-4dea-8f16-ae7e3b750d2e
Jobs completed: 1. Time elapsed: 0.0s.
configured_attr
{}
metadata type = opaque_metadata
metadata value = OpaqueMetadata
coerced_attr
{}
metadata type = opaque_metadata
metadata value = OpaqueMetadata
BXL SUCCEEDED

[zjturner]

Oh man, it looks like this works 🤣

metadata = json.decode(str(target.attrs_eager().metadata))

[Wendy Yu]

Ah, this is a weird thing specific to metadata attr. So, the metadata attribute can be printed out since it has a display representation within Rust, but when it gets turned into an actual Starlark value, under the hood we will just allocate an OpaqueMetadata type which hides the information. We intentionally hide this since we don't want users using the metadata within normal rule analysis. cc @scottcao who was in the internal discussions for this decision - should this restriction also apply to BXL? Seeing that @zjturner was already able to work around it via json decode, I feel like we should just support inspecting it

[zjturner]

If you can’t read it from bxl, then where are you supposed to read it from? Regardless of whether the restriction is lifted, it feels like I'm missing the point, because I don’t know what else it’s for if not reading it from bxl

[Neil Mitchell]

It is useful for generic tagging and using in non-Bxl systems - e.g. a CI system we have built (and are releasing early in the new year). But agreed, that Bxl and outside-Buck2 are approximately the same, but we encourage Bxl where possible. Making the metadata available to Bxl as raw/real data seems the right thing to do. At the same time, having it available in rule implementations/analysis would be bad, since we really don't want rules using it.

[zjturner]

Perhaps targets need a metatdata attribute directly from bxl. But still, the json decode will work around it from within a rule anyway it seems

[Neil Mitchell]

The original plan was that metadata shouldn't be available within a rule. I think the JSON decode trick wasn't something people expected. Would be good to stop it being possible at all.

[Wendy Yu]

@ndmitchell I think this is a weird restriction - why can't we allow it in normal rule analysis? Does accessing it cause anything actually detrimental in rule analysis? I think we should just allow it in both normal rule analysis and BXL.

[zjturner]

Not privvy to any internal discussions, but just thinking out loud. Metadata is arbitrary, untyped, and only loosely structured (insofar as json itself is structured). But rules have to be generic, so there's an impedance mismatch between the arbitrary nature of what the person invoking the rule can specify, and the fixed nature of what the rule is expecting.

If a rule were going to use metadata, it would have to make some very strict assumptions about the structure of the metadata. And if you're going to do that anyway, why not expose these as arguments to the rule so that it can benefit from all of the other things of the rule api?