Filtering for incoming webhooks

[Ruben Laban]

A more generic approach similar to the support for filtering on Events for GitHub, etc.

For example GCR/GAR sends a pretty simple, but also useful payload:

{
  "action":"INSERT",
  "digest":"us-east1-docker.pkg.dev/my-project/my-repo/hello-world@sha256:6ec128e26cd5...",
  "tag":"us-east1-docker.pkg.dev/my-project/my-repo/hello-world:1.1"
}

Ref: https://cloud.google.com/artifact-registry/docs/configure-notifications#examples

It would be nice to be able to map the digest to a ImageRepository to notify (either automatically or explicitly configured). This could potentially reduce the load within and caused by Flux drastically.

The current alternative (to reduce the load) is to configure separate incoming webhooks on both GCR/GAR's and Flux' side. Which is rather cumbersome.

[Max Jonas Werner]

I'm not sure I'm following what you're requesting. Would you mind describing the envisioned API changes and/or workflow you're aiming at, please?

[Ruben Laban]

Currently we have a single receiver to consume notifications from our GCR/GAR. In this receiver spec.resources lists 34 ImageRepository references. Whenever a notification arrives from GCR or GAR, all 34 image repositories get annotated and scanned for new images. What I'm proposing is instead of triggering a scan for all 34, only trigger those image repositories that match against the digestpart of the notification payload (without the @... part).

[Lucio Veloso]

I will go even deep. The primary concern isn't just that all images are annotated for reconciliation. The core issue is the reconciliation process itself. I have thousands of images, and with each new webhook received, there's a scan of all image tags in Harbor. This results in significant resource consumption in Harbor instance.

[Stefan Prodan]

Each provider has a unique payload schema, we'll need to get all those schemas as dependencies (does al of them have Go SDKs?). After that we'll need to write a custom parser for each one, to be able to extract some string that may or may not match the Flux resource name listed under .spec.resources. The Receiver API is common to all current and future Flux resource types which makes this issue about container images so much larger is scope.