A few more examples

[Aaron Gershman]

This concourse resource looks pretty interesting, but I can't really tell how it works from the first look of it. At some point I'll have to experiment with setting up a simple demo on my own time, but still:

Could you post a few more examples / use-cases in the README when/if you get a chance?

Thanks for your time.

[Danny Berger]

Hi - thanks for your interest.

Regarding the what/why of the "metalink" thing, I wrote a bit about it here which may help. In general though, a generic way to document and reference blobs. I added on the "repository" notion of it to allow for querying a collection of metalink files from a remote location like a git repo.

Specifically for this resource, its use cases are around making it easier for communicating assets between pipelines/teams. Often pipelines will upload to S3 and someone else watches for a regex. I prefer the metalink approach because it allows for more security (independent checksums, signing), separation of where blobs may be mirrored, and separation of how identical blobs may be classified in multiple taxonomies (e.g. release stability).

For more tangible examples, I use this in openvpn-bosh-release for managing releases as they securely go through the pipeline. Alpha/dev builds are produced normally within the pipeline; I have dev environments which automatically consume beta releases, and then eventually it promotes through to stable where more critical environments can watch. Internally to those environments, they are also then able to pass around the metalink file of the versions/releases they're working on - both for provenance, and also if another environment needs to treat it as a resource.

I use this for producing regular binaries as well. One example being ssoca. Those binaries end up being added as GitHub release assets when published, but I still maintain metalink files/repositories for the originals since it's helpful to consume dev builds and final builds in a consistent way in some cases. GitHub also doesn't track checksum details. Pipeline examples for ssoca and openvpn are on their master branches if you want to take a look. It looks like most consumption (outside of the internal references in those pipelines) happens in private repos; there might be a couple things I could opensource around that, if you're interested.

I've also used metalinks and this resource to help privately mirror external assets for some infrastructure-critical components if there's a concern that they may disappear.

I'll add a bit more detail in the README at some point - that's a good suggestion, but maybe this is enough to get you started and know if it might be relevant to you. Happy to answer more questions in the meantime.

[Aaron Gershman]

This is great, especially the blog post 👍 You should definitely consider include this in the README for context, I'm sure it would be helpful for others too.

I appreciate you writing this up, thanks a ton. I'll dig into more hands-on examples later and see how it can solve some of the painpoints of our release engineering / concourse workflows

[Danny Berger]

For the curious, I've started writing some posts with more concrete examples of how's and why's of this approach:

• Background Motivation
• Mirroring Third-Party Dependencies
• Stability Channels