[RFE] allow OEMs to configure URL where to download oem-sysext updates
LittleFox94 opened this issue · comments
Current situation
oem-sysext image updates are downloaded from a URL provided by Nebraska or, as a fallback, from the release-server.
Impact
Neither of those download locations work for out-of-tree OEMs, for example for cloud providers just starting to develop an OEM sysext for flatcar (Hi).
Ideal future situation
Out-of-tree OEMs should be able to configure download locations for the oem-sysext image and to provide a set of public keys to verify those image updates.
OEMs should also be able to configure if the upstream sources should be tried first or if the OEM-specific location is to be preferred (or even the only one to try). Preferring the upstream download servers would allow a smooth migration from out-of-tree to in-tree, while requiring the oem-sysext images being loaded from the OEM-specific location could be an easy way to block flatcar updates when they are not (yet) compatible or untested with the OEM.
Implementation options
Both configuration flags (download location and download location preference) could be stored on the OEM partition in the oem-release file.
Public keys to verify the updated oem-sysext images could be provided in a GPG keyring file stored in the oem-sysext to verify signatures of those updated oem-sysext images. This would allow updating the keys over time (by providing new oem-sysext images). Having this as a keyring file instead of a just a key would allow having multiple keys, e.g. the "current" and "next" (with "next" being in cold-storage or something).
Additional information
We are currently working on building OEM support for Anexia, which is required to bring support cloud-init. I'm not yet happy enough with the way we support cloud-init here, so I hesitate to upstream our current state - but it is good enough to use for us right now, besides our Flatcar systems being unable to update due to this issue.
Several workarounds for this problem and alternative solutions to providing our cloud-init support were explored so far, but having the here-documented implementation option would just make the apparently nicest way (oem-sysext image) very easy for us and seems like it could be useful for other folks as well.
If this feature request and the implementation options given here are deemed reasonable, we are willing to contribute the actual implementation :)
Hello, thanks for this detailed issue and happy to see the interest of using Flatcar on Anexia.
Usually, the providers OEM are maintained in the scripts repository (see there: https://github.com/flatcar/scripts/tree/main/sdk_container/src/third_party/coreos-overlay/coreos-base the oem-* directories).
Some questions:
- What, roughly, would be the content of the OEM?
- Is
cloud-initmandatory for Anexia? Usually Flatcar is provisioned via Ignition (Ignition can fetch its configuration from a metadata service)
Hi @tormath1 :)
I found where the OEMs are stored in the Flatcar repository, but currently not yet too happy with what we have to upstream it - hence thinking about improving the situation for out-of-tree OEMs. I'm pretty sure there are other cases where an out-of-tree OEM might be a sensible option, not just for cloud providers working on their initial OEM support.
The contents of our OEM would be a program to adapt our cloud-config volume to cloud-init - what we have there is (currently, I hope this can be changed but would not be short-term) very Canonical cloud-init-brained with some extra twists. It should at some point also contain a guest agent and similar stuff.
Right now we can only do cloud-init since several layers in our VM provisioning workflow touch that. We also (sadly, again, maybe I can push for changes) don't have a metadata service but deliver the cloud-config by attaching a disk to the VM (NoCloud Datasource Canonicals cloud-init) - but that could be added to ignition when adding our OEM there (once the rest of our platform allows for ignition).
Hi @LittleFox94,
Something I have been thinking - since this OEM image would be out-of-tree, I think we could consider it as a regular sysext image and then leverage the systemd-sysupdate1 mechanism to deliver updates.
You could provide a systemd-sysupdate configuration file to target your download location + key to use to verify the download. This is an example of sysupdate configuration file to automatically pull new Kubernetes sysext image from Github releases2:
[Transfer]
Verify=false
[Source]
Type=url-file
Path=https://github.com/flatcar/sysext-bakery/releases/latest/download/
MatchPattern=kubernetes-@v-%a.raw
[Target]
InstancesMax=3
Type=regular-file
Path=/opt/extensions/kubernetes
CurrentSymlink=/etc/extensions/kubernetes.raw