agent: On delete of configmap or secret remove Cloudwatch resources

[Lili Cosic]

On delete we currently don't do anything. We need to somehow detect which resources we deployed and delete them one by one.

cc @dlespiau

Leftover from #210

[Lili Cosic]

Few observations and findings.

We can't just do kubectl delete -f cloudwatch.yml (even if we managed to request a fake manifest file), because it contains the weave namespace object and with that we would delete the rest of resources in that ns including the agent.

First thing is we always need to make sure that the other resource (configmap or secret) is gone as well and produce correct errors.

I see two ways, neither which I like that much:

1. Look for all resources in the weave namespace that are named cloudwatch-xxxxx for example and delete those resources.
2. Just hardcode the removal of resources, a configmap weave-cloudwatch-exporter-config and a deployment cloudwatch-exporter, as we always create those anyways in any case.

[Lili Cosic]

cc @marccarre @dlespiau any preferences or other thoughts/suggestions?

[Damien]

Would changing the generated manifests to not include the weave namespace and do kubectl delete -f be a better option?

[Lili Cosic]

I would then prefer option 3. Change generated manifest to not include the weave ns and delete each individual resource via the k8s api. That's the cleanest and proper way IMO.

[Lili Cosic]

As discussed earlier we decided that doing it with ownerReferences might be better choice.
Example:

  ownerReferences:
    - kind: ConfigMap
      apiVersion: v1
      name: cloudwatch
      uid: <random-uid-here>

But I think this might after all be easier to be done in the launcher, as we would need to pass at least the resource name and the resource uid. So in total we would need to pass onwards 4 more values. And besides that we would need to hardcode the apiVersion and kind, albeit those api versions have not changed in a while, I think. WDYT @marccarre @dlespiau

[Damien]

It's not shocking to me to have the full deletion logic in the launcher, that's generic-ish code that could be reused if we do something like this again, which is not unlikely!

[Lili Cosic]

Note: Only "problem" with creating an ownerReference is we need to add both a reference to the Secret and the ConfigMap but for that both need to be deleted for the resources that reference them to be removed by k8s. But I guess that is the correct way of doing it anyways IMO.