layer processing order not correct
nab-gha opened this issue · comments
Seeing an issue where the 'apps' layer applies before the layers it depends on.
I delete HelmReleases and update AddonsLayers as follows:
sed -i s/0.1.13/0.1.14/g $REPOS_PATH/addons-config/testdata/addons/addons.yaml;kubectl delete helmreleases.helm.fluxcd.io -A --all;kubectl apply -f $REPOS_PATH/addons-config/testdata/addons/addons.yaml;watch kubectl get al
This causes all layers to process. Layer 'apps' that depends on 'base' and 'mgmt' is immediately processed and starts applying.
From debugging output it appear that the old 0.1.13 version of the custom resource is being processed and seeing the existing version 0.1.13 of 'base' and 'mgmt' layers are deployed so it proceeds to apply.
This reconcile is in response to the helm release being deleted.
2020-09-09T12:42:37.010+0100 DEBUG hr sync HR update, requeuing {"Name": "apps", "HR": "apps/microservice-1", "at": "controllers.indexHelmReleaseByOwner() - addons_controller.go(267)"}
....
2020-09-09T12:42:37.510+0100 DEBUG controllers.AddonsLayer apply required {"Name": "apps", "Spec": {"source":{"name":"addons-config","namespace":"gitops-system","path":"./testdata/addons/apps"},"prereqs":{"k8sVersion":"v1.16","dependsOn":["base@0.1.13","mgmt@0.1.13"]},"interval":"0s","version":"0.1.13"}, "Status": {"conditions":[{"type":"Deployed","status":"False","version":"0.1.08","lastTransitionTime":"2020-09-09T10:37:22Z","reason":"AddonsLayer is Deployed"},{"type":"Deployed","status":"False","version":"0.1.09","lastTransitionTime":"2020-09-09T10:44:05Z","reason":"AddonsLayer version 0.1.08 is Deployed"},{"type":"Deployed","status":"False","version":"0.1.09","lastTransitionTime":"2020-09-09T10:44:09Z","reason":"AddonsLayer version 0.1.09 is Deployed"},{"type":"Deployed","status":"False","version":"0.1.11","lastTransitionTime":"2020-09-09T10:54:53Z","reason":"AddonsLayer version 0.1.11 is Deployed"},{"type":"Applying","status":"False","version":"0.1.11","lastTransitionTime":"2020-09-09T11:01:36Z","reason":"AddonsLayer is being applied","message":"The applying status means the manager is either applying the yaml files or waiting for the HelmReleases to successfully deploy."},{"type":"Deployed","status":"False","version":"0.1.12","lastTransitionTime":"2020-09-09T11:01:48Z","reason":"AddonsLayer version 0.1.12 is Deployed"},{"type":"Applying","status":"False","version":"0.1.12","lastTransitionTime":"2020-09-09T11:17:31Z","reason":"AddonsLayer is being applied","message":"The applying status means the manager is either applying the yaml files or waiting for the HelmReleases to successfully deploy."},{"type":"Deployed","status":"False","version":"0.1.12","lastTransitionTime":"2020-09-09T11:17:45Z","reason":"AddonsLayer version 0.1.12 is Deployed"},{"type":"Applying","status":"False","version":"0.1.12","lastTransitionTime":"2020-09-09T11:19:11Z","reason":"AddonsLayer is being applied","message":"The applying status means the manager is either applying the yaml files or waiting for the HelmReleases to successfully deploy."},{"type":"Deployed","status":"True","version":"0.1.13","lastTransitionTime":"2020-09-09T11:19:19Z","reason":"AddonsLayer version 0.1.13 is Deployed"}],"state":"Deployed","version":"0.1.13"}, "at": "controllers.(*AddonsLayerReconciler).processApply() - addons_controller.go(110)"}
Need to do more debugging to understand how to address this scenario.
superseded by #64