Consider using lifecycle for Policy Set Definition updates

[szczyrja]

Issue

Prerequisites

• I am running the latest version
• I checked the documentation and found no answer
• I checked to make sure that this issue has not already been filed

Context

When updating Policy Set Definitions I often have problems like:

│ Error: updating Policy Set Definition "corp_monitoring": policy.SetDefinitionsClient#CreateOrUpdateAtManagementGroup: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPolicySetParameterUpdate" Message="The existing policy has '68' parameter(s) which is greater than the count of parameter(s) '3' in the policy being added. Policy parameters cannot be removed during policy update."

│ Error: updating Policy Set Definition "corp_monitoring": policy.SetDefinitionsClient#CreateOrUpdateAtManagementGroup: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPolicySetParameterUpdate" Message="The existing policy set parameter(s) 'logsEnabled' type is being updated which is not allowed."

Error: updating Policy Set Definition "corp_monitoring_diagnostic_settings": policy.SetDefinitionsClient#CreateOrUpdateAtManagementGroup: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidPolicySetParameterUpdate" Message="The policy contains new parameter(s) 'rgName,storagePrefix' which are not present in the existing policy and have no default value. New parameters may be added to a policy only if they have a default value."

Expected Behavior

If update of the Policy Set Definition is not possible, it should be replaced.

Current Behavior

Errors mentioned above.

Possible Solution

Consider adding replace_triggered_by lifecycle meta-argument to module:
https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle?optInFrom=terraform-io#replace_triggered_by

The only hesitation I have is that maybe this should be solved on azurerm provider level.

[toby-p9]

Unfortunately, the lifecycle meta-argument is not supported for modules yet. There is a feature request open for it, but who knows when that will be implemented...

[Sadik Tekin]

Hey @szczyrja, thanks for raising this.

It interestingly points to multiple issues I've also tried tacking in the past, and agree the most efficient fix would be handled at provider level.

The lifecycle replacement would be perfect if only at module scope (mentioned by @toby-p9) as these changes would also require replacement of any set assignment(s) when triggered. In the meantime may I suggest using terraform replace or taint to overcome the shortfall until that functionality is added. see here

When adding new parameters to custom definitions within a set however its always best practice to include the defaultValue attribute as seen here: #36

[szczyrja]

@toby-p9 totally right, no lifecycle in modules

@gettek I also considered replace/taint, using it in pipeline is not very convenient though, so I choose the change then name of the initiative to recreate it as workaround if needed.
Good hint about defaultValue when I have a choice. Usually I use builtin policies and there it's not possible.

Thanks for the good work!

[Sadik Tekin]

@szczyrja

Ran some tests using the time_rotating resource and the following functionality will help when adding/removing parameters, however maybe you can tweak to trigger a replacement when updating parameter names or other attributes too...

modules/initiative/main.tf

resource azurerm_policy_set_definition set {
#...
  lifecycle {
    replace_triggered_by = [
      time_rotating.tst.rotation_rfc3339
    ]
  }
}

resource "time_rotating" "tst" {
  rotation_years = length(local.parameters)
}

[szczyrja]

@gettek

Interesting hack, I will try this, thanks.