Policies are failing when live resources (already deployed) are non-compliant (aws_security_group_rule)

[Gabriel Soltz]

Description

Terraform Compliance is checking the live resources instead the changes to be deployed (the before instead of the after). If you have a plan pointing to a live resource that is non compliant, it's fails even after fixing the code that will fix that live resource. This only happens when resources were already deployed, so with new resources, this is not a problem.
I reproduced this bug (?) using the resources aws_security_group and aws_security_group_rule.

To Reproduce

1. Create a Security Group (aws_security_group) with a Security Group rule (aws_security_group_rule) and deploy it.

resource "aws_security_group" "sg_test" {
  name        = "test"
  description = "Allow ports"
  vpc_id      = ***
}

resource "aws_security_group_rule" "test_rule" {
  security_group_id = aws_security_group.sg_test.id
  from_port         = 8081
  protocol          = "tcp"
  to_port           = 8081
  type              = "ingress"
  cidr_blocks       = [0.0.0.0/0"]
}

2. Create a Terraform Compliance scenario to check the previous rule:

Feature: AwsEc2SecurityGroup feature

    Scenario: AwsEc2SecurityGroup 8081 must be restricted
        Given I have AWS Security Group defined
        When it has ingress
        Then it must have ingress
        Then it must not have tcp protocol and port 8081 for 0.0.0.0/0

3. Test the terraform compliance policy against the code from point 1, should fail:

terraform-compliance -p test-plan.tfplan -f .
terraform-compliance v1.3.34 initiated

. Converting terraform plan file.
🚩 Features	: xxx
🚩 Plan File	: xxx

🚩 Running tests. 🎉

Feature: AwsEc2SecurityGroup feature  # xxx

    Scenario: AwsEc2SecurityGroup high risk ports must be restricted
        Given I have AWS Security Group defined
        When it has ingress
        Then it must have ingress
		Failure: tcp/8081 port is defined within 0.0.0.0/0 network in aws_security_group.sg_test.
        Then it must not have tcp protocol and port 8081 for 0.0.0.0/0
          Failure:

1 features (0 passed, 1 failed)
1 scenarios (0 passed, 1 failed)
4 steps (3 passed, 1 failed)
Run 1669647146 finished within a moment

4. Now "fix" the terraform code to be compliant:

resource "aws_security_group_rule" "test_rule" {
  security_group_id = aws_security_group.sg_test.id
  from_port         = 8081
  protocol          = "tcp"
  to_port           = 8081
  type              = "ingress"
  cidr_blocks       = ["1.1.1.1/32"]
}

5. Generate a new plan with the fixed code, you should see the changes to be applied (from 0.0.0.0/0 to 1.1.1.1/32):

terraform plan -out=test-plan.tfplan
  # aws_security_group_rule.test_rule must be replaced
-/+ resource "aws_security_group_rule" "test_rule" {
      ~ cidr_blocks              = [ # forces replacement
          - "0.0.0.0/0",
          + "1.1.1.1/32",
        ]
      ~ id                       = "sgrule-1753760828" -> (known after apply)
      ~ security_group_rule_id   = "sgr-09d6c0860ec3e4af9" -> (known after apply)
      + source_security_group_id = (known after apply)
        # (6 unchanged attributes hidden)

6. Check with Terraform Compliance:

terraform-compliance -p test-plan.tfplan -f .
terraform-compliance v1.3.34 initiated

. Converting terraform plan file.
🚩 Features	: xxx
🚩 Plan File	: xxx

🚩 Running tests. 🎉

Feature: AwsEc2SecurityGroup feature  # xxx

    Scenario: AwsEc2SecurityGroup high risk ports must be restricted
        Given I have AWS Security Group defined
        When it has ingress
        Then it must have ingress
		Failure: tcp/8081 port is defined within 0.0.0.0/0 network in aws_security_group.sg_test.
        Then it must not have tcp protocol and port 8081 for 0.0.0.0/0
          Failure:

1 features (0 passed, 1 failed)
1 scenarios (0 passed, 1 failed)
4 steps (3 passed, 1 failed)
Run 1669647329 finished within a moment

It still failing, but the code is now correct. Terraform Compliance is failing against the "live" resource instead of the changes we want to apply.

7. The easiest way to confirm this, is going to AWS, remove the offending rule, and then generate the plan again and test it:

terraform-compliance -p test-plan.tfplan -f .
terraform-compliance v1.3.34 initiated

. Converting terraform plan file.
🚩 Features	: xxx
🚩 Plan File	: xxx

🚩 Running tests. 🎉

Feature: AwsEc2SecurityGroup feature  # xxx

    Scenario: AwsEc2SecurityGroup high risk ports must be restricted
        Given I have AWS Security Group defined
        When it has ingress
        Then it must have ingress
        Then it must not have tcp protocol and port 8081 for 0.0.0.0/0

1 features (1 passed)
1 scenarios (1 passed)
4 steps (4 passed)
Run 1669647468 finished within a moment

Feature File:

Feature: AwsEc2SecurityGroup feature

    Scenario: AwsEc2SecurityGroup 8081 must be restricted
        Given I have AWS Security Group defined
        When it has ingress
        Then it must have ingress
        Then it must not have tcp protocol and port 8081 for 0.0.0.0/0

Sample Terraform Code:
BEFORE:

resource "aws_security_group" "sg_test" {
  name        = "test"
  description = "Allow ports"
  vpc_id      = ***
}

resource "aws_security_group_rule" "test_rule" {
  security_group_id = aws_security_group.sg_test.id
  from_port         = 8081
  protocol          = "tcp"
  to_port           = 8081
  type              = "ingress"
  cidr_blocks       = ["0.0.0.0/0"]
}

AFTER:

resource "aws_security_group" "sg_test" {
  name        = "test"
  description = "Allow ports"
  vpc_id      = ***
}

resource "aws_security_group_rule" "test_rule" {
  security_group_id = aws_security_group.sg_test.id
  from_port         = 8081
  protocol          = "tcp"
  to_port           = 8081
  type              = "ingress"
  cidr_blocks       = ["1.1.1.1/32"]
}

Expected Behavior:
Terraform Compliance should check the changes, not the live resource, so after fixing the code, should be green.

Tested Versions:

• terraform-compliance version: v1.3.34
• terraform version: Terraform v1.0.2