notification wrapper submodule `Invalid for_each argument` error

[pyo_counting]

Description

Please provide a clear and concise description of the issue you are encountering, and a reproduction of your configuration (see the examples/* directory for references that you can copy+paste and tailor to match your configs if you are unable to copy your exact configuration). The reproduction MUST be executable by running terraform init && terraform apply without any further changes.

If your request is for a new feature, please use the Feature request template.

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: 3.3.0

• Terraform version: Terraform v1.3.8 on darwin_arm64

• Provider version(s): registry.terraform.io/hashicorp/aws v4.55.0

Reproduction Code [Required]

module "aws_sqs" {
  source  = "terraform-aws-modules/sqs/aws//wrappers"
  version = "4.1.1"

  defaults = {}

  items = {
    lambda_promtail = {
      crate                      = true
      name                       = "kps-psy-sqs"
      create_queue_policy        = false
      delay_seconds              = 0
      message_retention_seconds  = 345600 # 4d
      receive_wait_time_seconds  = 20
      visibility_timeout_seconds = 20
      sqs_managed_sse_enabled    = true
      # kms_data_key_reuse_period_seconds
      create_dlq                      = true
      dlq_name                        = "kps-psy-dlq"
      create_dlq_queue_policy         = false
      create_dlq_redrive_allow_policy = true
      dlq_delay_seconds               = 0
      dlq_message_retention_seconds   = 1209600 # 14d
      dlq_sqs_managed_sse_enabled     = true
      # dlq_kms_data_key_reuse_period_seconds
    }
  }
}

module "aws_s3_bucket" {
  source  = "terraform-aws-modules/s3-bucket/aws//wrappers"
  version = "3.3.0"

  defaults = {}

  items = {
    lambda_promtail = {
      create_bucket           = true
      bucket                  = "kps-psy-s3"
      force_destroy           = true
      attach_policy           = false
      attach_public_policy    = true
      block_public_acls       = true
      block_public_policy     = true
      ignore_public_acls      = true
      restrict_public_buckets = true
      #   policy                  = <<-EOF
      #   {
      #      "Version": "2012-10-17",
      #      "Statement": [
      #         {
      #            "Sid": "forLambdaFunction",
      #            "Effect": "Allow",
      #            "Action": "s3:GetObject",
      #            "Resource": "arn:aws:s3:::kps-psy-s3/*",
      #            "Principal": {
      #               "AWS" : role
      #            }
      #         }
      #      ]
      #   }
      #   EOF
    }
  }
}

module "aws_s3_bucket_notification" {
  source  = "terraform-aws-modules/s3-bucket/aws//wrappers/notification"
  version = "3.3.0"

  defaults = {}

  items = {
    test = {
      create            = true
      bucket            = module.aws_s3_bucket.wrapper["lambda_promtail"].s3_bucket_id
      create_sqs_policy = true
      sqs_notifications = {
        sqs1 = {
          queue_arn     = module.aws_sqs.wrapper["lambda_promtail"].queue_arn
          events        = ["s3:ObjectCreated:*"]
          filter_prefix = "prefix2/"
          filter_suffix = ".txt"
        }
      }
    }
  }
}

Steps to reproduce the behavior:

1. terraform init
2. terraform plan

Expected behavior

Actual behavior

module.aws_s3_bucket.module.wrapper["lambda_promtail"].data.aws_canonical_user_id.this: Reading...
module.aws_s3_bucket.module.wrapper["lambda_promtail"].data.aws_canonical_user_id.this: Read complete after 1s [id=f2a92297db65fb672f6df659d1907b1efbbdd2f92f5e89d7c2af6626bdc21166]
module.aws_s3_bucket_notification.module.wrapper["test"].data.aws_partition.this: Reading...
module.aws_s3_bucket_notification.module.wrapper["test"].data.aws_partition.this: Read complete after 0s [id=aws]
╷
│ Warning: Deprecated attribute
│ 
│   on .terraform/modules/aws_s3_bucket/main.tf line 27, in resource "aws_s3_bucket" "this":
│   27:       acceleration_status,
│ 
│ The attribute "acceleration_status" is deprecated. Refer to the provider documentation for details.
│ 
│ (and 2 more similar warnings elsewhere)
╵
╷
│ Error: Invalid for_each argument
│ 
│   on .terraform/modules/aws_s3_bucket_notification/modules/notification/main.tf line 76, in data "aws_arn" "queue":
│   76:   for_each = var.sqs_notifications
│     ├────────────────
│     │ var.sqs_notifications will be known only after apply
│ 
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the
│ instances of this resource.
│ 
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│ 
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.
╵
╷
│ Error: Invalid for_each argument
│ 
│   on .terraform/modules/aws_s3_bucket_notification/modules/notification/main.tf line 82, in data "aws_iam_policy_document" "sqs":
│   82:   for_each = { for k, v in var.sqs_notifications : k => v if var.create_sqs_policy }
│     ├────────────────
│     │ var.create_sqs_policy is true
│     │ var.sqs_notifications will be known only after apply
│ 
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the
│ instances of this resource.
│ 
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│ 
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.

Terminal Output Screenshot(s)

Additional context

[pyo_counting]

We conducted the test with a few simpler sample codes. As a result, we found that when using the wrapper module, values(unknown before terraform apply) cannot be used in the input variable.

In my case, input variable(items.test. sqs_notifications.sqs1. queue_arn) of module.aws_s3_bucket_notification cannot be unknwon unknown value before terraform apply(module.aws_sqs.wrapper["lambda_promtail"].queue_arn)

Please check my community post.

[pyo_counting]

Through the test, we found that it works well with the lookup function instead of the try function. But i am not sure that lookup function can replace all the intentions of using the try function.

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[Anton Babenko]

Please use the SQS module version 4.2.0 and use output queue_arn_static instead of queue_arn.