EKS AccessControlListNotSupported: The bucket does not allow ACLs

[Ted Gelpi]

When creating a new EKS environment using OOTB config you encounter errors regarding long term storage creation

│ Error: error creating S3 bucket ACL for logs-jx3fri-20230421172407921400000009: AccessControlListNotSupported: The bucket does not allow ACLs
│ 	status code: 400, request id: FBT5A0MKD0FFVN7K, host id: 7FR0M1P79alIyR0mT5YxbKuaxtuLdVsoLkhwa5Avxpkmz/ZenqArzwx4lxZuKDTgtOrdz4fOXRk=
│ 
│   with module.eks-jx.module.cluster.aws_s3_bucket_acl.logs_jenkins_x[0],
│   on .terraform/modules/eks-jx/modules/cluster/storage.tf line 22, in resource "aws_s3_bucket_acl" "logs_jenkins_x":
│   22: resource "aws_s3_bucket_acl" "logs_jenkins_x" {
│ 

A notice from Amazon indicates that Amazon S3 will automatically enable S3 Block Public Access and disable access control lists for all new buckets starting in April 2023

Currently storage allocation defaults to ACL

resource "aws_s3_bucket_acl" "logs_jenkins_x" {
  count  = var.enable_logs_storage ? 1 : 0
  bucket = aws_s3_bucket.logs_jenkins_x[0].bucket
  acl    = "private"
}

Tried correcting this issue by updating “aws_s3_bucket_acl” resource with “aws_s3_bucket_ownership_controls” resource setting the object_ownership rule to “BucketOwnerEnforced”

resource "aws_s3_bucket_ownership_controls" "logs_jenkins_x" {
  count  = var.enable_logs_storage ? 1 : 0
  bucket = aws_s3_bucket.logs_jenkins_x[0].bucket

  rule {
      object_ownership = "BucketOwnerEnforced"
    }
}

This type of storage update results in a clean build of the S3 buckets but it doesn't appear to be any content written to those buckets. Currently not clear on how to test and validate this change.

Using the latest version of cli, plugins, etc.

[Ted Gelpi]

The issue potentially impacts the creation of 5 buckets

vault-unseal-bucket
logs_jenkins_x
reports_jenkins_x
repository_jenkins_x
backup_bucket

The files that need updating are:

modules/vault/main.tf
modules/cluster/storage.tf
modules/backup/main.tf

A new variable "enable_acl" is being considered for backward compatibility

[Ted Gelpi]

I have a configuration that resolves this issue of ACL not supported by setting object ownership controls instead of ACL for the five S3 buckets.

EKS Resources

A new variable "enable_acl" is created and defaults to false. 

Five S3 buckets are adjusted
  backup_bucket
  logs_jenkins_x
  report_jenkins_x
  repositor_jenkins_x
  vault-unseal-bucket

Terraform Bucket Resources

## Original setting using ACL  variable enable_acl=true
resource "aws_s3_bucket_acl" "<jx3-bucket>" {
bucket = aws_s3_bucket.<jx3-bucket>[0].bucket
  acl    = "private"
}

## Setting with ACL Disabled.  variable enable_acl=false  (default)
resource "aws_s3_bucket_ownership_controls" "<jx3-bucket>" {
  bucket = aws_s3_bucket.<jx3-bucket>[0].bucket
  rule {
      object_ownership = "BucketOwnerEnforced"
    }
}

## All buckets continue to be encrypted
resource "aws_s3_bucket_server_side_encryption_configuration" "<jx3-bucket>" {
  bucket = aws_s3_bucket.<jx3-bucket>[0].bucket
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = local.encryption_algo
      kms_master_key_id = var.s3_kms_arn
    }
  }
}

Changed Files

 README.md
 main.tf
 variables.tf
 modules/backup/main.tf
 modules/backup/variables.tf
 modules/cluster/storage.tf
 modules/cluster/variables.tf
 modules/vault/main.tf

Git Diff

The new Infrastructure was tested agains both the Vault and AWS Secret Manager cluster configs. The buckets get created and is viewable from the portal or s3cli. It appears that the vault cluster configuration was creating folders under the log bucket whereas the ASM version was not. I believe this limitation on the ASM version is currently an issue and is not caused by this recent update.

The code can be tested using the following branch:
source = "github.com/jx3rocks/terraform-aws-eks-jx?ref=enable_acl"

[Ted Gelpi]

This issue has been fixed with PR #362

[Ted Gelpi]

This issue has been fixed with PR jenkins-x/terraform-aws-eks-jx#362