Giters

opensearch-project / data-prepper

Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.

Home Page:https://opensearch.org/docs/latest/clients/data-prepper/index/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support S3 bucket ownership validation on the S3 sink

dlvenable opened this issue · comments

commented

Is your feature request related to a problem? Please describe.

The s3 source supports S3's bucket ownership verification to protect against reading from buckets in unexpected accounts.

The s3 sink does not have this feature.

Describe the solution you'd like

Provide the same configurations for S3 bucket ownership as are provided in the s3 source. Use those to define the ExpectedBucketOwner parameter when writing to S3.

sink:
  - s3:
      default_bucket_owner: 000000000000
      bucket_owners:
        my-bucket-01: 123456789012
        my-bucket-02: 999999999999

Conceptual PutObjectRequest:

PutObjectRequest.builder().bucket(defaultBucket).key(objectKey).expectedBucketOwner(buckerOwner).build()

Additionally, this check should occur based on either of those fields being set. If they are not set, then there is no check. This is the current default and this will not break anything. As a result, I don't see any need to have a disable flag.

Additional context

Issue to add these configurations to the s3 source: #2012.

Original PR adding the check to the s3 source: #1526

commented

Hello @dlvenable,
Wanted to check, if someone is actively working on this, or not. If not, can I work on this ?