The Amazon S3 Checklist is an exhaustive list of all elements you need to have / to test before using S3 in production.
All items in the Amazon S3 Checklist are required for the majority of the projects, but some elements can be omitted or are not essential. We choose to use 3 levels of flexibility:
- 🟢 means that the item is recommended but can be omitted in some particular situations.
- 🟡 means that the item is highly recommended and can eventually be omitted in some really particular cases.
- 🔴 means that the item can't be omitted for any reason.
Some resources possess an emoticon to help you understand which type of content / help you may find on the checklist:
- 📖 documentation or article
- 🔧 online tool / testing tool
- 📹 media or video content
-
🔴 Choose Bucket Policy vs IAM Policy vs ACL: Setting up the right access controls for your S3 buckets and objects.
-
🟡 Use Access Points: Simplifies managing data access at scale for applications using shared data sets on S3.
-
🟡 Create object lifecycle policies: Implement data lifecycle needs according to frequency, durability, and latency requirements. Object lifecycle policies will move data between the various storage classes, including Glacier and even the deletion of objects.
-
🟢 Have an object tagging policy: Object tagging is a powerful mechanism to attach metadata to objects managed in S3.
-
🔴 Have a backup plan: 11 9s of durability is not bulletproof. Consider cross-region replication or multi-cloud backups.
-
🟡 Enable object versioning: Object versioning, in conjunction with lifecycle management enhances application resilience.
-
🔴 Have a monitoring plan: What metrics are recorded? Who is notified? How often are metrics monitored?
-
🔴 Enable CloudTrail: CloudTrail logs cover general interactions with the S3 service.
-
🔴 Enable server access logging: Server access logging provides detailed records for the requests that are made to a bucket.
-
🔴 Enable block all public access: Blocks public access to S3 buckets and prevents S3 buckets from being made public unless this setting is turned off.
-
🔴 Enforce server-side or client-side encryption: Enabling SSE-S3, SSE-KMS or SSE-C to encrypt data at rest by AWS, or use envelope encryption (client-side) to encrypt data prior to it landing on S3.
-
🔴 Enforce encryption in-transit: Enforce the use of Secure Socket Layer/Transport Layer Security (SSL/TLS) for all S3 requests.
-
🟡 Enable MFA delete: Adds another layer of security requiring additional authentication.
-
🟡 Use VPC endpoints: Where traffic being routed over the Internet is undesirable, VPC endpoints should be used to access S3.
-
🟡 🆕 Use Amazon GuardDuty: Detect suspicious activities such as requests coming from an unusual geo-location, disabling of preventative controls such as S3 block public access, or API call patterns consistent with an attempt to discover misconfigured bucket permissions.
-
🟢 Use Glacier Vault Lock: Immutable policy for enforcing controls such as "write once read many" (WORM).
-
🟢 Use Amazon Macie: Macie automates the discovery of sensitive data, such as personally identifiable information (PII) and intellectual property, to provide you with a better understanding of the data that your organization stores in Amazon S3.
-
🟡 Have a partitioning strategy: Consider storing your data in a fit-for-purpose directory structure for better read and write performance.
-
🟢 Use S3 Select: Enables applications to retrieve only a subset of data from an object by using simple SQL expressions.
-
🟢 Use Glacier Select: Allows you to to perform filtering directly against a Glacier object using standard SQL statements.
-
🔴 Use Intelligent-Tiering: Intelligent-Tiering storage class is designed to optimize costs by automatically moving data to the most cost-effective access tier.
-
🟢 Retrieve S3 inventory: Outputs files that list your objects and their corresponding metadata on a daily or weekly basis. Can be useful to setup your own lifecycle management or for big data jobs that require S3 object metadata without having to call individual object APIs.
Open an issue or a pull request to suggest changes or additions.