traveloka / terraform-aws-waf-webacl-supporting-resources

A module to create several resources needed by AWS WAF WebACL.

terraform-aws-waf-webacl-supporting-resources

Deprecation Notice

Hi everyone, this module is now deprecated and will no longer be supported or updated.

For Travelokans, please contact the Cloud Infra Team on slack to discuss WAFv2 implementation

Description

Terraform module to create resources needed by AWS WAF WebACL to:

• Enable logging of traffic information[1].
• Store logs in Parquet format[2] for more optimized query using Athena[3].
• Provision query-ready Athena Database and Table which based on AWS Glue Data Catalog [4].

Based on the diagram above, the resources going to be created are:

• S3 Bucket to store all traffic logs.
• Kinesis Data Firehose[5] to deliver traffic logs from WAF WebACL to the S3.
• Cloudwatch Log Group and Stream to store the Firehose delivery error information.
• AWS Glue Catalog Database and Table which store metadata/schema of the log data.
  • One function of those resources is to make it possible the conversion from JSON to Parquet.
  • The other function is to provision Amazon Athena Database and Table which is ready to use to perform queries.
• IAM Role and Permissions for the Firehose to do all those actions above.

This module WILL NOT CREATE AWS WAF Rules and WebACL.

To get a full picture on how to make use of this module together with AWS WAF WebACL and Rules, check examples:

• WAF global: for Cloudfront
• WAF regional: for ALB and API Gateway

References

• [1] : https://docs.aws.amazon.com/waf/latest/developerguide/logging.html
• [2] : https://cwiki.apache.org/confluence/display/Hive/Parquet
• [3] : https://docs.aws.amazon.com/athena/latest/ug/columnar-storage.html
• [4] : https://docs.aws.amazon.com/athena/latest/ug/glue-athena.html
• [5] : https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html

Table of Content

• terraform-aws-waf-webacl-supporting-resources
  • Description
  • Table of Content
  • Prerequisites
  • Dependencies
  • Terraform Version
  • Getting Started
  • Requirements
  • Providers
  • Modules
  • Resources
  • Inputs
  • Outputs
  • Contributing
  • License

Prerequisites

In order to provision this module, it is require some information from an existing resources as input parameter, those resources are:

• S3 Bucket, input variable that require the information from this resource are, s3_logging_bucket
• AWS KMS, input variable that require the information from this resource are, s3_kms_key_arn

Dependencies

Doesn't have any dependencies to any other Terraform module

Terraform Versions

Created and tested using Terraform version 0.12.31 The latest stable version of Terraform which this module tested working is Terraform 0.13.7 on 2021/10/11

Getting Started

Requirements

Name	Version
terraform	>= 0.13

Providers

Name	Version
aws	n/a
random	n/a

Modules

No modules.

Resources

Name	Type
aws_cloudwatch_log_group.firehose_error_logs	resource
aws_cloudwatch_log_stream.firehose_error_logs	resource
aws_glue_catalog_database.database	resource
aws_glue_catalog_table.table	resource
aws_iam_role.firehose	resource
aws_iam_role_policy.allow_glue_get_table_versions	resource
aws_iam_role_policy.allow_put_log_events	resource
aws_kinesis_firehose_delivery_stream.waf	resource
aws_s3_bucket.webacl_traffic_information	resource
aws_s3_bucket_policy.webacl_traffic_information	resource
aws_s3_bucket_public_access_block.this	resource
random_id.this	resource
aws_caller_identity.this	data source
aws_iam_policy_document.allow_glue_get_table_versions	data source
aws_iam_policy_document.allow_put_log_events	data source
aws_iam_policy_document.allow_s3_actions	data source
aws_iam_policy_document.firehose_assume_role_policy	data source
aws_region.this	data source

Inputs

Name	Description	Type	Default	Required
description	The description of these resources.	string	n/a	yes
environment	The environment of these resources belong to.	string	n/a	yes
firehose_buffer_interval	Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. Valid value is between 60-900. Smaller value makes the logs delivered faster. Bigger value increase the chance to make the file size bigger, which are more efficient to query.	string	"900"	no
firehose_buffer_size	Buffer incoming data to the specified size, in MBs, before delivering it to the destination. Valid value is between 64-128. Recommended is 128, specifying a smaller buffer size can result in the delivery of very small S3 objects, which are less efficient to query.	string	"128"	no
product_domain	The name of the product domain these resources belong to.	string	n/a	yes
s3_kms_key_arn	KMS key ARN for S3 encryption	string	n/a	yes
s3_logging_bucket	The name of the target S3 Bucket which store Access Logs for WebACL Bucket created by this module	string	n/a	yes
service_name	The name of the service these resources belong to.	string	n/a	yes

Outputs

Name	Description
firehose_delivery_stream_arn	The ARN of Kinesis Firehose which are going to be used for delivering all traffic information from WAF WebACL to S3 bucket.
webacl_traffic_logging_bucket_name	The name of the bucket which store WebACL traffic information.

Contributing

This module accepting or open for any contributions from anyone, please see the CONTRIBUTING.md for more detail about how to contribute to this module.

License

This module is under Apache License 2.0 - see the LICENSE file for details.