terraform-aws-ml-model

This is an example module to accompany the Code & Wild blog post Building a Machine Learning Orchestration Platform: Part 1.

As mentioned on the blog post, this module makes a few assumptions when it comes to naming conventions and to some of the monitoring tools used which are external to AWS, like Datadog for managing your application logs.

The module will link a GitHub repository to a CodePipeline pipeline which will check out the code and run a CodeBuild project that will execute the codebuild/buildspec.yml file. An example GitHub repository that can be combined with this module can be found in our public GitHub profile, under the name opensource-ml-model-template. That repository will allow the pipeline to build and push a Docker image to ECR which then can be run via ECS Fargate.

The module also creates an ECR repository, an ECS task definition and some other supporting infrastructure.

Requirements

Name	Version
terraform	~> 1.0.0
aws	~> 3.0

Providers

Name	Version
aws	~> 3.0

Modules

No modules.

Resources

Name	Type
aws_codebuild_project.build	resource
aws_codepipeline.this	resource
aws_ecr_repository.this	resource
aws_ecs_task_definition.primary	resource
aws_iam_policy.additional_ecs_task_policy	resource
aws_iam_policy.codebuild_policy	resource
aws_iam_policy.codepipeline_policy	resource
aws_iam_policy.ecs_execution_policy	resource
aws_iam_policy.ecs_task_policy	resource
aws_iam_policy_attachment.codebuild_policy_attachment	resource
aws_iam_policy_attachment.codepipeline_policy_attachment	resource
aws_iam_role.codebuild_role	resource
aws_iam_role.codepipeline_role	resource
aws_iam_role.ecs_execution_role	resource
aws_iam_role.ecs_task_role	resource
aws_iam_role_policy_attachment.execution_role_allow_ecs_execution	resource
aws_iam_role_policy_attachment.task_role_allow_additional_ecs_task	resource
aws_iam_role_policy_attachment.task_role_allow_ecs_task	resource
aws_s3_bucket.app	resource
aws_s3_bucket.pipeline	resource
aws_caller_identity.current	data source
aws_ecs_task_definition.primary	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
additional_iam_policy	Additional IAM policy to attach to the execution role of the ECS task	`string`	`""`	no
codebuild_security_group_ids	Security Group IDs where we want the CD Pipeline CodeBuild Project to run in	`list(string)`	n/a	yes
codebuild_subnets	Subnets where we want the CD Pipeline CodeBuild Project to run in	`list(string)`	n/a	yes
codebuild_vpc_id	VPC id where we want the CD Pipeline CodeBuild Project to run in	`string`	n/a	yes
codestar_connection_arn	AWS CodeStar Connection ARN to use to connect GitHub with the CD Pipeline	`string`	n/a	yes
cpu	CPU allocation for the ECS task	`string`	`"256"`	no
datadog_api_key	Datadog API key to use for the Firelens log driver	`string`	n/a	yes
environment	Environment of the model (e.g. production, staging, development)	`string`	`"production"`	no
github_branch	GitHub branch that should trigger the CD Pipeline	`string`	`"main"`	no
github_repository	GitHub repository for the model code	`string`	n/a	yes
memory	Memory allocation for the ECS task	`string`	`"512"`	no
model_name	Name that uniquely identifies the ML model	`string`	n/a	yes
parameter_store_kms_key	KMS key ID to use to decrypt SSM parameters at run time on the ECS task	`string`	n/a	yes

Outputs

No outputs.

Example usage

Replace values as required.

module "ml_model" {
  source = "github.com/BloomAndWild/terraform-aws-ml-model"

  model_name        = "example-model"
  github_repository = "BloomAndWild/opensource-ml-model-template"
  codestar_connection_arn = "arn:aws:codestar-connections:eu-west-1:XXXXXXXXXXXX:connection/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  codebuild_vpc_id = "vpc-xxxxxxx"
  codebuild_security_group_ids = ["sg-xxxxxxx"]
  codebuild_subnets = ["subnet-xxxxxxxxxxxx", "subnet-xxxxxxxxxxxx"]
  datadog_api_key = "xxxxxxxxxxxxxxxxxxxxxx"
  parameter_store_kms_key = "xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

BloomAndWild / terraform-aws-ml-model