crucible
Create better cloudformation templates with Clojure
Installation
Crucible depends on clojure.spec, currently available in Clojure 1.9 alpha 10+ (breaking changes in spec around alpha 9)
Examples
(ns crucible.examples-test
(:require [crucible.core :refer [template parameter resource output xref encode join]]
[crucible.aws.ec2 :as ec2]))
(def simple (template "A simple sample template"
:my-vpc-cidr (parameter)
:my-vpc (ec2/vpc {::ec2/cidr-block (xref :my-vpc-cidr)})
:vpc (output (join "/" ["foo" (xref :my-vpc)]))))
repl> (clojure.pprint/pprint (encode simple))
{"AWSTemplateFormatVersion" "2010-09-09"
"Description" "A simple sample template"
"Parameters" {"MyVpcCidr" {"Type" "String"}}
"Resources" {"MyVpc"
{"Type" "AWS::EC2::VPC",
"Properties" {"CidrBlock" {"Ref" "MyVpcCidr"}}}},
"Outputs" {"Vpc" {"Value" {"Fn::Join" ["/" ["foo" {"Ref" "MyVpc"}]]}}}}
Alternative template construction function accepts map and string arguments for building a template from partials, for example:
(def simple (-> {:my-vpc-cidr (parameter)}
(assoc :igw (ec2/internet-gateway {}))
(assoc :my-vpc (ec2/vpc {::ec2/cidr-block (xref :my-vpc-cidr)}))
(assoc :vpc (output (join "/" ["foo" (xref :my-vpc)])))
(template "A simple sample template")))
Parameter Options
See crucible.parameters
namespace, required as param
in this example:
:my-vpc-cidr (parameter ::param/type ::param/number
::param/description "A demonstration of parameter options"
::param/allowed-values [1 2 3]
::param/no-echo true)
Resource Policies
See crucible.policies
namespace, required as policies
in this example:
:my-vpc (ec2/vpc {::ec2/cidr-block (xref :my-vpc-cidr)}
(policies/deletion ::policies/retain)
(policies/depends-on :my-vpc-cidr))
Resource Types
Standard AWS resource types can be found as children of the crucible.aws namespace.
Examples of resource type usage can be found in the tests.
- AWS::EC2::* partial coverage
- AWS::ElasticLoadBalancingV2::*
- AWS::ApiGateway::*
- AWS::DynamoDB::Table
- AWS::CloudWatch::Alarm
- AWS::Lambda::Function
- AWS::Lambda::EventSourceMapping
- AWS::IAM::Role (basic support for Lambda applications)
- AWS::ECR::Repository
- AWS::S3::Bucket
- AWS::CloudFormation::Stack
- AWS::Kinesis::Stream
- AWS::KinesisFirehose::DeliveryStream
- AWS::Route53::RecordSet
- AWS::SNS::Topic
- AWS::SNS::TopicPolicy
- AWS::SQS::Queue
- AWS::Events::Rule
- AWS::AutoScaling::AutoScalingGroup/LaunchConfiguration
- Custom::* custom resources
Writing your own resource type
The easiest way is to use defresource
and spec-or-ref
from the crucible.resources
namespace, eg.
(ns crucible.aws.ec2
"Resources in AWS::EC2::*"
(:require [crucible.resources :refer [spec-or-ref defresource] :as res]
[clojure.spec :as s]))
;; spec-or-ref applies your spec if a literal value is given,
;; but also allows a parameter or function to be given instead of a literal.
(s/def ::cidr-block (spec-or-ref string?))
;; ::res/tags reuses the tags spec defined in the crucible/resources namespace
(s/def ::vpc (s/keys :req [::cidr-block]
:opt [::enable-dns-support
::enable-dns-hostnames
::instance-tenancy
::res/tags]))
;; creates resource factory crucible.aws.ec2/vpc with type "AWS::EC2::VPC"
;; and validates the data structure using the ::vpc spec
(defresource vpc "AWS::EC2::VPC" ::vpc)
Pull requests to add or enhance resource types available in Crucible will be welcomed. If it's a standard AWS type please place it in the crucible.aws
namespace and use defresource
as it documents the resource type for you. At lest one test for the update would be great!
Testing a Resource Type
Although you can test a resource by testing the data structure and validity directly, testing the conversion from the crucible code to a map ready to encode as CloudFormation-valid JSON is good minimal coverage. The default clojure.test
behaviour does not pretty-print the ex-data
map on validation exceptions, which makes testing and debugging validation failures painful. A custom assertion crucible.assertion/resource=
is available to ensure any failure map is pretty-printed. See the resource tests for examples.
Overriding JSON Keys
Crucible uses camel-snake-kebab's ->PascalCase
function to convert
Clojure map keys into JSON map keys. That takes care of most
translations between Clojure-style :keyword-key
and
JSON/CloudFormation-style KeywordKey
. To handle the occasional
mistranslation, typically due to capitalisation,
clojure.encoding.keys
exposes a ->key
multimethod, allowing
overriding of the translation. For example, this problem occurs in
AWS::CloudFormation::Stack, where a required key is TemplateURL
. The
following overrides the natural translation of :template-url
to
TemplateUrl
.
(ns crucible.aws.cloudformation
(:require [crucible.encoding.keys :refer [->key]]))
(defmethod ->key :template-url [_] "TemplateURL")
Note, these translations take place during the final JSON encoding step and do not see keyword namespacing.
Severless Application Model templates
The AWS Serverless Application Model (SAM) is supported by using SAM resources and the SAM encoder. All CloudFormation resources are valid SAM resources and they can be combined in the same template. The crucible.core/template
function can be used to generate a template data structure for SAM resources.
Globals
AWS SAM supports global properties for functions and APIs. There is a two-arity version of build
and encode
in the SAM encoder that accepts a template and a globals object.
Example
In the example below, the second argument to encoding.sam/build
can be omitted if globals are not used.
(ns crucible.sam-example
(:require [crucible.aws.kinesis :as k]
[crucible.aws.serverless.function :as f]
[crucible.aws.serverless.function.event-source :as es]
[crucible.aws.serverless.function.event-source.kinesis :as es.k]
[crucible.aws.serverless.globals :as g]
[crucible.core :refer [template xref]]
[crucible.encoding.serverless :as encoding.sam]))
(-> {:stream-processor (f/function
{::f/handler "index.handler"
::f/runtime "nodejs6.10"
::f/code-uri "src/"
::f/events {:stream {::es/type "Kinesis"
::es.k/properties {::es.k/stream (xref :stream :arn)
::es.k/starting-position "TRIM_HORIZON"}}}})
:stream (k/stream {::k/shard-count 1})}
(template "A function that processes data from a Kinesis stream.")
(encoding.sam/build (g/globals {::g/function {::f/memory-size 1024
::f/timeout 15}})))
CLI Support
Basic CLI support, intended for use with Leiningen, is provided in the crucible.encoding.main/-main
function. Running this function will reload the namespaces available in the project, then enumerate any vars that have a metadata tag provided by the crucible.core/template
function. These vars are then encoded into CloudFormation templates and exported to the local filesystem. They can then be used directly or uploaded to S3 for use with CloudFormation.
Flag -h
for help. Templates are exported to target/templates
by default, override with -o output-dir
. Namespaces are converted to filesystem locations by replacing .
characters with /
characters.
I create a templates directory within my project and then add it as a source-path and crucible as a dependency to the dev profile. Then I can work at the repl, write tests for my templates and use this tooling without having my template code or crucible mixed with my source code.
demo-project is an example of setting up a project with crucible templates defined alongside the code. The templates can see the code to verify any references they might have, but not the other way round. Run lein templates
in the demo project to generate templates.
:aliases {"templates" ["run" "-m" crucible.encoding.main]}
:profiles {:dev {:source-paths ["templates"]
:dependencies [[crucible "0.10.0-SNAPSHOT"]]}}
Helping Out
Any help appreciated! Happy to receive any issues and pull requests. See CONTRIBUTING.md.
License
Distributed under the Eclipse Public License