vortarian / serverless-sqs-fifo

Serverless plugin to handle the create of sqs fifo queues - stop gap until Cloud Formation implements it

serverless-sqs-fifo

Summary

Setups up SQS fifo queues - at the time of this writing cloud formation does not support them, so serverless does not support them. This is considered a stop gap until cloudformation provides support.

Usage

Add the yaml to create the queues in custom.sqs.queues, then reference the queue arn & url via the <logical_name>.<url|arn>, as below with custom.sqs.queues.deaLetterQueue.arn, custom.sqs.queues.deaLetterQueue.url, custom.sqs.queues.applicationQueue.arn, custom.sqs.queues.applicationQueue.url. The queue's are created in the order they are specified and deleted in the reverse order they are specified. This is important for depdenencies. The below example includes a modified s3 crypt mod from github.com/vortarian/serverless-crypt

plugins:
  - serverless-crypt
  - serverless-sqs-fifo

custom:
  variables:
    local: ${file(./config/local.json)}
  crypt:
    keyId: ${self:custom.variables.${self:provider.stage}.kms.key}
    location: "s3://${self:provider.deploymentBucket}/crypt/${self:provider.stage}/serverless-crypt.json"
  sqs:
    queues:
      deadLetterQueue:
        QueueName: ${self:service}-DeadLetterQueue.fifo
        Properties:
          ContentBasedDeduplication: true
          FifoQueue: True
          DelaySeconds: 0
          MaximumMessageSize: 262144
          MessageRetentionPeriod: 1209600
          ReceiveMessageWaitTimeSeconds: 20
          VisibilityTimeout: 170
      applicationQueue:
        QueueName: ${self:service}-ApplicationQueue.fifo
        Properties:
          ContentBasedDeduplication: true
          FifoQueue: True
          DelaySeconds: 0
          MaximumMessageSize: 262144
          MessageRetentionPeriod: 1209600
          ReceiveMessageWaitTimeSeconds: 20
          RedrivePolicy:
            deadLetterTargetArn: custom.sqs.queues.deaLetterQueue.arn
            maxReceiveCount: 2
          VisibilityTimeout: 170
provider:
  name: aws
  runtime: nodejs4.3
  stage: ${env:STAGE} # Set the default stage used. Default is dev
  region: ${env:AWS_REGION} # Overwrite the default region used. Default is us-east-1
  profile: ${env:AWS_PROFILE} # The default profile to use with this service
  memorySize: 512 # Overwrite the default memory size. Default is 1024
  timeout: 30 # The default is 6
  deploymentBucket: ${env:STAGE}-serverless-${env:AWS_ACCOUNT}-${self:provider.region}.deploy # Deployment bucket name. Default is generated by the framework
  versionFunctions: true # Optional function versioning
  environment: # Service wide environment variables
    CRYPT_LOCATION: "${self:custom.crypt.location}"
  stackTags: # Optional CF stack tags
   env: ${env:STAGE}
   service: ${self:service}
  iamRoleStatements: # IAM role statements so that services can be accessed in the AWS account
    -  Effect: 'Allow'
       Action:
         - 'kms:decrypt'
       Resource:
         - Fn::Join: [":", [ "arn:aws:kms", {"Ref": "AWS::Region"}, {"Ref": "AWS::AccountId"}, "key/${self:custom.variables.${self:provider.stage}.kms.key}"] ]
    -  Effect: 'Allow'
       Action:
         - 's3:Get*'
       Resource:
         - "arn:aws:s3:::${self:provider.deploymentBucket}/crypt/${self:provider.stage}/serverless-crypt.json"
    - 
      Effect: "Allow"
      Action:
        - "sqs:ChangeMessageVisibility"
        - "sqs:ChangeMessageVisibilityBatch"
        - "sqs:DeleteMessage"
        - "sqs:DeleteMessageBatch"
        - "sqs:GetQueueAttributes"
        - "sqs:GetQueueUrl"
        - "sqs:ReceiveMessage"
        - "sqs:SendMessage"
        - "sqs:SendMessageBatch"
      Resource:
        - custom.sqs.queues.applicationQueue.arn
        - custom.sqs.queues.deadapplicationQueue.arn


functions:
  processQueue:
    handler: processQueue.handler
    memorySize: 128 # memorySize for this specific function.
    timeout: 240 # Timeout for this specific function.  Overrides the default set above.
    environment: # Function level environment variables
      SANDBOX: 1
      QUEUE_URL: custom.sqs.queues.applicationQueue.url
    events: 
      - schedule:
          rate: rate(1 minute) # Keep this frequent cause we want lambdas to be around to deal with failures
        enabled: true
      - sns:
          topicName: "R${env:STAGE}TriggerProcessQueue"
          displayName: "trigger"
        enabled: true