Deploy with VPC and custom role fails

[Jakub Matějka]

This is a Bug Report

Description

Deploy fails on Requires capabilities : [CAPABILITY_NAMED_IAM]. I want to deploy functions to a VPC so my serverless.yml looks like this:

provider:
  role: developerPortalLambdaRole
  vpc:
    securityGroupIds:
      - ${file(./env.yml):VPC_SECURITY_GROUP}
    subnetIds:
      - ${file(./env.yml):VPC_SUBNET1}
      - ${file(./env.yml):VPC_SUBNET2}
resources:
  Resources:
    developerPortalLambdaRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: DeveloperPortalLambdaRole
        AssumeRolePolicyDocument:
          Version: '2017'
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - lambda.amazonaws.com
              Action: sts:AssumeRole
        Policies:
          - PolicyName: DeveloperPortalLambdaPolicy
            PolicyDocument:
              Version: '2017'
              Statement:
                - Effect: Allow
                  Action:
                    - logs:CreateLogGroup
                    - logs:CreateLogStream
                    - logs:PutLogEvents
                  Resource: "arn:aws:logs:${file(./env.yml):REGION}:${file(./env.yml):ACCOUNT_ID}:log-group:/aws/lambda/*:*:*"
                - Effect: Allow
                  Action:
                    - ec2:CreateNetworkInterface
                    - ec2:DescribeNetworkInterfaces
                    - ec2:DetachNetworkInterface
                    - ec2:DeleteNetworkInterface
                    - ses:SendEmail
                  Resource: "*"
                - Effect: Allow
                  Action:
                    - "s3:*"
                  Resource:
                    - "arn:aws:s3:::${file(./env.yml):S3_BUCKET}/*"
                - Effect: Allow
                  Action:
                    - "cognito-identity:*"
                    - "cognito-idp:*"
                  Resource:
                    - "arn:aws:cognito-idp:${file(./env.yml):REGION}:${file(./env.yml):ACCOUNT_ID}:userpool/${file(./env.yml):COGNITO_POOL_ID}"

(Complete file is here: https://github.com/keboola/developer-portal/blob/68b88cb7f0fa088235f7d8e887176f75921d757b/serverless.yml)

Originally I had all policies in iamRoleStatements but putting ec2.CreateNetworkInterface there (https://github.com/keboola/developer-portal/blob/4c3be3b73de511e121dc55e2050d26c661eec546/serverless.yml#L37) didn't work, deploy was failing on Your access has been denied by EC2, please make sure your function execution role have permission to CreateNetworkInterface. So I rewrote it to use custom role according to https://serverless.com/framework/docs/providers/aws/guide/iam#provide-a-default-role-for-all-lambdas-except-those-overriding-the-default but it doesn't work either.

Additional Data

• Serverless Framework Version you're using: 1.1.0
• Operating System: macOS 10.12.1
• Stack Trace:
• Provider Error messages:

[Philipp Muens]

Hey @JakubMatejka thanks for reporting this. 👍

Can you check out the latest master and try to re-deploy again? We've added the CAPABILITY_NAMED_IAM recently in #2584.

Furthermore we're working on making this easier pretty soon! (see #2127).

[Jakub Matějka]

Hi @pmuens, that's great. But I'm afraid there is another problem in master. Deploy fails on Template error: instance of Fn::GetAtt references undefined resource AppsversionsrollbackLambdaFunction and stack trace again does not say anything interesting (fails on lib/plugins/aws/provider/awsProvider.js:142 ant the rest of it is from aws-sdk).

[Philipp Muens]

Hmm... @JakubMatejka thanks for reporting! 👍

That definitely needs further investigation!

/cc @eahefnawy and @flomotlik

[marjanSterjev]

I think this is the same CloudFormation resource creation concurrency issue (see #2697)

[Nick den Engelsman]

@pmuens As @marjanSterjev mentioned in his issue, we also managed to get around it with the DependsOn properties. I'll happily send in a PR tomorrow ;) let me know if that's ok?

[Philipp Muens]

@nicka sounds good! 👍

Thanks for jumping into this! 💯

BTW. We have another dependsOn issue with lambda permissions and events (The PR for that is here: #2704).

[Nick den Engelsman]

@pmuens Since #2127 is already in progress might make sense to put the DependsOn within the same PR? Especially if we want to combine profile and policy in one resource.

[Philipp Muens]

@nicka thanks for the suggestion. 👍

Yes, I think it would be a good place to put this as well!

[Jeremy Daly]

Same problem. Deploying without VPC config to create the IAM role and then redeploying with VPC info works.

[Nick den Engelsman]

@jeremydaly Could you share your serverless config? Because I do expect one issue with a missing DependsOn. Working on #2743 which should fix some of the default issues.

[Jeremy Daly]

Here is the relevant section:

provider:
  name: aws
  runtime: nodejs4.3
  stage: dev
  region: us-east-1
  cfLogs: true
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "logs:CreateLogGroup"
        - "logs:CreateLogStream"
        - "logs:PutLogEvents"
        - "ec2:CreateNetworkInterface"
        - "ec2:DescribeNetworkInterfaces"
        - "ec2:DeleteNetworkInterface"
        - "lambda:InvokeFunction"
      Resource: "*"
  vpc:
    securityGroupIds:
      - sg-XXXXXXXX
    subnetIds:
      - subnet-XXXXXXX
      - subnet-XXXXXXX
      - subnet-XXXXXXX

[Nick den Engelsman]

@jeremydaly This should def be fixed when #2743 is finalized 👍

[Maciej Winnicki]

Fixed in #2743. Will be released with 1.3 version.

[Philipp Muens]

Should be fixed with #2743 thanks to @nicka 🎉

[jarrett jordaan]

Hi,

Has the initial issue ben resolved? I'm having a similar issue following https://serverless.com/framework/docs/providers/aws/guide/iam guide.

It can't create that PolicyDocument in the guide. There seems to be a syntax error. Just tried the first example. "One Custom IAM Role For All Functions".

First error:

• The policy must contain a valid version string.

After some googling, I changed the Version to "2012-10-17" instead of "2017" just below AssumeRolePolicyDocument and PolicyDocument.

Second error:

• Syntax errors in policy.

If this is the wrong place to ask, where would one ask help on documentation issues? Any help would be much appreciated. Thanks.

Regards.
JJ

[Philipp Muens]

Hey @jarrettj I think your problem is related to #3450

Could you export the SLS_DEBUG=* env variable share your policy here? Thanks!

[jarrett jordaan]

Thanks for the reply. The problem is with the documentation. There is a typo in the policy. Spent a couple of hours trying to figure that out. The 2nd "Resources" should be "Resource".

[Philipp Muens]

@jarrettj thanks for following up! Hmm that's a nasty one 😬

Do you want to PR a change? Thanks!

[jarrett jordaan]

Cool man, will do. All part of learning. :)

[Philipp Muens]

Thanks for that @jarrettj Merged!

[Adam Biggs]

I'm getting this error now using Serverless v1.40.0. Trying to deploy a service with VPC functions and a custom IAM role defined in resources.

I tried rolling back to v1.39.1 and got the same issue. I'll investigate some more tomorrow.

[Adam Biggs]

For anyone else who might run into this, the problem was Version of the custom PolicyDocument in our serverless.yml wasn't wrapped in quotes.

When defined like Version: 2012-10-17, Serverless parsed the value as a date and tried to set it as 2012-10-17T00:00:00.000Z in the cloud formation template. Changing the value to Version: "2012-10-17" fixed the issue.