amplify-education / serverless-log-forwarding

Serverless plugin for forwarding CloudWatch logs to another Lambda function.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error: The log group provided is reserved for the function logs of the destination function.

richburdon opened this issue · comments

#I'm working through the README example and hitting the error below on deploy:

Is there a working example anywhere? (I don't see a reference in any of your other repos).

An error occurred: SubscriptionFilterForwarder - The log group provided is reserved for the function logs of the destination function..

Here's my config:

custom:
  logForwarding:
    destinationARN: "arn:aws:lambda:us-east-1:XXX:function:xxx-metrics-dev-forwarder"
    stages:
      - "dev"

functions:
  forwarder:
    handler: handler.forwarder

resources:
  Resources:
    ForwarderLambdaPermission:
      Type: "AWS::Lambda::Permission"
      Properties:
        Action: "lambda:InvokeFunction"
        FunctionName:
          Fn::GetAtt:
            - ForwarderLambdaFunction
            - Arn
        Principal: logs.${self:provider.region}.amazonaws.com

It seems that in the cloud formation file there's a cyclic dependency? (although I haven't fully understood this file yet).

    "ForwarderLogGroup": {
      "Type": "AWS::Logs::LogGroup",
      "Properties": {
        "LogGroupName": "/aws/lambda/wireline-metrics-dev-forwarder"
      }
    },

    ...
    "ForwarderLambdaFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Code": {
          "S3Bucket": {
            "Ref": "ServerlessDeploymentBucket"
          },
          "S3Key": "serverless/wireline-metrics/dev/1511053240560-2017-11-19T01:00:40.560Z/wireline-metrics.zip"
        },
        "FunctionName": "wireline-metrics-dev-forwarder",
        "Handler": "handler.forwarder",
        "MemorySize": 128,
        "Role": {
          "Fn::GetAtt": [
            "IamRoleLambdaExecution",
            "Arn"
          ]
        },
        "Runtime": "nodejs6.10",
        "Timeout": 30,
        "Description": "Converts CloudWatch logs into metrics."
      },
      "DependsOn": [
        "ForwarderLogGroup",
        "IamRoleLambdaExecution"
      ]
    },

    ....

    "SubscriptionFilterForwarder": {
      "Type": "AWS::Logs::SubscriptionFilter",
      "Properties": {
        "DestinationArn": "arn:aws:lambda:us-east-1:370180315098:function:wireline-metrics-dev-forwarder",
        "FilterPattern": "",
        "LogGroupName": "/aws/lambda/wireline-metrics-dev-forwarder"
      },
      "DependsOn": [
        "LogForwardingLambdaPermission",
        "ForwarderLogGroup"
      ]
    }

Another thing: even if this worked, how would there not be a cycle where CL logs trigger another call to the ForwarderLambdaFunction after each time it runs.

NOTE: I think the docs should explain that the FN::GetAtt function name must correspond to the capitalized function name with the LambdaFunction suffix (i.e., "forwarder" => "ForwarderLambdaFunction"). That wasn't obvious; is this a serverless convention or an AWS thing?

Agh, I think I just answered my question?

how would there not be a cycle where CL logs trigger another call to the ForwarderLambdaFunction after each time it runs.

The forwarder must be in a different stage than the funcitons it's subscribing to?

Now, deploy works, but when I try to log the function I get a different error.

sls logs -f formatter
No existing streams for the function

@richburdon hi!
I have got the same SubscriptionFilterForwarder - The log group provided is reserved for the function logs of the destination function.. error, but can not understand how did you fix it, can you give me some hints, thank you!

@stremglav I think the intended use case is that you use this plugin in a separate serverless project to the one that the destination function is defined in, e.g.

  • project one, defines the destination function, does not use this plugin
  • project two, defines other functions, uses this plugin to forward their logs to the function from the first function

This could be better explained in the docs, and could possibly be improved by the addition of some sort of excludePattern option which could be used to exclude functions from being forwarded (which would allow you to have them all in the same project if you wanted to, by excluding the destination function).