Deploying policy as lambda through c7n-org produces erratic resource count

[Boris Erdmann]

Given for example this policy:

policies:
- name: cloudwatch-set-log-group-retention-test
  mode:
    execution-options:
      metrics_enabled: true
      output_dir: s3://moia-cloud-custodian-reports/{account_id}/
    type: periodic
    schedule: "rate(1 hour)"
    role: arn:aws:iam::{account_id}:role/rol-cloud-custodian-execution
  resource: log-group
  tags: region:eu-west-1
  filters:
    - type: value
      key: "retentionInDays"
      op: not-equal
      value: 14

and deploying it via c7n-org with e.g. this accounts.yaml:

accounts:
- account_id: '123456789012'
  name: '123456789012'
  role: 'arn:aws:iam::123456789012:role/rol-custodian-pipeline'

will actually deploy the lambda function but also reports:

2018-06-19 13:19:04,688: c7n_org:DEBUG Running policy:cloudwatch-set-log-group-retention-test account:123456789012 region:eu-west-1
2018-06-19 13:19:09,201: c7n_org:INFO Ran account:123456789012 region:eu-west-1 policy:cloudwatch-set-log-group-retention-test matched:17 time:4.51
2018-06-19 13:19:09,275: c7n_org:INFO Policy resource counts Counter({'cloudwatch-set-log-group-retention-test': 17})

The resource count seems unexpected, since no policy is actually being executed, but a lambda is being deployed.

[Kapil Thangavelu]

thanks for filing an issue, I've noticed that before and agree is less than ideal as a ux/cli output without additional context.

[Samindra Dey]

Hi Kapilt,

Can you please let me know how to log executions in multi-account?

I am executing from a central account in a aws organization with following options:-
c7n-org run -c accounts.yml -u policy/terminate_pending_state_ec2_new.yml -s s3://xxx-cloudcustodian/CloudCustodianLogs/
This is creating a file in s3 but the actual Lambdas from the resource accounts are not writing to S3.
Following are the content:-
accounts.yml

accounts:

• account_id: '123456789012'
  name: 123456789012
  regions:
  • eu-west-1
  • eu-west-2
    role: arn:aws:iam::123456789012:role/cloudcustodian-role

terminate_pending_state_ec2_new.yml

policies:

• name: subnet-audit
  resource: ec2
  mode:
  type: ec2-instance-state
  events:
  - running
  execution-options:
  metrics_enabled: true
  output_dir: s3://xxx-cloudcustodian/CloudCustodianLogs/123456789012/
  type: periodic
  schedule: "rate(1 hour)"
  filters:
  • type: subnet
    key: "tag:Location"
    value: "Internet"
    actions:
  • terminate

arn:aws:iam::123456789012:role/cloudcustodian-role have following role

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::xxx-cloudcustodian/CloudCustodianLogs/*"
}
]
}

xxx-cloudcustodian bucket policy:-

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::212121212121:role/cloud-custodian-cross-account-role",
"arn:aws:iam::123456789012:role/cloudcustodian-role"
]
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::xxx-cloudcustodian/CloudCustodianLogs/*"
}
]
}

Although the policy is doing what is expected but the Lambda logs are not written to S3. Can you please suggest?

[Kapil Thangavelu]

resolved, this was specific to lambda execution modes re odd resource counts