Ansible Role: christiangda.amazon_cloudwatch_agent

This role Install AWS CloudWatch Agent

The best wayt to install this role is using the command ansible-galaxy install christiangda.amazon_cloudwatch_agent, the Ansible Galaxy repository is christiangda.amazon_cloudwatch_agent

The repository code is https://github.com/christiangda/ansible-role-amazon-cloudwatch-agent

Features:

Works on AWS EC2 instances/On-Premise Instances
Support different versions of Python, Ansible and Operating Systems (see the Continuos integration matrix for details)
Downloads and installs CloudWatch Agent from AWS distribution package
Reload service when configuration is changed
Provides a default agent configuration file (a minimal configuration, It is not recommended)
Rotate CloudWatch Agent Log file
Allow you to load your own JSON file / YAML file or INLINE configuration for agent, metrics and logs sections

Notes:

The version 2.x.y is not compatible with version 1.x.y
Since version 2.x.y this role no longer installs EPEL Repository by default, now you need to take care of this, I recommend my ansible role christiangda.epel_repo instead
Since version 2.x.y this role no longer creates AWS CLI profile (config and credentials), now you need to take care of this, I recommend my ansible roles christiangda.awscli and christiangda.awscli_configure instead. See examples under (when cwa_agent_mode: "onPremise")
Installs automatically the collectd OS package when detecting the use of Retrieve Custom Metrics with collectd in the configuration file. Depending on the OS the collectd package could need EPEL Repository
RedHat users It is important to have a valid subscription in order to avoid dependencies packages installation problems.

More details in VERSION.md

Requirements

This role work on RedHat, CentOS, Amazon Linux, Debian and Ubuntu distributions

RedHat
- 6 (*)
- 7
- 8
CentOS
- 6 (*)
- 7
- 8
Amazon Linux
- 1 (*)
- 2
Ubuntu
- 14.04 (*)
- 16.04 (*)
- 18.04
- 19.04
Debian
- jessie (8) (*)
- stretch (9)
- buster (10)
- sid (unstable)

(*) Out of tests, maybe you find problems with this OS

To see the compatibility matrix of Python vs. Ansible versions see Travis-CI build matrix

Role Variables

Variable	Default Value
cwa_conf_json_file_content	"" --> Empty
cwa_agent_mode	"ec2"
cwa_aws_region	"eu-west-1"
cwa_use_credentials	false
cwa_profile	"AmazonCloudWatchAgent"
cwa_agent_profile_path	/root
cwa_http_proxy	"" --> Empty
cwa_https_proxy	"" --> Empty
cwa_no_proxy	"169.254.169.254"
cwa_logrotate_file_size	"10M"
cwa_logrotate_files	5

More Details: See the file defaults/main.yaml

Dependencies

In case of OS Family RedHat/Centos EPEL Repository could be necessary
If you set cwa_agent_mode: "onPremise" the AWS CLI Profile configuration is needed
If you Retrieve Custom Metrics with collectd the package collectd will installed automatically and depending of the OS the EPEL Repository could be necessary

Example Playbook

When cwa_agent_mode: "onPremise"

# You should use ansible-vault to provision it
# Example:
# ansible-vault encrypt_string --ask-vault-pass --name 'cwa_access_key' 'AKIAIOSFODNN7EXAMPLE'
# ansible-vault encrypt_string --ask-vault-pass --name 'cwa_secret_key' 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'

- hosts: servers
    gather_facts: True
    roles:
      - role: christiangda.epel_repo # If you don't have installed EPEL Repository
        when: >
          ansible_os_family == 'RedHat' and (
            ansible_distribution == 'CentOS' or
            ansible_distribution == 'RedHat' or
            ansible_distribution == 'Amazon'
          )
      - role: christiangda.awscli_configure # If you don't have configured AWS CLI Profiles
        vars:
          awscliconf_path: '/root'
          awscliconf_files:
            credentials:
              - AmazonCloudWatchAgent:
                  aws_access_key_id: !vault |
                    $ANSIBLE_VAULT;1.1;AES256
                    30376338613338326663373366303234623665633339303338613463313564633832363237306137
                    3362643039616631323339383332306536333962346133310a383265376665316235653261616136
                    61306566623531356263346439633761633830323636646236373736353530396134636536666532
                    3939636433636364310a316639366139366566623337623536346661633339343766323936346336
                    65333035366635396138656132643262626438333961326266396466626464643766
                  aws_secret_access_key: !vault |
                    $ANSIBLE_VAULT;1.1;AES256
                    65643230613939303737336632346432393234616437383532386139616364316233333933643735
                    6633636261383163323362623562323333323533336564310a323030343431366135343035326635
                    33623161376634643939636464306139386662303034616531346632303039643238373834616266
                    3064623232373233610a346432646565396235316631626137653731376365333531323866626665
                    62656638623330643539653763636364363738653932653831316238633939356462653636633463
                    6130613761633565616533633332376565373062396565396261
            config:
              - profile AmazonCloudWatchAgent:
                  region: "eu-west-1"
      - role: christiangda.amazon_cloudwatch_agent # Using minimal agent configuration provided by the role
          vars:
              cwa_agent_mode: "onPremise"
              cwa_aws_region: "eu-west-1"
              cwa_profile: "AmazonCloudWatchAgent"

When cwa_agent_mode: "ec2"

Reading config file from JSON configuration file

---
- hosts: servers
    gather_facts: True
    roles:
      - role: christiangda.amazon_cloudwatch_agent # Using a JSON file provided by you at location `files/CloudWatch.json` relative to this playbook
          vars:
              cwa_agent_mode: "ec2"
              cwa_conf_json_file_content: "{{ lookup('file', 'files/CloudWatch.json') | from_json }}"

---
- hosts: servers
    gather_facts: True
    roles:
      - role: christiangda.epel_repo
        when: >
          ansible_os_family == 'RedHat' and (
            ansible_distribution == 'CentOS' or
            ansible_distribution == 'RedHat' or
            ansible_distribution == 'Amazon'
          )
      - role: christiangda.awscli_configure
        vars:
          awscliconf_path: '/root'
          awscliconf_files:
            credentials:
              - AmazonCloudWatchAgent:
                  aws_access_key_id: 'AKIAIOSFODNN7EXAMPLE'
                  aws_secret_access_key: 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'
      - role: christiangda.amazon_cloudwatch_agent # Using minimal agent configuration provided by the role
          vars:
              cwa_agent_mode: "ec2"

Reading config file from YAML configuration file

---
- hosts: servers
    gather_facts: True
    roles:
      - role: christiangda.amazon_cloudwatch_agent # Using a YAML file provided by you at location `files/CloudWatch.yaml` relative to this playbook
          vars:
              cwa_agent_mode: "ec2"
              cwa_conf_json_file_content: "{{ lookup('file', 'files/CloudWatch.yaml') | from_yaml }}"

Using INLINE YAML configuration file

---
- hosts: centos7, centos6, ubuntu1804, ubuntu1810, debian8, debian9, amzn2
  become: True
  roles:
    - role: christiangda.epel_repo # If you don't have installed EPEL Repository
        when: >
          ansible_os_family == 'RedHat' and (
            ansible_distribution == 'CentOS' or
            ansible_distribution == 'RedHat' or
            ansible_distribution == 'Amazon'
          )
      - role: christiangda.awscli_configure # If you don't have configure AWS CLI Profiles
        vars:
          awscliconf_path: '/root'
          awscliconf_files:
            credentials:
              - AmazonCloudWatchAgent:
                  aws_access_key_id: !vault |
                    $ANSIBLE_VAULT;1.1;AES256
                    30376338613338326663373366303234623665633339303338613463313564633832363237306137
                    3362643039616631323339383332306536333962346133310a383265376665316235653261616136
                    61306566623531356263346439633761633830323636646236373736353530396134636536666532
                    3939636433636364310a316639366139366566623337623536346661633339343766323936346336
                    65333035366635396138656132643262626438333961326266396466626464643766
                  aws_secret_access_key: !vault |
                    $ANSIBLE_VAULT;1.1;AES256
                    65643230613939303737336632346432393234616437383532386139616364316233333933643735
                    6633636261383163323362623562323333323533336564310a323030343431366135343035326635
                    33623161376634643939636464306139386662303034616531346632303039643238373834616266
                    3064623232373233610a346432646565396235316631626137653731376365333531323866626665
                    62656638623330643539653763636364363738653932653831316238633939356462653636633463
                    6130613761633565616533633332376565373062396565396261
            config:
              - profile AmazonCloudWatchAgent:
                  region: "eu-west-1"
    - role: christiangda.amazon_cloudwatch_agent # Using ONLINE YAML configuration for your agent
      vars:
        cwa_agent_mode: onPremise
        cwa_aws_region: "eu-west-1"
        cwa_profile: "AmazonCloudWatchAgent"
        cwa_conf_json_file_content:
          agent:
            metrics_collection_interval: 60
            region: es-west-1
            logfile: "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
            debug: false
          metrics:
            metrics_collected:
              cpu:
                resources:
                  - "*"
                measurement:
                  - name: cpu_usage_idle
                    rename: CPU_USAGE_IDLE
                    unit: Percent
                  - name: cpu_usage_nice
                    unit: Percent
                  - cpu_usage_guest
                totalcpu: false
                metrics_collection_interval: 10
                append_dimensions:
                  test: test1
                  date: "2017-10-01"
              netstat:
                measurement:
                  - tcp_established
                  - tcp_syn_sent
                  - tcp_close
                metrics_collection_interval: 60
              processes:
                measurement:
                  - running
                  - sleeping
                  - dead
            append_dimensions:
              ImageId: "${!aws:ImageId}"
              InstanceId: "${!aws:InstanceId}"
              InstanceType: "${!aws:InstanceType}"
              AutoScalingGroupName: "${!aws:AutoScalingGroupName}"
            aggregation_dimensions:
              - - AutoScalingGroupName
              - - InstanceId
                - InstanceType
              - []
          logs:
            logs_collected:
              files:
                collect_list:
                  - file_path: "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
                    log_group_name: amazon-cloudwatch-agent.log
                    log_stream_name: amazon-cloudwatch-agent.log
                    timezone: UTC
                  - file_path: "/opt/aws/amazon-cloudwatch-agent/logs/test.log"
                    log_group_name: test.log
                    log_stream_name: test.log
                    timezone: Local
            log_stream_name: my_log_stream_name
            force_flush_interval: 15

Minimal configuration on AWS EC2 instance

---
- hosts: all
    gather_facts: True
    become: true
    become_user: root
    become_method: sudo
    remote_user: ec2-user
    roles:
    - role: christiangda.amazon_cloudwatch_agent # Assuming you are on "EC2 instance" and the default agent configuration from role

Inventory file sample (inventory)

[all]
10.14.x.y
10.14.v.z

[amazon-1]
10.14.x.y

[amazon-2]
10.14.v.z

How to used it

ansible-playbook my-playbook.yml \
    --inventory inventory \
    --private-key [~/location of my key.pem] \
    --become \
    --become-user=ec2-user \
    --user ec2-user

Development / Contributing

This role is tested using Molecule and was developed using Python Virtual Environments

Prepare your environment

Python 3

mkdir ansible-roles
cd ansible-roles/

python3 -m venv venv
source venv/bin/activate
pip install pip --upgrade
pip install ansible
pip install molecule
pip install molecule[vagrant]
pip install python-vagrant
pip install selinux
pip install docker
pip install pytest
pip install pytest-mock
pip install pylint
pip install rope
pip install autopep8
pip install yamllint
pip install flake8

Clone the role repository and create symbolic link

git clone https://github.com/christiangda/ansible-role-amazon-cloudwatch-agent.git
ln -s ansible-role-amazon-cloudwatch-agent christiangda.amazon_cloudwatch_agent
cd christiangda.amazon_cloudwatch_agent

Execute the test

Using docker in local

molecule test [--scenario-name default]

Using vagrant in local

molecule create --scenario-name vagrant
molecule converge --scenario-name vagrant
molecule verify --scenario-name vagrant

molecule test --scenario-name vagrant

Additionally if you want to test it using VMs, I have a very nice ansible-playground project that use Vagrant and VirtualBox, try it!.

License

This module is released under the GNU General Public License Version 3:

http://www.gnu.org/licenses/gpl-3.0-standalone.html

Author Information

Christian González Di Antonio

thilinapiy / ansible-role-amazon-cloudwatch-agent