How to setup AWS CloudWatch Agent for Custom Metrics?

How to setup AWS CloudWatch Agent for Custom Metrics?

August 16, 2018 / Nirav Shah

AWS provides lots of services and data. But it is also true that some metrics are not available in aws default metrics. That’s why we have to setup custom metrics like Memory, Disk and Swap uses etc. which are very necessary to monitor and maintain the servers.

We can push custom metrics by two methods:

1. MonScripts Setup
2. CloudWatch Agent setup

Monitoring By CloudWatch Agent:

We can setup custom metrics data pushing setup in ec2 instance by above methods. CloudWatch Agent Method is much reliable and effective with compare to Monscripts setup. In Cloudwatch Agent method, we don’t need to setup any cron job to send custom metrics data to cloudwatch. The CloudWatch Agent manage all these automatically as predefined. It works simultaneously with AWS cloudWatch service.

CloudWatch agent enables you to do Collect more system-level metrics from Amazon EC2 instances, including in-guest metrics, in addition to the metrics listed in Amazon EC2 Metrics and Dimensions. The additional metrics are listed in Metrics Collected by the CloudWatch Agent.

Here are some simple steps to setup Custom Metrics in Amazon linux Ec2 instance using CloudWatch Agent –

1. Create the IAM roles and users that you need for the CloudWatch agent. They enable CloudWatch to collect metrics from the server, and to integrate with AWS Systems Manager.
   Create Role and User with CloudWatchAgentServerPolicy and AmazonEC2RoleforSSM policy.
2. Download the agent package. First make directory cloudwatch agent and change directory to cloudwatch agent.

   $ mkdir cloudwatch-agent

   $ cd cloudwatch-agent/

   $ wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip

3. Unzip the package –

   $ unzip AmazonCloudWatchAgent.zip

4. Then run the install.sh –

   $ sudo ./install.sh

5. Whether you are installing the CloudWatch agent on an Amazon EC2 instance or an on-premises server, you must create the CloudWatch agent configuration file before starting the agent. The agent configuration file is a JSON file that specifies the metrics and logs that the agent is to collect. You can create it by using the wizard, or by creating it yourself from scratch. You could also use the wizard to initially create the configuration file, and then modify it manually.

   So now here we will use wizard to create cloudwatch agent configuration file. It will create file automatically as our answer.

   The agent configuration file wizard, amazon-cloudwatch-agent-config-wizard, asks a series of questions, including the following:

   • Are you installing the agent on an Amazon EC2 instance or an on-premises server?
   • Is the server running Linux or Windows Server?
   • Do you want the agent to also send log files to CloudWatch Logs? If so, do you have an existing CloudWatch Logs agent configuration file? If yes, the CloudWatch agent can use this file to determine the logs to collect from the server.
   • If you are going to collect metrics from the server, do you want to monitor one of the default sets of metrics, or customize the list of metrics that you collect?
   • Are you migrating from an existing SSM Agent?

   Run command-

   $ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

   To know CloudWatch Agent Predefined Metric Sets click the link below-

   https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html

6. Now install AWS Systems manager to start agent and run commands. Create directory /tmp/ssm and change directory to the /tmp/ssm

   $ mkdir /tmp/ssm

   $ cd /tmp/ssm

   $ sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm

7. Now Run these commands to start agent and push the metrics in cloudwatch custom metrics. Replace the SSM_PARAMETER_NAME with your ssm parameter name.

   $ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:SSM-PARAMETER_NAME -s

   $ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -s

Now Check your CloudWatch Service. In metrics section you would be seeing the data collected by CloudWatch Agent metrics. Explore these metrics and set dashboard and create necessary alarms as you want for seamless experience of AWS infrastructure.

If you are having any trouble in setting up a AWS infrastructure, you can contact us and we will design the AWS architecture as per your requirements and help you to set-up the whole AWS infrastructure.