Network Performance Monitoring on EC2 Instances

This article will walk you through how to set up enhanced network performance monitoring on EC2 instances.

The Elastic Network Adapter (ENA) driver publishes network performance metrics from the instances where they are enabled. You can use these metrics to troubleshoot instance performance issues, choose the right instance size for a workload, plan scaling activities proactively, and benchmark applications to determine whether they maximize the performance available on an instance.

We will basically setup the Cloudwatch Agent on the instances to send these 5 metrics to Cloudwatch Dashboard.

bw_in_allowance_exceeded: The number of packets queued or dropped because the inbound aggregate bandwidth exceeded the maximum for the instance.

bw_out_allowance_exceeded: The number of packets queued or dropped because the outbound aggregate bandwidth exceeded the maximum for the instance.

conntrack_allowance_exceeded: The number of packets dropped because connection tracking exceeded the maximum for the instance and new connections could not be established. This can result in packet loss for traffic to or from the instance.

linklocal_allowance_exceeded: The number of packets dropped because the PPS of the traffic to local proxy services exceeded the maximum for the network interface. This impacts traffic to the DNS service, the Instance Metadata Service, and the Amazon Time Sync Service.

pps_allowance_exceeded: The number of packets queued or dropped because the bidirectional PPS exceeded the maximum for the instance.

More information can be found here.

1. Setup an EC2 IAM role with a policy that allows EC2 to send metrics to CloudWatch.

Follow the link here

If you already have an existing IAM role attached to the EC2 instance then attach the CloudWatchAgentServerPolicy to it

2. Install the CWAgent on the EC2 instance

sudo yum install amazon-cloudwatch-agent

More instructions here

3. Configure the CWAgent

Create the CWAgent Config File

/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

Refer this link for more detailed configurations for metics and logs.

Restart the CWAgent

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

Test the metrics using Iperf

1. Install Iperf on the instances

Benchmark network throughput between EC2 Linux instances in the same VPC

yum -y install epel-release && yum -y install iperf

2. Enable iperf for testing TCP

Server config

sudo iperf -s [-p 5001]

Client config

iperf -c <server-ip> --parallel 40 -i 1 -t 2

3. Enable iperf for testing UDP

Server Config

sudo iperf -s -u [-p 5001]

Client Config

iperf -c <server-ip> -u -b 5g

View the metrics using command line

[ec2-user ~]$ ethtool -S eth0 
bw_in_allowance_exceeded: 0 
bw_out_allowance_exceeded: 0 
pps_allowance_exceeded: 0 
conntrack_allowance_exceeded: 0 
linklocal_allowance_exceeded: 0

View the metrics in CloudWatch Dashboard

CloudWatch Metrics Dashboard