Netdata CloudWatch Plugin

Help
,
1

Hi Everyone,

is there a netdata agent plugin for collecting data from AWS CloudWatch? I have a couple of EFS and S3 instances I’d like to monitor using netdata running on an EC2 instance.

The AWS CloudWatch API Python SDK seemed like a promising place to start, but I wanted to ask before starting something from scratch.

1 Like
2

We’re actually currently looking to retrieve AWS RDS data from CloudWatch, so we’ll create a new CloudWatch collector soon. See CloudWatch Metric Streams – Send AWS Metrics to Partners and to Your Apps in Real Time | AWS News Blog on why the API isn’t a great solution for this.

We’d like from you some assistance with the exact charts, dimensions and labels you’d like to see. Existing examples are fine.

3

Interesting read - thanks for sharing. For EFS monitoring, there’s a good list of parameters here. The ones I’m primarily interested in are:

  • DataReadIOBytes
  • DataWriteIOBytes
  • TotalIOBytes
  • ClientConnections
  • StorageBytes
  • PercentIOLimit

Similar for S3 monitoring (read/write/total, connections & total size)

1 Like
4

AWS wants us to provide them with 3 customers who will make use of the Firehose adapter we will built.
@outlyer and anyone else interested in this, please send the following info to chris@netdata.cloud, so we can get started (this is a screenshot of the “Account” info in the AWS console, accessible via the top right menu).
Don’t redact the Account Id from the email you send me, so they can identify you.

image
image1758×675 54.9 KB

5

Sent!

BTW for RDS monitoring I currently use the netdata agent on an EC2 instance that uses the db I want to monitor, which works really well (so you may not need to go via CloudWatch). Since it polls performance data from the db directly, it does report stats for all users connected to the db (not just the EC2 instance doing the monitoring).

I’ve tried a similar setup using the netdata agent to monitor an NFS (AWS EFS) and s3 bucket (mounted using s3fuse) on that EC2 machine, but in that case you get only the performance data for that particular EC2 instance connected to the storage service, not that of all users connected to it. That’s what lead me to the idea of creating a netdata agent plugin for CloudWatch in the end.