Cloud Custodian Policy Health Checks

An easy way to diagnose if the Cloud Custodian policies are in good health!

Now that you have hundreds of Cloud Custodian policies deployed to your AWS cloud environment. It is very important to know that those policies are working as designed and operating without giving any errors. This is similar to any internal controls that you measure for design and operating effectiveness. The YAML policy is a part of your design where you have defined the resources, filters, mode, and actions. This design makes sure that the policy meets your requirement and does what it suppose to do. The operating effectiveness is where we want to make sure that the policy continues to work without giving any errors (over time).

In this story, we will discuss an alternate way of diagnosing and alerting on policies that are giving errors. For the purpose of the story, we will assume that the Cloud Custodian logs are ingested into one of the SIEM solutions. We have a separate story about the Cloud Custodian [GZ] output and how to ingest Cloud Custodian Logs into Sumo Logic (SIEM). Out of the 3 output files that Cloud Custodian produces, the “custodian-run.log” file is very important to identify the policy that is giving errors. This file contains the DEBUG message. It includes region, custodian version, filtered items, count, and errors. See the below screenshot for an example of…