DevOps teams often use infrastructure-as-code (IaC) to provision, deploy, and manage applications in the cloud. AWS CloudFormation is one tool that provides IaC functionality, and AWS customers can now use third-party resource providers in their application stacks configured with CloudFormation.
New Relic is excited to announce initial support for this with the ability to create NRQL alert conditions as resources directly in CloudFormation templates.
At New Relic, we believe that observability is fundamental to building great software, and part of any successful observability practice is noticing when things go wrong. New Relic Alerts allows you to flexibly create and configure alerting for your services and applications. Alert conditions describe when the behavior of a monitored service or application is considered a violation. For example, DevOps teams at New Relic use alert conditions to watch for increases in resource use that may require them to scale their infrastructure capacity.
In this post, we’ll execute an example CloudFormation template that creates an AWS Lambda function with a built-in New Relic alert condition resource type.
About AWS CloudFormation
In CloudFormation, you use templates written in JSON or YAML to express a high-level description of how your various AWS resources and the interactions between them form a “stack.” An execution engine uses this template to build the stack using the resources you specified. Since this infrastructure is expressed as code, CloudFormation can build this stack again and again, and it will be the same every time.
AWS services typically have well-documented APIs. CloudFormation simply calls these APIs to create, delete, and update resources (all of which are described in terms of a resource type, a name, and a set of properties). When you tell CloudFormation to create your stack, the execution engine makes API calls to the AWS service APIs and supplies your resource properties as parameters to those calls. The translation layer between the stack and the API calls is the “resource provider.”
Step 1: Install and register the resource provider
To use New Relic NRQL Alert creation, you must first register it as a resource provider with CloudFormation. After you registered a resource provider, it will appear in the CloudFormation registry for that account and region, and you can use it in your stack templates.
You register resource providers using the RegisterType action, or by using the
submit command of the CloudFormation CLI. To register a resource provider using the CloudFormation CLI, see Registering Resource Providers in the CloudFormation CLI User Guide.
To register New Relic NRQL Alerts using the CloudFormation API:
- Use the RegisterType action to register New Relic NRQL Alerts in your account:
aws cloudformation register-type \ --type-name "NewRelic::Alerts::NrqlAlert" --schema-handler-package "s3://nr-cloudformation-downloads/newrelic-alerts-nrqlalert.zip" \ --type RESOURCE \
RegisterTypeis an asynchronous action, and returns a registration token you can use to track the progress of your registration request.
- Optional: Use the registration token with the DescribeTypeRegistration action to track the progress of your registration request:
aws cloudformation describe-type-registration --registration-token token
When CloudFormation completes the registration request, it sets the progress status of the request to
Step 2: Execute the example CloudFormation stack
For alerts to be interesting, you need something to alert on. In this example, we’ll draw on New Relic Monitoring for AWS Lambda and use an AWS Lambda function, which you can manage and deploy with CloudFormation.
Our example CloudFormation template deploys a New Relic-instrumented Node.js Lambda function in an S3 bucket and creates an alert condition in New Relic that triggers when you invoke the function. (Follow our documentation for instructions on linking your AWS account to your New Relic account, enabling Lambda monitoring, and instrumenting your Lambda functions via our new no-code installation method.)
We’ve included inline documentation to explain what’s happening in the template. Feel free to customize the template to better fit your situation.
Resources: # This defines the Lambda function LambdaNode: Type: AWS::Lambda::Function Properties: FunctionName: LambdaNode Handler: index.handler Role: !GetAtt LambdaNodeRole.Arn Code: S3Bucket: nr-my-lambda-functions S3Key: LambdaNode Environment: Variables: NEW_RELIC_NO_CONFIG_FILE: true NEW_RELIC_APP_NAME: lambda_team NEW_RELIC_ACCOUNT_ID: <your account ID> NEW_RELIC_PRIMARY_APPLICATION_ID: <your account ID> NEW_RELIC_TRUSTED_ACCOUNT_KEY: <your account ID> NEW_RELIC_SERVERLESS_MODE_ENABLED: true NEW_RELIC_DISTRIBUTED_TRACING_ENABLED: true Runtime: nodejs8.10 Timeout: 300 # This is the execution rule for the function LambdaNodeRole: Type: "AWS::IAM::Role" Properties: RoleName: LambdaNodeRole, AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: [ "lambda.amazonaws.com" ] Action: [ "sts:AssumeRole" ] Path: / Policies: - PolicyName: AWSLambdaBasicExecutionRole PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"] Resource: [ "*", "arn:aws:lambda:us-west-2:466768951184:function:newrelic-log-ingestion"] - PolicyName: AmazonS3FullAccess, PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: s3:* Resource: [ "arn:aws:s3:::nr-my-lambda-functions", "arn:aws:s3:::nr-my-lambda-functions/*"] # This is the log group that we'll log into. The New Relic agent will produce events into the CloudWatch log stream LambdaNodeLogGroup: Type: AWS::Logs::LogGroup Properties: LogGroupName: /aws/lambda/LambdaNode # This pipes the CloudWatch logs from our function into the newrelic-log-ingestion Lambda, which sends them to New Relic SubscriptionFilter: Type: AWS::Logs::SubscriptionFilter Properties: LogGroupName: /aws/lambda/LambdaNode FilterPattern: "" DestinationArn: "arn:aws:lambda:us-west-2:466768951184:function:newrelic-log-ingestion" DependsOn: LambdaNodeLogGroup # Here's our custom resource type, which creates an alert in New Relic that triggers when the function is invoked LamdaNodeAlert: Type: NewRelic::Alerts::NrqlAlert Properties: #TODO: Add your values here ApiKey: <your api key> PolicyId: <your policy ID> NrqlCondition: Name: Alert Condition Test RunbookUrl: http://example.com/runbook Enabled: true ExpectedGroups: 0 IgnoreOverlap: true ValueFunction: single_value Terms: - Duration: "1" Operator: "equal" Priority: "critical" Threshold: "1" TimeFunction: "all" Nrql: Query: "SELECT count(*) FROM AwsLambdaInvocation WHERE provider.functionName = 'LambdaNode'" SinceValue: "1"
When you’re ready to execute the stack, run:
aws cloudformation create-stack --region us-west-2 \ --template-body "file://stack.yaml" \ --stack-name NewRelicAlert
CloudFormation will create an alert condition in New Relic, which will alert you when your function has been invoked.
With infrastructure-as-a-service forecasted to be the fastest-growing cloud services segment in 2020, services like AWS CloudFormation have rapidly gained traction within organizations of all sizes. The ability to configure NRQL custom alerts via CloudFormation templates enables DevOps teams to set alert conditions on critical performance issues and resolve those issues faster.
Learn more about setting up NRQL Alerts, including best practices, here. To get started monitoring, visualizing, troubleshooting, and alerting on your AWS Lambda functions, sign up for a free trial of New Relic Serverless.