The Art of Publishing New Relic Data to Your Application Status Page

This is a guest blog post from Alex Bloom, co-founder at Uptime.ly, a provider of application downtime server status pages, and a New Relic partner.

Information is power.  In the case of communicating application downtime to your customers and employees, that power is a delicate balance between too much information and not enough.  Not communicating properly or timely can leave your customers frustrated and wanting more.  Putting too much information in front of that same customer may leave them scratching their head trying to either interpret the data, or worse yet, making improper assumptions about your application’s quality and uptime.

Mitigating this balance requires a coordinated communication plan and process within your organization.  You need to ensure that when your application becomes unavailable either unexpectedly or from scheduled maintenance, that the right information is delivered to the right people at the right time.  For this reason, we’ve always considered the act of providing your end-users with direct access to real-time and historical metrics collected by your application monitoring services minimally as information overload, but at worst, dangerous.

Uptime.ly is a new breed of applications that focus solely on managing how your end users consume application status information.  At its heart, Uptime.ly is an application status page that provides a one-stop-shop for users to peer in to your application’s current status.  While providing a number of ways to proactively get this information into your end-user hands, one of the primary services we focus on is integrating with application monitoring services, such as New Relic, to help ensure a seamless flow of information.

This blog will walk you through setting up Uptime.ly’s integration with New Relic, while demonstrating how we utilized their API and Webhooks to provide a powerful integration that combines the full life-cycle of application monitoring with customer communications. You will need an Uptime.ly account and a New Relic account. If you do not have a New Relic account, you can sign up for a free upgraded account through our partnership at newrelic.com/uptimely.

1.  Setting Up Your API Key

Before you get started, find your New Relic API key by logging in to your account and selecting Account Settings > Integrations > Data Settings.

1_settingapikey

Copy and paste your API key directly in to your Uptime.ly account by selecting Application > Integrations from the Uptime.ly administration dashboard.

nr_api_key

2.  Create a New Uptime.ly Monitor

This is where the real fun begins. Selecting Application > Monitors > New Monitor from within Uptime.ly is where we first start communicating with New Relic via their API.

2_createuptimelymonitor

Our task here is to retrieve a list of all alerts that you have set up within New Relic and display them within the “Selected Alert” drop down list (as pictured above).  As Uptime.ly is written using Microsoft.net, we chose to use the Json.Newtonsoft libraries for parsing the Json results.

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Serialization;

// Helper method to retrieve list of alerts from New Relic API.
public JObject GetAlerts()
{
            var client = new RestClient("https://api.newrelic.com/v2/alert_policies.json");
            var request = new RestRequest(Method.GET);                       
            request.AddHeader("X-Api-Key", _apiKey);
            RestResponse response = (RestResponse)client.Execute(request);
            return JObject.Parse(response.Content);           
}

List<string> alertNames = null;
JObject alerts =GetAlerts();
if (alerts != null)
{
            JArray ja = (JArray)alerts["alert_policies"];
            alertNames = (from c in ja.Children()["name"].Values<string>() select c).ToList();
}
3.  Create your New Relic Webhook

When you create your Uptime.ly monitor, you are provided with the URL of the webhook that needs to be configued within New Relic.  This URL represents the location that New Relic will post new information whenever there is a change in one of your New Relic alert statuses.  In the above example, the URL provided was http://jasentest.uptime.ly/webhook/newrelic.aspx

Log into your New Relic account, and create a new communication channel for your alert polices.  Navigate to Tools > Alert Policies > Channels and Groups, and create a new Webhook channel pointing to the previously mentioned URL.

3_createwebhook

*After you have created the Webhook channel, make sure that you assign it to the same alert that you configured Uptime.ly to monitor.

Receiving Alerts from New Relic

Once your integration set up is complete, Uptime.ly will start receiving alerts directly from your New Relic account.  New Relic Webhook alerts come in the form of HTTP POST requests and contain another chunk of Json data that needs to be parsed:

// Retrieve the policy name of the alert retrieved from New Relic Webhook
// using Newtonsoft.Json
if(Request.Form["alert"]!=null)
{
            JObject d = JObject.Parse(Request.Form["alert"]);
            string ap = d["alert_policy_name"].ToString().ToLower();
}

The data received from a New Relic Webhook call merely point out the specific alert whose status has changed.  It is still the responsibility of the consumer to go back out to your New Relic account and retrieve more detailed information via their API.  In our case, we need to retrieve the list of statuses from your account and match them with the specific alert that has been configured.  To do this, we use code similar to this:

// Determine the status of an application based on a New Relic alert.
JObject alerts = nrc.GetAlerts();
JArray ja = (JArray)alerts["alert_policies"];
JToken alert = null;

// Iterate the alerts and match it to the one we are looking for.
foreach(JObject j in alerts["alert_policies"])
{
            if(j["name"].ToString().ToLower()=="alertNameToFind")
            {
                        alert = j;
                        break;
            }
}
if(alert!=null)
{
            // build list of application ids associated with this alert.
            // this wasn't built for easy application consumption so we need to
            // parse it.
            String apps = alert["links"]["applications"].ToString();
            List<int> appIds = new List<int>();
            if(!string.IsNullOrEmpty(apps))
            {                      
                        string[] s = apps.Replace("[", "").Replace("]", "").Replace("\r", "")
                                                                        .Replace("\n", "").Replace(" ", "").Split(',');
                        foreach(string si in s)
                        {
                                    int sis = 0;
                                    if(int.TryParse(si, out sis))
                                                appIds.Add(sis);
                        }                                   
            }
            if(appIds.Count>0)
            {
                        // only continue if we have a list of valid app id's.
                        // now get all the app statuses from new relic.  for our
                        // situation we want to return the worst status from all
                        // apps.                      
                        JObject aps = nrc.GetApplications();
                        string worstStatus = "green";
                        foreach(JToken ap in aps["applications"])
                        {
                                    int id = int.Parse(ap["id"].ToString());
                                    string health_status = ap["health_status"].ToString();
                                    if ((health_status != "green") && (worstStatus == "green")
 && (appIds.Contains(id)))
                                                worstStatus = health_status;                       
                        }

                        return worstStatus;
            }          
}

With this information in hand, Uptime.ly creates a new incident on your application status page and all your application subscribers are notified.  You may also set up Uptime.ly to wait a minimum amount of time for a subsequent “all systems normal” Webhook communication to be received from New Relic before deciding to publish the incident.  Additionally, you can set up incoming alerts from New Relic to be approved manually by one of your Uptime.ly administrators.  If this is the case, administrators can simply respond to an e-mail alert they receive, or login to the Uptime.ly dashboard to publish the new incident.

Conclusion

As organizations rely more and more on SaaS providers to manage their DevOps and customer communication, it’s imperative that these systems learn to work together.  Facilitating these integrations are open protocols and data sharing standards like HTTP and Json.  New Relic and Uptime.ly customers are enjoying the benefit of the integrations these standards make possible – get started here!

This post was written by a guest contributor. Please see their biographical details at the top of the post above. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!