The Inside Story on Developing a New Relic Plugin

Deploying an application often seems like a blind flight, but a quick New Relic integration can help shed light on an app deployment.

Out of the box, New Relic provides analytics for your servers and web or mobile applications. To include data from other components New Relic offers plugins. Through the New Relic Platform, developers have already created and shared nearly 100 plugins. All the plugins are free and most are open sourced.

plugin

To monitor most of our stack here at Joltem, we need one only plugin. MeetMe’s New Relic Plugin Agent collects data from our Redis, PostgreSQL, and Nginx servers. However, to monitor our git server we needed to develop a custom plugin. The process of how we did it makes a great case study to illustrate just how easy it is to create a New Relic plugin.

Git Server

To understand how we developed the plugin requires a small introduction to the Git server: Basically, a process is executed when a user connects. The process allows the client and server to communicate using a native git protocol. A ‘git-upload-pack’ process fetches Git data from the server, while a ‘git-receive-pack’ process handles pushes to the server. (For a more in-depth explanation, see our Building a Git Server write-up on Medium.)

Specifications

The next step is to outline the data we want to collect. For each process’ ‘git-upload-pack’ and ‘git-receive-pack,’ we want to collect the following metrics:

  •  **Bytes In**, bytes transferred to server from client.
  •  **Bytes Out**, bytes transferred to client from server.
  • **Bytes Total**, total bytes transferred.
  • **Bit Rate**, total bytes transferred divided by time from the start to end of process.

 Collection

New Relic’s API uses a sensible limit of two requests per minute. Aggregation therefore must occur on our servers. Each transfer event is initially recorded in our database. Every thirty seconds a task aggregates the transfer events. Using New Relic’s event records, it calculates a total, minimum, maximum, count, and sum of squares for the each metric during the duration.

Reporting

Once aggregates are calculated they need to be reported to New Relic. Reports are reported as JSON hashes. A report contains information about the reporting agent and the components being reported.

{
 "agent": {
 "version": "0.1.0",
 "host": "joltem.com"
},
 "components": [
 ...
 ]
}

Unlike MeetMe’s plugin agent that reports on multiple components (Redis, Nginx, etc.), our reports contain just one component: the Git server.

A component is represented by a hash that identifies the component and the plugin it uses. The `guid` attribute represents a Global Unique Identifier and should be unique for each plugin. A `name` separates multiple components reporting to the same plugin. Since we run only one Git server per host, we simply set `name` to the hostname. The time over which the aggregate data was gathered is specified by `duration` in seconds.

{
 "name": "joltem.com",
 "guid": "com.joltem.git",
 "duration": 30,
 "metrics": [
 ...
 ]
}

The `metrics` attribute contains a hash with metric names mapping to metric values. While there are several ways to pass the metric value, we chose to pass it as an array:

An array of five required integers or floating point numbers that represent, in order:

  1. Total value over the time period
  2. Count of the number of events this value represents over the time period; the average is calculated by dividing total by count
  3. Minimum value over the time period
  4. Maximum value over the time period
  5. Sum of squares for the samples over the time period

Here is an example of our `metrics` hash:

{
 "Component/Upload/Bytes In[bytes]": [180,5,5,90,5000],
 "Component/Upload/Bytes Out[bytes]": [180,5,5,90,5000],
 "Component/Upload/Bytes Total[bytes]": [180,5,5,90,5000],
 "Component/Upload/Bit Rate[bytes/second]": [180,5,5,90,5000],
 "Component/Receive/Bytes In[bytes]": [180,5,5,90,5000],
 "Component/Receive/Bytes Out[bytes]": [180,5,5,90,5000],
 "Component/Receive/Bytes Total[bytes]": [180,5,5,90,5000],
 "Component/Receive/Bit Rate[bytes/second]": [180,5,5,90,5000]
}

Once the report is generated, a POST request delivers it to New Relic. To associate with an account, the account license key is provided through the “X-License-Key” header.

POST http://platform-api.newrelic.com/platform/v1/metrics
X-License-Key: settings.NEW_RELIC_LICENSE_KEY,
Content-Type: ‘application/json’,
Accept: ‘application/json’

{
 "agent": {
  "version": "0.1.0",
  "host": "joltem.com"
},
 "components": [ {
  "name": "joltem.com",
  "guid": "com.joltem.git",
  "duration": 30,
  "metrics": [ {
   "Component/Upload/Bytes In[bytes]": [180,5,5,90,5000],
   "Component/Upload/Bytes Out[bytes]": [180,5,5,90,5000],
   "Component/Upload/Bytes Total[bytes]": [180,5,5,90,5000],
   "Component/Upload/Bit Rate[bytes/second]": [180,5,5,90,5000],
   "Component/Receive/Bytes In[bytes]": [180,5,5,90,5000],
   "Component/Receive/Bytes Out[bytes]": [180,5,5,90,5000],
   "Component/Receive/Bytes Total[bytes]": [180,5,5,90,5000],
   "Component/Receive/Bit Rate[bytes/second]": [180,5,5,90,5000]
   } ]
  } ]
}

We flush out the reported transfer records from the database on a successful `200` response. In case of a `5XX` response, signifying temporary failure, we attempt to aggregate the transfers into the next report. The `duration` is adjusted accordingly.

Dashboard

Once New Relic receives data from your plugin, it will appear in your New Relic dashboard. To prepare the plugin’s interface, press `Edit` to setup Summary Metrics and Dashboards for the plugin.

Image2

Summary metrics are shown at the bottom of the screen as you browse the plugins dashboard.

image3

You can edit each individual dashboard by creating and placing charts.

image4

Charts can display multiple metrics by using wildcard notation (i.e. `Component/Upload/Bytes *[bytes] `) when specifying a metric.

image5

When the dashboards are ready you can publish the dashboard to New Relic’s Plugin Central.

image6

That’s it, your plugin is ready. Enjoy your new insight.

Emil Davtyan is an entrepreneur who has started several ventures since 2005. He created and ran an image-hosting site that reached two million users per a month. Later, he developed an Android application with more than 200,000 installations. At his current venture, Joltem (http://joltem.com), he is working to open up the startup process. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!