Good news, PHP developers! If you use the PHPUnit library to manage and run your unit tests, you can now send test summary results to New Relic Insights and display useful information about test successes and failures.

PHPUnit dashboard

[click to enlarge]

Why this is useful

A dashboard of test summary data can allow you to quickly see a snapshot of overall test success as well as dig into failing suites to determine their cause. With a dashboard you can see at a glance that, for example, a particular test suite has suddenly started failing after that last commit. Clicking into that suite’s run ID lets you see that a particular test was the culprit because it failed to assert that a variable was true.

Tracking your test history also allows you to see patterns in the results. You can compare the number of failing tests this week to the number from last week. You can determine that a failing test only fails on a particular test machine, or perhaps it only fails 20% of the time. Perhaps you discover that one integration test normally completes quickly, except for 10% of the time when it takes forever.

New Relic is designed to make it easy to generate and meaningfully display your test events. Let’s walk through some examples and spark some ideas for your own test dashboard.

How to get started

This feature is disabled by default. To enable it, find or add newrelic.phpunit_events.enabled in your newrelic.ini file and set it to true. You’ll need to restart Apache or your alternate web dispatcher for the change to take effect.

Please note that exception messages are collected and sent with events. Additionally, if you use PHPUnit’s –disallow-test-output option, which flags tests that print output as “risky,” the test event will include the offending output.

When enabled, the PHP Agent detects PHPUnit commands and populates New Relic Insights with a set of attributes for the test suite and its individual test data. You can query the data with NRQL queries and set up a dashboard of information important to you.

Sample queries

The examples below illustrate the kinds of widgets you can create to answer questions and gain insights. For the NRQL query strings behind the widgets, check out our documentation page.

Percent success: What percent of suites or tests are passing?

suites passing screenshot

 

Test outcome: What is the breakdown of test outcomes?

test outcomes screenshot

 

Test failures: What percent of the time does each test pass?

test failure screenshot

 

Test suite failures: What percent of the time does each suite pass, and is that consistent over time?

passing frequency screenshot

 

Duration: How long does each test suite take to run, and is that consistent over time?

duration frequency screenshot

Linked facets

Because a PHPUnit test suite is linked to individual tests via its run ID, you can use FACET widgets to filter results for a specific test run. For example, if you create a widget with the most recent failing test suites and link it to the current dashboard, you can click on a suite run ID and the surrounding widgets will update with information for just that suite.

In the example below, you can see that by clicking on run 77a075a14a078e81, which ran the suite FlakyTest, we learn that one of two tests, testFoo, failed.

recent unsuccessful suites screenshot

[click to enlarge]

More information

To learn more about how to build and edit Insight dashboards, check out our awesome documentation.

Do you want to make a unit test dashboard and you’re bummed that you don’t use PHPUnit? You can send your own custom events with a language agent or with the Insights custom events API.

 

Erika Arnold works on the PHP Agent team at New Relic. A former biologist, she’s collected bugs all over the world. Now she just enjoys finding bugs in code. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!