New Relic Insights On Big Data’s Chicken and Egg Question


Big data projects have long faced a critical chicken and egg question: Is it better to capture as much data you can right from the beginning and worry about what to do with it later? Or does it make more sense for companies to figure out the specific questions they need to answer and then go out and collect only the data they need to do that?

There’s no simple, one-size-fits all answer. A strategy that calls for spending a lot of money gathering data without a clear ROI may not fly in today’s cash-strapped IT organizations. But limiting your big data approach only to those things you know about in advance risks missing the most important insights that might not be clear until you start to dig into the data.

Insights already collects the data

New Relic Insights, however, solves the problem for New Relic users. If you’ve got our agents installed, we’re already collecting data from your live, production software right now. So you can start running big data queries straight out of the box.

The default set of PageView and Transaction events already comes with a number of built-in attributes to query with. If you want to ask more in-depth questions, simply add your own custom attributes by copy and pasting a snippet of code into your app.

As a real-time analytics platform, Insights makes it easy to instantly access that data to better understand the implications of how customers use your site or interact with your organization. With Insights, you can quickly ask questions of the data you already have – helping to make better decisions and grow your business.

It’s easy to add even more data

There’s a lot of data already available. Even before Insights went into public beta last month, New Relic had collected and stored more than 1 trillion events from our customers’ software.

But best of all, mining the existing data generated by your software is only the beginning. Custom events let you incorporate additional business data to expand the scope and usefulness of the analytics. For example, you can collect data from third-party web services or infrastructure components like servers and load balancers. There’s no limit to how deep into your business you can see with New Relic Insights.

*Image courtesy of Ruben Alexander'

Fredric Paul (aka The Freditor) is Editor in Chief for New Relic. He's an award-winning writer, editor, and content strategist who has held senior editorial positions at ReadWrite,, InformationWeek, CNET, Electronic Entertainment, PC World, and PC|Computing. His writing has appeared in MIT Technology Review, Omni, Conde Nast Traveler, and Newsweek, among other places. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!