Macro vs. Micro Metrics: Why You Should Be Collecting Each and How

When asked about helpful tips or best practices regarding mobile application development, one of the common responses I give is to “measure early, measure often.” Mapping real measurements to your overall objectives is a good way to ensure that all the work you put into your app delivers the highest return. But that then leads us to the discussion of how to measure (i.e. what type of metrics should you be collecting?).

So in this post, I’ll walk you through the two major categories of metrics that I believe all software developers should be collecting: macro and micro metrics.

Macro metrics: Ideal for understanding user behavior

Macro metrics help you see the bigger picture. They help you understand group behavior relating to a specific area of your application (for example, the percentage of users in a given month that used feature x). Macro metrics don’t show you exactly what they did with the feature, but they show you how popular the feature is based on usage and help you understand navigation patterns within a given feature or between different sections of the application.

Forest image

When adding or changing features of an application, being able to make informed decisions based on real usage patterns makes a huge difference. What you want to avoid is making decisions solely based on a few reviewers’ comments or only based on support calls. When you’re doing that you have only a portion of the story. Having metrics that confirm usage patterns instead allows you to make changes based on the entire user base.

The first step toward understanding user behavior is making sure you collect view count for all the screens in your application. Using an analytics SDK (such as Google Analytics, Mixpanel or Localytics), you should create an event to track when each screen is presented to a user. After that, you’ll want to add events that represent feature usage.

(Example: Recording launch delays in an analytics app)



//

//  AppDelegate.m

//

 

#import "AppDelegate.h"

#import <NewRelicAgent/NewRelic.h>

 

#import <GoogleAnalytics-iOS-SDK/GAI.h>

#import <GoogleAnalytics-iOS-SDK/GAIDictionaryBuilder.h>

#import <MCDateExtensions/NSDate+MCExtensions.h>

 

#define LAST_LAUNCH_KEY @"lastLaunch"

#define LAST_ACTIVE_KEY @"lastActive"

 

@interfaceAppDelegate ()

- (NSDate *)lastLaunchDate;

- (NSDate *)lastActiveDate;

- (NSDate *)dateForKey:(NSString *)key;

@end

 

 

@implementation AppDelegate

 

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {

// Initialize the trackers

[[GAIsharedInstance] trackerWithTrackingId:@"<tracking id>"];

[NewRelicAgentstartWithApplicationToken:@"<app token>"];

 

 

//--------------------------------------------------------------------------

// Handle Last Launch statistics

//--------------------------------------------------------------------------

 

 

// Record time since last launch

NSNumber *daysSinceLastLaunch = [NSNumbernumberWithInteger:[NSDatedaysBetweenDate:[NSDatedate] andDate:[selflastLaunchDate]]];

[[[GAIsharedInstance] defaultTracker] send:[[GAIDictionaryBuildercreateEventWithCategory:@"life-cycle"

action:@"last-launch-delay"

label:@"Number of days since last launch"

value:daysSinceLastLaunch] build]];

 

// Record the new launch time

[[NSUserDefaultsstandardUserDefaults] setObject:[NSDatedate] forKey:@"lastLaunch"];

 

returnYES;

}

 

- (void)applicationWillEnterForeground:(UIApplication *)application {

// Called as part of the transition from the background to the inactive state; here you can undo many of the changes made on entering the background.

}

 

 

- (void)applicationDidBecomeActive:(UIApplication *)application {

NSNumber *hoursSinceLastActive = [NSNumbernumberWithInteger:[NSDatedaysBetweenDate:[NSDatedate] andDate:[selflastActiveDate]]];

[[[GAIsharedInstance] defaultTracker] send:[[GAIDictionaryBuildercreateEventWithCategory:@"life-cycle"

action:@"last-active-delay"

label:@"Number of houra since last active"

value:hoursSinceLastActive] build]];

 

// Record the new active time

[[NSUserDefaultsstandardUserDefaults] setObject:[NSDatedate] forKey:LAST_ACTIVE_KEY];

}

 

- (void)applicationWillResignActive:(UIApplication *)application {

 

}

 

- (void)applicationDidEnterBackground:(UIApplication *)application {

 

}

 

- (void)applicationWillTerminate:(UIApplication *)application {

}

 

//------------------------------------------------------------------------------

// Private

//------------------------------------------------------------------------------

- (NSDate *)dateForKey:(NSString *)key {

// Get the last launch date

NSDate* date = [[NSUserDefaultsstandardUserDefaults] objectForKey:key];

if(date == nil) {

date = [NSDatedate];

}

return date;

}

 

- (NSDate *)lastLaunchDate {

return [selfdateForKey:LAST_LAUNCH_KEY];

}

 

- (NSDate*)lastActiveDate {

return [selfdateForKey:LAST_ACTIVE_KEY];

}

 

 

@end


“Usage count” is different from “views,” as it should count the number of times a user completed a goal within that feature, and consequently, your application. “Views” will allow you to compare usage between screens, whereas a feature “usage count” will allow you to understand more closely what percentage of your users are interacting with a specific feature and correlate it to their exposure to it. One thing you want to look at is the number of times a feature is used compared to the number of views of a key screen. For example, in a social application, you could look at the number of times a user saw an activity feed versus the number of times she liked something in the feed.

Once you have both “views” and “usage count” set up, you then want to gather some goal and timing-related information (for example, the time it took for a user to attain certain goals within the application). This helps us understand if assumptions we’re making about the use of the application are correct, and if users are actually getting to the part of the application you want them to reach.

My previous post on mobile measurement walks you through the process of setting up metrics that tie your overall mission to specific objectives, so check that out for more details. But the idea here is to find out how long it took for the user, in real time, to get from first launch to first use of the application and how long they spent in the session.

Micro metrics: Perfect for tracking performance

shutterstock_127946846Unlike macro metrics, which look at a large portion of your application usage, micro metrics look at usage relating to a small portion of your application (for example, the average rendering time of a specific screen). Micro metrics are extremely helpful for the operations side of the house, since they help you understand if there are performance problems in the application. And this is really where mobile APM (or m-APM) shines.

If you’re using an m-APM solution like New Relic Mobile, you immediately start collecting micro metrics without having to instrument your application. The dashboard will automatically show you the performance of your API calls and screen render times, giving you a good idea of the wait times of your users right off the bat. If you’re missing some metrics, you can create a custom event and measure pretty much anything you want.

A good m-APM will allow you to track interactions on top of already tracking individual actions (API requests, page render time). Using interactions allows you to track a group of actions (simultaneous or sequential) and report on their performance. Let’s say that you had a running app, and storing all the recorded run data requires three API calls (one to create the run, followed by an upload locations sample, and a simultaneous API call to upload an image of the map).

From a performance standpoint you want to see how long the three calls took, and especially, the two simultaneous calls. For the user, the upload finished when all three calls finished. Using interaction metrics is great for tracking that experience since they have a start and stop concept. Having individual call and grouped call data allows you to see the real impact of performance degradation (or optimization) on the overall user experience while keeping the data needed to perform a root cause analysis.

Thus, micro metrics help you get things right the first time around. If you do end up making changes to your app, you’re doing so with a full understanding of whether or not they are likely to make things better or worse for your end users. As many mobile application developers and product managers know, one of the biggest problems with user retention is that the attention span of a user on a mobile device is so short. People get emails, they get all types of app notifications—there are all types of distractions that can easily take the focus away from your app. But micro metrics make sure that everything gets delivered seamlessly, so that there’s no downtime in which their attention can wander.

Metrics overall: Critical in driving business success

Both macro and micro metrics serve their own unique purpose in your overall app development lifecycle. But the beauty of collecting these metrics together is that they’ll help you avoid making a shot in the dark. They give you a much more reliable and comprehensive way to know how and where to improve your app (as opposed to simply relying on app store ratings and reviews).

When you do decide to make a major change in your app, as Foursquare did earlier this year, you’ll be in a much better position to make that pivot without upsetting your core user base and features. And if a problem does happen to occur, you’ll be able to show your users that you’re in control of the situation and have a fix in place, which is something they’ll appreciate. In short, collecting metrics helps you make more targeted decisions that help steer the business—and its reputation—in the right direction.

This is a guest post from Pierre-Luc Simard, CTO of Mirego. Mirego designs and develops innovative mobile strategies for iPhone, iPad, Android, and other mobile devices. Forest image and leaves image courtesy Shutterstock.com.

plsimard@mirego.com'

As Mirego’s CTO Pierre-Luc spends most of his time working on projects revolving around Mobile. He’s all about software running on small devices. He spent a big portion of the past 16 years working on embedded and mobile platforms in different environments. When he’s not working, Pierre-Luc is probably out on a hike somewhere or out on a run. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!