Summary
As developers, we’re taught the importance of testing right from the beginning. Testing helps us write better code in less time and makes us feel more comfortable with the eventual outcome. I agree with all of these benefits. But sadly, I don’t always do it. And my guess is that you don’t always test either.

What’s more, I haven’t done a lot of JavaScript testing. Like a number of developers, I’m guilty of leaving JavaScript as an afterthought, whereas my PHP, Bash and Python code are first-class citizens.

However, I’ve started building more complex, object-driven code. As the complexity grows, it’s become rather obvious that it needs to be covered by proper unit tests. The question was how. So I started looking around for the best available JavaScript unit testing libraries and came across the following:

* QUnit
* Mocha
* Jasmine
* Sinon.JS

They all seem to offer solid feature sets for testing JavaScript. So to improve my JavaScript code quality and help you with yours, I’ll walk through each of them in a series of articles on the New Relic blog.

We’ll cover how the testing units work and what they can do, along with what they’re like to use. You’ll get an informed and honest opinion before you try them out with your own code. Sound good? Let’s start with QUnit.

Installing QUnit
QUnit is a self-contained library, requiring only one JavaScript file (qunit.js) and one CSS file (qunit.css). You can either download them from the QUnit site or clone them from the QUnit GitHub repository, as follows:

git clone git://github.com/jquery/qunit.git

After you get a copy of the files, add them into your project web root and include them directly or with the framework or application that you’re already using (such as Zend Framework or Oxid eSales). Then you’re ready to start testing.

The JavaScript Code
This isn’t a complex example, but I believe it’s enough to write some meaningful tests. The following code example creates a jQuery object that contains the properties and functions required to perform a basic mortgage calculation. Let’s walk through it slowly to give you an overview of how it works.

    $(document).ready(function() {

		var rateCalculator = {

The properties of the object store the following information (pre-initialized with some meaningful defaults):

* The amount of the loan
* The regular repayment amount
* The repayment frequency (monthly, fortnightly, weekly)
* The term of the loan
* The current interest rate of the loan

The defaults:

			loanAmount: 150000,
			repaymentAmount: 700,
			repaymentFrequency: "monthly",
			loanTerm: 35,
			interestRate: 9.15,

We then have a simple function that allows us to override the existing defaults with the information specified in the form, which we’ll see later.

			setLoanProperties: function(loanAmount, repaymentFrequency, loanTerm, interestRate) {
				this.loanAmount = new Number(loanAmount);
				this.repaymentFrequency = repaymentFrequency;
				this.loanTerm = new Number(loanTerm);
				this.interestRate = interestRate / 100;
				console.log(this);
			},

The calculateDifference method uses a mortgage calculation formula that I found on Wolfram. It calculates the recurring payment amount and the total repayable amount.

It initializes a series of local variables as Number objects and uses the pow function from the Math library to complete the calculation. I’ve split the calculation across a number of variables, more for readability and maintainability than anything else.

			calculateDifference: function() {
				switch (this.repaymentFrequency) {
					case ("weekly"):
						var period = 52;
						break;
					case ("fortnightly"):
						var period = 26;
						break;
					case ("monthly"):
					default:
						var period = 12;
						break;
				}
				var rateByPeriod = new Number(this.interestRate / period);
				var numerator = new Number(Math.pow((1 + rateByPeriod), (this.loanTerm * period)));
				var denominator = new Number(numerator.toFixed(2) - 1);
				var regularRepayment = new Number(this.loanAmount * rateByPeriod * (numerator / denominator));
				var totalAmount = new Number(regularRepayment * (period * this.loanTerm));
				
				$("#repaymentAmount").html(regularRepayment.toFixed(2));
				$("#totalToRepay").html(totalAmount.toFixed(2));
			},
		};

When the calculation is complete, it displays the two calculated values in the form so the user can see them. The values the user enters in the form are used to initialize the object and then to calculate the amounts. There’s no validation in the form, just the key functionality required.

		$("#submit").click(function() {
			rateCalculator.setLoanProperties($("#originalAmount").val(), $("#repaymentFrequency").val(), $("#loanTerm").val(), $("#interestRate").val());
			rateCalculator.calculateDifference();
		});
	});

The HTML itself is rather trivial, so I won’t spend time here going through it. It’s available with the rest of the code.

QUnit Functions
Although simple in scope, the QUnit Framework is rather effective. Unlike other unit testing frameworks such as PHPUnit or SimpleTest, it doesn’t have a lot of methods and assertions.

That’s not to say that isn’t fully capable. This is a list of the key tests available:

While I haven’t used all the tests above, they show that QUnit is a solid package for testing. What’s more, we can use them in combination with the power of jQuery and JavaScript to do what we need to do. Now let’s walk through the unit tests to see how they work.

The Basic Unit Test
Tests can be grouped together into modules. Any call to test – which is preceded by a call to module – is considered part of a group until the next call to module or the end of the test suite. I’ve done that below and included the classic setup and teardown functions. Actually, I’ve done nothing in either of them, but you can use these functions as needed to set up your DOM or do any other work.

    module( "module", {
		setup: function() {
			//ok( true, "one extra assert per test" );
		}, teardown: function() {
			//ok( true, "and one extra assert after each test" );
		}
	});

Another good thing about using QUnit is that you can set up tests to run as an atomic unit. I’ve broken the testing for my code into two parts:

1. Check the availability of the required form elements.
2. Ensure that the calculation operates as expected with a correct set of inputs.

The first test checks to make sure that all the required form elements are in place. I used simple CSS Selectors to access the elements in question, then checked that the length property is not equal to zero – which is a simple way to ensure it’s present.

The second element is the message for when the test passes or fails, so you know which one worked or didn’t. In the call to test, I provided a meaningful description for it (which is output when the test runs), the number of tests to expect and an anonymous function containing the tests.

		// ensure all the required form elements are in place
		test( "All required form elements exist", 7, function() {
			ok($("#originalAmount").length != 0, "original amount element exists");
			ok($("#repaymentFrequency").length != 0, "repayment frequency element exists");
			ok($("#loanTerm").length != 0, "loan term element exists");
			ok($("#interestRate").length != 0, "interest rate element exists");
			ok($("#submit").length != 0, "submit button element exists");
			ok($("#repaymentAmount").length != 0, "total repayment amount output element exists");
			ok($("#totalToRepay").length != 0, "total amount to repay output button element exists");
		});

As I did in the first test, I’ve provided a description for the second one: the number of tests that will run and an anonymous function containing the tests. In this one, I’ve provided a bit of setup, which I could have provided in the setup method. I initialized variables to predefined values and then set the various form elements accordingly.

After that, I leveraged QUnit’s ability to manually build an event object that can trigger the click event on the Submit button as if the user had done it. Assuming the events fire and the values are generated correctly, I’ve run two tests with the QUnit equal method to check that the calculated values are what I expected.

		test( "test calculation output", 2, function() {
			var dummyLoanAmount = 270000;
			var dummyYearsRemaining = 30;
			var dummyInterestRate = 8.00;
			
			$( "#originalAmount" ).val(dummyLoanAmount);
			$( "#loanTerm" ).val(dummyYearsRemaining);
			$( "#interestRate" ).val(dummyInterestRate);

			var event,
				$submitButton = $( "#submit" );

			// trigger event
			event = $.Event( "click" );
			event.keyCode = 9;
			$submitButton.trigger( event );

			var monthlyRep = $( "#repaymentAmount" ).val();
			var totalRep = $( "#totalToRepay" ).val();
			equal(monthlyRep, 457.04, "monthly repayment amount correctly totals" );
			equal(totalRep, 712985.68, "total repayment amount correctly totals" );
		});

The Test Output
Now that we’ve set up the tests and run them, how do they look? In the screenshot below you can see that the tests ran and the first test suite passed, but there’s a problem in the second one.

I expected the DOM to get updated after the click event was triggered, just like when a user clicks the Submit button. Then I could compare the calculated values against the expected amounts.

But, according to QUnit, the value of the two div elements is empty. So I’d like to throw this one out to you. Have you had a similar experience? If so, what solution did you come up with? Is it something small that I’ve missed? Let me know in the comments.

Enhancing the Tests
We’ve covered the core of the library. But let’s take it further. QUnit has a number of functions that allow us to register callbacks around various events in the unit testing lifecycle. They are:

To create a simple example, I’ve implemented all of them in the code below. I’ve rerun the code with them in place so you can see the output.

    QUnit.begin(function( details ) {
		console.log( "Test Suit Starting." );
	});

	QUnit.done(function( details ) {
		console.log( "Test Suit Ending. Results: Total: ", details.total, " Failed: ", details.failed, " Passed: ", details.passed, " Runtime: ", details.runtime );
	});

	QUnit.log(function( details ) {
		console.log( "Assertion complete. Details: ", details.result, details.message );
	});

	QUnit.moduleStart(function( details ) {
		console.log( "Starting module: ", details.module );
	});

	QUnit.moduleDone(function( details ) {
		console.log( "Finished Running Module: ", details.name, "Failed/total: ", details.failed, details.total );
	});

	QUnit.testStart(function( details ) {
		console.log( "Now Running Test: ", details.module, details.name );
	});

	QUnit.testDone(function( details ) {
		console.log( "Finished Running Test: ", details.module, details.name, "Failed/total: ", details.failed, details.total, details.duration );
	});

By displaying the console window, you can see all the events were intercepted and view their respective details. Now granted, this isn’t always necessary. But it can be handy to have when debugging the tests — especially if you want to know as much as possible about what’s going on in your test suite. Full credit to the QUnit manual for a number of the test examples above.

Conclusion
Whew! This was a rapid introduction to QUnit, one of the simplest and most effective unit testing libraries for JavaScript. What did you think? I realize that I haven’t covered everything available, but it’s more than enough to get started.

Personally, I was surprised by its functionality. I had expected it to be less capable that what I found it to be. One thing to note, I’m not sure how QUnit will fare if you run it from the command line or include it in an automated testing workflow.

There’s a post from 2010 about QUnit and the command line, which indicates that it’s rather difficult to do. The post mentions that the library contains a lot of browser-related assumptions. I’ve wondered whether using PhantomJS could get around any of these aforementioned issues, but I haven’t had time to test it.

A few more questions before I wrap up: Have you used QUnit in an automated context? Do you use QUnit as a normal part of your development workflow? We’d like to hear about your experience with QUnit in the comments below.

Next Time
I hope this article gave you a good idea of what’s possible with unit testing in JavaScript. In my next article, I’ll show you another unit testing library. See you there.

Further Reading

* Mortgage Calculation from Wolfram
* JQuery Unit Testing on Github
* QUnit Javascript Unit Testing Framework
* Unit Testing with QUnit

The post Simple JavaScript Testing with QUnit appeared first on New Relic blog.

The new metrics are located under the JVM tab and include the following:

* Loaded and unloaded class count for the JVM
* Active thread count for the JVM
* Active and idle thread count for each thread pool
* The ratio of active to maximum thread count for each thread pool
* Active, expired and rejected HTTP session counts per application
* Active, finished and created transaction counts per application server

Now let’s take a closer look at each of them:

Loaded & Unloaded Class Count

Location: Under the Memory tab in the bottom-right corner of the screen.

Supported Application Servers: All application servers that have JMX enabled.

Use Cases: The loaded and unloaded class count can be used for a variety of purposes. For example, if the loaded class count is constantly going up, then the app server or a class loader may have a bug. Or if you perform upgrades without bringing down the JVM, you can use it to verify that classes were unloaded and then reloaded.

Active Thread Count

Location: Under the Threads tab.

Supported Application Servers: All application servers that have JMX enabled.

Use Cases: You can use the thread count to determine how many active threads are running in your application. This is useful for determining usage trends. For example, it can show the time of day and the day of the week in which you usually reach peak thread count. In addition, the creation of too many threads can result in out of memory errors or thrashing. By watching this metric, you can reduce excessive memory consumption before it’s too late.

Thread Pool Metrics

Location: Under the Threads tab.

Supported Application Servers: Tomcat, JBoss 5 and 6, Resin, Jetty, WebLogic, TomEE, Glassfish, and WebSphere

Use Cases: Thread pools are typically used to service multiple requests simultaneously. However, to get the best throughput, thread pools must be configured appropriately. For example, if the maximum thread count is set too high, the app will slow down from excessive memory usage. But if the maximum thread count is too low, it will cause requests to block or timeout.

You can use these metrics to see if you are reaching the maximum thread count in a pool. In addition, they can be used to tune other properties – such as the amount of time before an idle thread is destroyed and the frequency of when new threads are created.

This graph displays information on the http-bio-8080 thread pool on a Tomcat 7.0 application server. It shows that the thread pool starts with 10 idle threads and never handles more than five active requests at a time. The 0.23% capacity indicates that the number of active threads is well under the limit.

Session Metrics

Location: Under the HTTP Sessions tab.

Supported Application Servers: Tomcat, JBoss 5 and 6, Resin, TomEE, Glassfish, and WebSphere

Use Cases: HTTP session information is used to determine usage trends such as the time of day when an application is getting the most amount of traffic. It can also be used to tune configuration properties such as the maximum number of active sessions allowed at one time and the amount of time a session remains active. For example, a high rejected session count usually indicates that the maximum active session count should be increased. Meanwhile, a high expired session count can suggest that the session timeout is too low.

The graphs below show session information for two applications. The first indicates that a maximum of two sessions have been created for the application ‘examples’, but the sessions are constantly expiring. After increasing the timeout, the number of expired sessions reduces to zero. From the second graph, we see that zero sessions have been created for the application ‘host-manager’.

Transaction Metrics

Location: Under the App Server Transaction tab.

Supported Application Servers: JBoss 7, Resin, and Glassfish

Use Cases: These metrics show info on transactions that go through the application server’s transaction manager. They are used to show transaction traffic patterns and help to configure the transaction manager.

Get Started Today
New Relic uses Java Management Extensions (JMX) to gather data on these new metrics. Before you get started using these new metrics, you must update to our latest Java agent and enable JMX on your application server.

You can also set up New Relic to show custom JMX metrics. To see how to display custom metrics, watch this video.

The post Dive into your JVM with New Relic appeared first on New Relic blog.

So, how do we test code that’s heavily dependent on other components? How do we validate different combinations of frameworks that can occur in the wild? Read on for the nitty gritty of testing the Ruby agent in multiple environments.

The Uncertainties of Life (and Testing)
Before coming to New Relic, I was an app developer. At that time I never realized how much simpler life was when you could choose 1) what dependencies to take, 2) how the production environment was configured and 3) the fundamentals, like language versions. Unfortunately, library authors don’t get these certainties.

The richness of the gem ecosystem keeps the Ruby agent team on our toes. We support at least six different Ruby versions (depending on exactly how you count), eight versions of Rails, and a pile of other frameworks. If you ever need to be sure your code runs against Rails 2.0 and Ruby 1.8.6, I’ve got the test suite for you!

At the highest level, we follow a standard unit / functions (or integration) test breakdown. Let’s take a look at the unit tests first.

Unit Testing
Unit tests are the front line of our testing. They run quickly (about 10 seconds locally for the full suite of > 1000 tests), with the lightest dependencies. We execute them across the broadest number of combinations of Ruby and Rails versions.

Historically Rails has been a big focus for the agent, so the unit tests run in the context of a Rails application. While most of the agent’s unit tests could run without any external dependencies, we’ve encountered many subtle bugs from interesting intersections between Rails (I’m looking at you, ActiveSupport) and the various flavors of Ruby. Running the unit tests under Rails helps flush out these odd interactions. For example, we found one problem with subtly incompatible versions of the to_json method floating around during the 2.x versions of ActiveSupport.

Which version of Rails do we run against? Well, if you download the agent source and run the unit tests, you’ll see this:

♥♥♥~/source/newrelic/ruby_agent [1.9.3] [release]:rake test

/Users/jclark/.rbenv/versions/1.9.3-p374/bin/ruby -I"lib:/Users/jclark/source/newrelic/ruby_agent/test:/Users/jclark/source/newrelic/ruby_agent/lib" -I"/Users/jclark/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/rake-10.0.3/lib" "/Users/jclark/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/rake-10.0.3/lib/rake/rake_test_loader.rb" "/Users/jclark/source/newrelic/ruby_agent/test/new_relic/**/*_test.rb"

Running tests in standalone mode.
Run options:

# Running tests:

...more dots...

Finished tests in 8.206038s, 129.6606 tests/s, 338.4094 assertions/s.

1064 tests, 2777 assertions, 0 failures, 0 errors, 0 skips

The line ‘Running tests in standalone mode’ hints at what’s going on. In ‘standalone mode’ we construct a tiny Rails app in the agent that – by default – runs a Rails 3.2 app.

But what about all those other versions of Rails? That’s where rpm_test_app comes in. It’s a full, separate application with branches for each flavor of Rails we support. These get stitched together with some Rake goodness to run the unit tests for newrelic_rpm in rpm_test_app’s context:

Rake::TestTask.new(:newrelic) do |t|
  t.libs << "#{agent_home}/test"
  t.libs << "#{agent_home}/lib"
  t.pattern = "#{agent_home}/test/new_relic/**/*_test.rb"
  t.verbose = true
end

With `rake test:newrelic`, our CI system can run the gambit of Rails and Ruby versions. Jenkins configurations let us dictate which combinations make sense given that, for instance, you can’t run Rails 3.x and greater on Ruby 1.8.6.

Not All About Rails
That’s all good, but there’s much more to Ruby than just Rails. With the rich variety of frameworks to support, we needed an approach to mix and match dependencies.

This need led to our functional/integration testing layer, affectionately referred to as the ‘Pangalactic Multiverse’ tests.

At heart, Multiverse is a series of suites, each representing a collection of dependencies. We have one for Sinatra, one for Resque and one for ActiveRecord outside of Rails. The exact versions are specified by an Envfile:

suite_condition("Sinatra not compatible with 1.8.6") do
  RUBY_VERSION != '1.8.6'
end

gemfile <<-RB
  gem 'sinatra', '1.3.3'
  gem 'rack-test', :require => 'rack/test'
RB

gemfile <<-RB
  gem 'sinatra', '1.2.8'
  gem 'rack-test', :require => 'rack/test'
RB

Multiverse reads the Envfile and generates a Gemfile for each call to `gemfile`. It will bundle install those dependencies and run the suite’s tests in a separate child process.

Unlike the unit tests that focus on individual classes, Multiverse tests run a full agent cycle – starting up, monitoring work, shutting down and transmitting data. They even run a fake service endpoint so data flows over HTTP, keeping as close to the runtime reality of the agent as possible.

With the additional environment spin-up and running our full stack, Multiverse tests are slower, but vital for coverage. We can easily write targeted automated tests against multiple different versions of individual frameworks, and validate that the agent works with new releases as they come out.

The Envfile lets each suite run multiple sets of dependent versions. And our CI system takes care of running Multiverse against various Ruby versions, too. So many combinations!

The Hard Part
Testing always presents unique challenges. And a library like the Ruby agent introduces its own set of fun twists.

It’s About the Environment
A lot of the work on our testing system has gone into getting environment setup scripted, both locally and in CI. We’re currently using a solution with Vagrant and Puppet that lets us spin up consistently formatted build boxes.

Leaky Test State
Testing needs to mimic reality to find some types of problems. This presents difficulties for the Ruby agent.

In production, the agent starts once per process. However, in a test run, the agent might restart hundreds of times. We’ve had to take care to avoid a leaking state between tests. Otherwise we could be faced with tests that work fine in isolation, but break when run in the suite. However, an upshot of this pain is the test_bisect gem my coworker Jon wrote. If you’ve ever suffered from leaky tests you should check it out.

Timeliness Counts
It’s also hard to write good tests for monitoring time and the underlying system. It’s much easier to write brittle timing tests that run fine locally, but break when deployed to the slower VM-based CI system under heavy load. Given how much of the Ruby agent’s functionality is about accurate timing, this is a continuing pain point we wrestle with in testing. In general, we’ve moved toward stubbing time values in unit tests and applying thresholds or percentage comparisons rather than strict timeframes whenever possible in functional tests.

Need for Speed
Running Multiverse is relatively slow. So, we’ve worked hard to prune that time. Two major approaches were: a) keeping suites coarse grained and b) applying local bundling. Both of these hinged off the setup being the slowest part of Multiverse.

In the first case, our early experience saw new Multiverse suites cropping up per feature. Often these would just copy an existing environment with minimal tweaks. Merging those suites along the lines of their gem dependencies gave a huge performance bump. Getting the right granularity for grouping tests was a big win.

Second, the Multiverse setup tries to bundle without contacting RubyGems, but re-runs a full bundle if that doesn’t work. Less network chatter = faster setup = faster tests.

Conclusion
That’s how we deal with testing multiple frameworks and Ruby environments on the Ruby agent team. It’s given us a nice balance of having our fast unit tests against the most common cases, with easy access to full-stack Multiverse tests to cover other configurations.

Are you a gem author who tests against different frameworks and Ruby implementations? Got a trick (or two) that’s helped you keep your dependency testing under control? Tell us about your experiences in the comments below.

The post Test All the Combinations! appeared first on New Relic blog.

Here’s an example of Green Code in Ruby:

class Customer < ActiveRecord::Base
  has_many :orders
end

In this code snippet, has_many is ActiveRecord’s Red Code. There’s a lot of machinery backing the has_many code that the Green Code doesn’t need to concern itself with.

You’ll often find Red Code in frameworks, APIs and DSLs. There are numerous examples including Gradle, Geb, jQuery, Mocha, Rake, Rails, RSpec, and Spock. And depending on the language, Red Code typically makes heavy use of some of the more powerful language features (e.g. meta programming in Ruby, byte code manipulation in Java).

However, the Red Code / Green Code concept isn’t a new one. In his book On Lisp, Paul Graham referred to this as Bottom-Up Programming. Graham encouraged his readers to increase the expressiveness of the language (Red Code) in order to make their code shorter and more readable (Green Code).

Depending on the task you’re trying to solve, you’ll probably want to be careful about writing any Red Code as it is apt to violate the principle of least surprise. For example, in Java you may come up with a clever solution using reflection. But reflection is unusual enough that it will likely trip up unsuspecting navigators of the code. As a good rule of thumb, candidates for Red Code are areas of your code that would make good open source libraries. To be worthwhile, the benefit of the Red Code has to outweigh the cost of discovering the unusual behavior, learning its proper usage and maintaining the more complex solution.

To illustrate this, take a look at the code behind Backbone.js. It’s very much Green Code and there are no big surprises. Then compare it with Angular, where there’s clearly Red Code under the hood. When you use the frameworks,  you’ll notice the difference, too. Backbone.js is easier to get into, but you need to write a fair amount of boiler place code. On the other hand, Angular has ‘magic’ conventions that take time to get use to. But once you’ve adjusted to how it work, you’ll benefit from the reduced boiler plate code.

Do you have experiences working with or writing Red Code and Green Code? Tell us about your experiences in the comments below.

The post Red Code, Green Code appeared first on New Relic blog.

RailsConf 2013 has begun! To celebrate, we’re publishing a series of blog posts that highlight what’s new and exciting in the world of New Relic’s Ruby support. Don’t forget to check out the entire series so far: Cross Application Tracing, Thread Profiling, Living on the Edge with Rails 4 & Ruby 2, Thread Safe APIs & Sidekiq Support for Your Threading, and Security Banners (For If You Forgot to Apply the Latest CVE Patch).

If you’ve spent enough time working with a production system (or even just a continuous integration server), you’ve surely encountered a Ruby process that was ‘stuck.’  You know the type – not crashing, but not making any real progress either. And giving you no obvious indication of what it was doing.

When finding a stuck process, your primary goal usually is to just get the wheels of progress turning again. And the quickest path to that goal is usually to `kill -9` the stuck process and restart it.

But I want to you to make your future self a promise. The next time this happens, don’t just kill that stuck process and move on. Dig in and figure out what’s really going on. You might just discover the root cause of your problem, hone your debugging techniques or discover a clue that helps you fix that tricky bug in your favorite open source project that no one has been able to pin down.

Since encountering a stuck process (especially in production) is often stressful, I find it helps to think about the debugging steps you plan to take ahead of time. Create a ‘checklist’ of things to run through so you don’t forget anything or skip any steps. Failing to capture enough information to reproduce a bug usually means you’ll hit it again. So make sure you seize the opportunity when it’s available.

Here are the steps I follow when debugging a stuck Ruby process:

Who Said What?
First, figure out exactly what process is stuck. This sounds trivial, but it’s especially important for systems like Unicorn, Passenger, Resque, etc. where the system is composed of multiple processes that have different roles (often a single master process plus a number of workers).

Identifying exactly which process is stuck is important in order to get a full picture of the behavior of the system as a whole. The trusty ps utility is usually my first stop here — a quick `ps aux | grep <process name>`.

I like to keep a transcript of the exact commands I’ve run, along with their output. And the ps aux output is usually the first thing that goes into that transcript.

Tools like htop and pstree can also be useful here. They give you a visual tree of the relationship between the stuck process, its sibling and its parent process (e.g. a Unicorn master and its workers).

Grab Any Logs that are Available
Again this may seem obvious, but the goal of the checklist is to remind you of everything you need to track. This will give you the best possible chance of debugging the problem offline. Before getting started, consider these questions:

* Where is STDOUT of the stuck process being sent?

* Where is STDERR of the stuck process being sent?

* Are there any other log files created by the stuck process? Check the command line and config file used for the process. Did either specify a log path?

* Are there any other log files created by processes related to the stuck process?

Other Non-Invasive Basics
Try gathering the following before you move into the more invasive actions:

* Grab the list of open files and sockets from the process using lsof -p <PID>

* If you’re on Linux, poke around in /proc/<PID> and consider gathering:

* cmdline contains the command line that was used to launch the process

* cwd is a link to the current working directory of the process

* environ is a list of environment variables set for the process

Check the CPU Usage of the Stuck Process
There are two basic flavors of hangs: static and dynamic. (Note: This is my own terminology, but I think it’s a useful taxonomy.) In a static hang, the threads of the stuck process are stuck in the same exact state for an extended period of time. Examples include traditional deadlocks and processes blocked for extended periods in system calls (often things like `read`, `select`, etc.). In a dynamic hang, the threads of the stuck process are changing state, but are caught in some kind of loop that prevents them from making progress (this is sometimes called a ‘livelock’).

The techniques used to debug the two types of hangs are different. So it’s useful to be able to differentiate between them in the wild. A quick heuristic for doing this is to check the CPU usage of the stuck process. Static hangs will generally result in processes with no CPU usage, whereas dynamic hangs will result in processes with non-zero CPU usage (often very high, if the process is stuck in a tight loop).

There are a variety of tools you can use to do this, but a quick look at `top` or `ps -o cpu <pid>` is often enough.

Static Hangs
For static hangs where the stuck process demonstrates no CPU usage, gather information on the backtraces of all threads in the stuck process. This is often the most important data. Since the threads aren’t changing state, a single snapshot of thread backtraces is usually sufficient.

There are a variety of different tools for accomplishing this goal, depending on the platform you’re running on and the Ruby implementation you’re using, but I’m going to focus on what is perhaps the most common case: MRI on a Linux host.

`gdb` is a time-tested and widely available tool that can be used to interrogate a running process on a variety of UNIX-ish platforms. You can attach to a running process with gdb by using the -p option, like this:

```
gdb -p
```

Once gdb has successfully attached to the target process, it will drop you at a prompt. The very first thing you should do here is to gather C-level backtraces for all threads of the stuck process, like this:

```
(gdb) t a a bt
```

This is short for `thread apply all backtrace`. Copy this output and save it to your transcript file, even if it’s not immediately informative.

Sometimes C-level backtraces are all you need in order to debug a hang. In other cases though, you really need a Ruby-level backtrace to see what’s going on. Luckily, gdb can help you generate those, too.

Getting a Ruby Backtrace with gdb
Getting a Ruby backtrace out of gdb involves interacting with the Ruby interpreter in the running process that you’ve attached to. And to get that, the interpreter needs to be in a semi-working state.  If the root cause of the issue is a Ruby bug, the process may be stuck in such a way that it isn’t possible to get the backtrace at all.

That said, it’s always worth trying. Here’s the technique I use:

Most mechanisms for dumping Ruby backtraces will output them to stdout or stderr. It’s fine if you know where the stdout and stderr for your process is going, and you have access to it. If not, first you’ll need to redirect the output streams to an accessible location.

To do this, we rely on POSIX conventions and some trickery. stdout and stderr are file descriptors 1 and 2 on POSIX-compliant systems. We want to close these file descriptors, and reopen them attached to files that we can actually read from. Closing is easy:

```
(gdb) call (void) close(1)
(gdb) call (void) close(2)
```

The call command tells gdb to invoke close from within the target process. Next, we need to re-associate file descriptors 1 and 2 with a file we can actually read from. We could use a file on the filesystem, but it’s even more convenient to see the output directly in gdb. To do that, we want to figure out the device file corresponding to our current TTY and open it twice. Since file descriptors are assigned sequentially, descriptors 1 and 2 will end up being re-associated with our TTY device. Here’s how this looks:

```
(gdb) shell tty
/dev/pts/0
(gdb) call (int) open("/dev/pts/0", 2, 0)
$1 = 1
(gdb) call (int) open("/dev/pts/0", 2, 0)
$2 = 2
```

Now, any output generated by the target process will be echoed directly to our console so we can see it in our gdb session.

Finally, we need to get the Ruby interpreter in the target process to spit out a backtrace. There are many ways to do this. But the easiest way is just to use gdb to call the rb_backtrace() function from the Ruby interpreter directly:

```
(gdb) call (void)rb_backtrace()
```

With luck, you’ll see a Ruby backtrace dumped to your console.

Digging Deeper with gdb
If you know what you’re looking for in the stuck process, there’s a lot more you can do with gdb. Jon Yurek at thoughtbot has a great post on this topic that explains how you can get gdb to evaluate arbitrary Ruby code in the target process to inspect the values of Ruby variables, get backtraces for all threads and any number of other things

Dynamic Hangs
A stuck process that shows high CPU usage is likely stuck in a tight loop. This means that the exact backtraces you get out of it may vary from one sample to the next, but backtraces can still be very helpful to point you in the right direction.

If you find a stuck process with high CPU usage, it may be worth gathering backtraces 2 or 3 times to get a more representative sample. (You can use gdb’s continue command to resume the process after you’ve attached).

You can also use other tracing tools to examine the behavior of the looping process. On Linux, strace -p <pid> allows you to view the system calls being made by the process. If you’re on an OS that has dtrace available, you can use dtruss -p <PID> instead to get a similar output.

Thanks, from your Future Self
You might not always have the luxury of being able to spend time debugging a stuck process. But when you do dive in, you’ll probably be glad you did. I’ve only scratched the surface of the tools available for analyzing problems like this, but hopefully I’ve given you enough information that you’ll be comfortable doing more than just sighing, SIGKILL-ing, and waiting for your problem to reappear.

Bonus: Mac OS X
If you happen to be debugging a stuck process on Mac OS X, check out this sample utility (a handy tool that ships with OS X). It will automate the collection of multiple C-level backtraces, aggregate them together and print out a nicely formatted call tree report showing where the stuck process is spending most of its time.

At RailsConf 2013? Stop by our booth to see the New Relic Ruby Agent in action, pick up your free Data Nerd t-shirt and more. You can even try New Relic Pro free for 30 days. For more information, go to newrelic.com/railsconf.

The post Debugging Stuck Ruby Processes – What to do Before You Kill -9 appeared first on New Relic blog.

The Case for New Relic
So you completed our SnapStack tutorial series, and released your app into the App Store. The work is over, right? Contrary to belief, the work has just begun. Even after your app has shipped out, the development process continues. No amount of testing will substitute for the experience of putting your app into the wild. Developers are constantly responding to users interacting with their app, sometimes in ways they might not have imagined.

Unfortunately, this makes the debugging process exponentially more difficult. Many developers spend large amounts of time scouring the reviews of their app, looking for reappearing bugs or issues. This leaves them firefighting issues blindly; relying on differing, incomplete accounts from user reviews. In addition to handling issues in a reactive manner, the practice comes at a price: developers often have to wait for a batch of poor reviews before they’re alerted to an issue.

New Relic solves the problem with its mobile SDK, which provides real time HTTP request and error monitoring. The mobile monitoring SDK enables developers to be proactive about identifying and fixing bugs. New Relic doesn’t stop at error logs; their dashboard presents plenty of useful and informative data about a live app, all in an easily accessible manner. The more context the better — with New Relic, app developers can make informed decisions at every turn.

Adding the New Relic SDK

1. If you don’t have one yet, sign up for a New Relic account.

2. Log in and click the Mobile tab on the left.

3. From the Mobile Dashboard, select your app (create one if you haven’t done so) and click the Settings tab.

4. From the Settings tab, click Installation.

5. Select the iOS SDK and download it. Unzip it and drag NewRelicAgent.framework into the frameworks folder of the project.

6. Add CoreTelephony.framework, libz.dylib and SystemConfiguration.framework to the project.

7. In the MOBAppDelegate.m file add the following import: #import

8. Add this line of code at the very beginning of the application:didFinishLaunchingWithOptions: method.

Replace YOUR_NEW_RELIC_KEY with your New Relic application token:

[NewRelicAgent startWithApplicationToken:@"YOUR_NEW_RELIC_KEY"];

9. Build your project and run, and you should see activity in your New Relic dashboard immediately.

Congrats!
We’ve reached the end of this tutorial. In it, we discussed the challenges and pitfalls that developers encounter with live app debugging. Next, we presented the solution that New Relic provides, and covered the benefits of the New Relic mobile monitoring SDK. Finally, we leveraged the partnership between StackMob and New Relic, and walked through adding it to our SnapStack project.

Are You a StackMob Customer?
Through our partnership, StackMob customers get New Relic Standard free of charge. You’ll even get a free 14 day trial of New Relic Pro. So, what are you waiting for? Sign up for your free account today!

The post SnapStack App Bootcamp: Adding New Relic Monitoring to SnapStack appeared first on New Relic blog.

* Steve Souders looks at how fast we really are in his slides from the HTML5 Conference.

* The Web Performance Today podcasts talks CDNs with guest Aaron Peters.

* Adams Wiggins, CTO and Co-Founder of Heroku, publishes an extensive FAQ on routing and web performance on Heroku.

* Anselm Hannemann describes why he uses a CDN for static files on GitHub.

* Harry Roberts has a few tips for designers and developers to make their frontends even faster.

* Joshua Bixby wonders why eCommerce sites are slower than they were last year.

* Johan Johansson shares tips for making your website faster on mobile devices.

* Intechnica looks at the worst 15 web performance nightmares and the damage they caused.

The post Community News: Speeding up the Mobile Web and More appeared first on New Relic blog.

Today Heroku’s CTO, Adam Wiggins, published an extensive FAQ discussing routing and web performance on Heroku. The FAQ is a complete guide to the routing layer, a retrospective on how things evolved and a look into the future. In the FAQ, you can find how to determine if your application is affected by the routing layer, as well as extensive help on what steps to take if it is.

This is essential reading for anyone running a Rails application on Heroku.

The post Heroku CTO Publishes Complete Routing and Performance FAQ appeared first on New Relic blog.

Our UI exposes a lot of information in a little bit of space and it can be easy to miss available functionality. To get the most from New Relic, we recommend that you get started with our existing site documentation beginning with the Application section. This area goes into the application dashboard screen in great detail. You can get information on each of the following categories:

* Overview
* Application Map (aka App Map)
* Web Transactions
* Database & Slow SQL
* External Services
* Background Tasks
* Geography
* Browsers

If you don’t have time to read through all that, here are some tips to get you started quickly.

The Application Overview Page
The Application Overview page has details about the overall state of your application and a quick look at the data that’s fleshed out in the other tabs. It’s a good place to see your performance trends at a glance – from page load times, to error rates, slow transactions, and a list of servers running the app. There’s a great deal of data here to be found and you can click through to take a deeper dive into any category.

Here you can experiment with the time picker and zoom in on any of the graphs. And if you’re using our Server Monitoring feature, you can see a list of servers at the bottom of the page. Click through and you’ll reach the server monitoring section of our UI.

In the Response Time graph, you can see how events like outages, setting changes and deployments are affecting your performance. And the Browser / App Server selector allows you to hide or expose Real User Monitoring (browser) data.

You’ll also find a historical comparison button that will let you see how your performance has changed over the past day or week.

The App Map Page
The App Map page is interesting if your app talks to external services or other apps New Relic knows about. If your app does have multiple tiers, this page provides a good summary dashboard of all your external (HTTP) services, databases, other apps being monitored by New Relic, and browser load times, with response times and throughputs for each of these services.

The Web Transactions Page
The Web Transactions page shows you where the app spends most of its time and effort, and causes the most user dissatisfaction. Its also where you can find overview details about a type of transaction, as well as specific recorded slow transactions including HTTP parameters, an execution waterfall graph and details about slow SQL queries.

On this page, the number one under-used feature is the Sort Order dropdown. There we show you the transactions chewing the most CPU, but you may be more interested in the highest throughput or flat-out slowest transaction, even if it’s infrequently called. Also note the Graph vs. Table View selector in the upper right. This will give you a more complete list instead of a top five graph.

And don’t miss the Show more slow transactions link in the bottom right. It will show you a more complete listing for the time window you’ve picked, sorted in order of slowest execution.

The Database Page
The Database page shows you information about your app’s SQL usage – where your app spends most of its database times and effort, and it’s where you can find details about your slowest queries. The Sort order dropdown can help you re-filter your results and don’t miss the ability to click through a specific trace to the transaction name.

The Externals Page
The Externals page shows you detailed throughput and response graphs about the services you contact via HTTP.

The Background Page
The Background page has information about tasks you’ve marked as background and/or we know are run from the command-line. We do collect transaction traces for background jobs as long as the run time exceeds your trace threshold. You can click through to any errors we’ve recorded.

The Geography Page
The Geography page helps you visualize how your RUM timing data varies by geographic location. Go to the Settings (Application) page to break out specific countries or states. Once again, sort order will let you see the list organized in the way that’s most important to you.

The Browsers Page
The Browsers page lets you visualize your RUM data per browser statistics. It’s a great place to find a focus for browser specific optimization or to see if optimizations are even necessary.

Next Steps
If you’re looking for a good follow up to this post, check out our general conventions for using the New Relic dashboard. It includes instructions for setting your view to include a date range or particular server.

We’ll be running a follow ups to this post soon, so remember to check back later. And if there’s anything you’d like us to cover in the future, let us know in the comments below.

The post A Quick Guide to Getting Started with New Relic appeared first on New Relic blog.

Even though I work at New Relic, I run a couple of personal websites that were affected by the event. And New Relic helped me quickly figure out what was going on and what I could do about it.

It All Started with a Page
My first indication that there was a problem came from a page sent to my iPhone. At 10:46 am, I got the following message:

“ALRT #212 on New Relic: Alert on nimbus secure opened”

The page came from PagerDuty (another great support tool), that I have setup to receive notifications of events from New Relic. Within minutes of the start of the event, I was engaged.

What Was Going On?
So, what was up? I logged into New Relic and immediately found that my applications were in distress:

Neither application was serving traffic. I’d already received an alert on one of them. And the second was about to … obviously something was very, very wrong.

I clicked on the first application. This confirmed no traffic was going through. My Apdex score had dropped to zero:

Given this, my next step was to look at my application servers:

I noticed two things here. First, half of my servers were being impacted – all within in single AWS availability zone. The number went up as the event went on and spread. Secondly, my servers were reporting that disk I/O went up to 100%. I clicked on ip-10-1-0-101, which is my primary webserver for my distressed application and got the following charts:

From these, I saw that the processes were getting backed up trying to do disk I/O which appeared to look backlogged. Looking further, I saw this:

I saw that I was no longer sending data over the network to my disks (in AWS, the disks are on remote servers, called EBS volumes.) Obviously, Amazon’s EBS service was having problems. This is a common cause of outages on AWS, so it wasn’t much of a surprise to me. I went to the Amazon website and confirmed they were aware of the problem and working on a resolution.

Now What …
You can see that within a few minutes of the problem occurring, and with just a couple minutes of evaluation of data on existing New Relic charts, I was able to determine exactly what the problem was and exactly which servers were impacted and in which availability zone. (Amazon never tells you which availability zones are impacted, just whether it’s one or more.)

As a result, I knew exactly what my options were. For example, I could have deployed new servers with backup images to other servers in other AWS availability zones (avoiding the problematic one) or have traffic routed to servers I setup in other backup service providers.

Summary
In this particular case, the problem came from a large-scale outage on AWS and it was difficult to self-recover since so much was impacted. However, this sort of problem occurs on a much smaller scale at other times with AWS and similar providers. Disk volumes go bad, servers go down, I/O gets backlogged, etc. The point is that I was able to immediately diagnose the problem in just a couple minutes using New Relic, and could quickly switch my focus to helping my customers and working around the issue. Using just the tools and data that AWS (or other service providers) give me, it’s a potentially significantly harder — and more time consuming — problem to solve.

Have you run into a similar situation? Let us know in the comments below.

The post Is Your Provider Down? New Relic Can Help! appeared first on New Relic blog.