We have a confession to make. New Relic is still running on Rails 2.3. We’ve finally decided to bite the bullet and make the upgrade to Rails 3. And it will happen with a three-line commit.
A Three-Line Commit?
No, we’re not talking about a commit to deploy a different branch. As David Celis mentioned a few weeks ago, at New Relic we try to avoid long-running feature branches whenever possible. That means we’ve gone to extraordinary lengths to prevent this upgrade from becoming the mother of all such branches.
We avoid long-running feature branches because they can introduce unnecessary risk. Specifically, a developer may find that his or her unconscious assumptions about how other code behaves may prove incorrect when it’s time to merge the code back in. Frequent rebasing can avoid this problem, but it’s particularly dangerous for what is essentially a wide-ranging refactoring project. Other developers still need to be able to do their work, so that daily rebase may be an exercise in frustration as they add new code that needs to be fixed.
1. Introduce a flag controlling whether the old or new implementation should be used.
2. Add the new implementation inline with the old, but have it turned off by the flag.
3. Test with the flag turned off and on.
4. Deploy with the flag turned on.
5. After ensuring that everything is working correctly, delete the old code and the flag.
A few weeks ago, we followed this pattern through to completion when we changed the underlying infrastructure of our long-running background jobs. Before, we used BackgroundJob, which has given as no end of trouble when upgrading Ruby and Rails. Instead of patching it again, we decided to move to Resque. We wrapped Bj’s queueing code in an abstraction, pointed everything that used Bj at the abstraction, and implemented a Resque version of Bj’s behavior. When we flipped the switch in production, all of our long-running jobs seamlessly started using Resque:
Our Rails 3 upgrade path follows the same pattern. But it has a few quirks of its own. As anyone who has done a Rails 3 upgrade knows, the worst and least compatible part of the upgrade are the changes to how Rails boots and initializes itself. Configuration files have to be moved around, names have to be changed, etc. By far, the worst part of the upgrade was jury rigging a set of files that boots our app correctly in both Rails 2.3 and Rails 3.
Here’s an example of a bit of the craziness we had to write:
if is_rails3 # We need to do two monkey patches to Bundler in order to support two different # Gemfile.lock files. # First, Bundler::DSL - at this stage eval has just been run on the Gemfile, and # we now have access to monkey patch Bundler. The Gemfile.lock has been stored in # a variable, and is being passed to "to_definition". We need to alias_method_chain # on to_definition to intercept and replace the old default_lockfile_path # # Secondly, we need to monkey patch the global default_lockfile definiton, in Bundler::SharedHelpers. class Bundler::Dsl if !self.method_defined?(:to_definition_without_rails3_lockfile) alias_method :to_definition_without_rails3_lockfile, :to_definition end def to_definition(old_lockfile, unlock) current = File.expand_path(Dir.pwd) filename = File.join(current, "Gemfile_rails3.lock") lockfile = Pathname.new(filename) to_definition_without_rails3_lockfile(lockfile, unlock) end end module Bundler::SharedHelpers def default_lockfile current = File.expand_path(Dir.pwd) filename = File.join(current, "Gemfile_rails3.lock") lockfile = Pathname.new(filename) Pathname.new(lockfile) end end end
That monkey patch to Bundler is within our Gemfile. We needed similarly unclean changes in other files in the Rails boot sequence. At the end of all of that, it booted correctly in both environments. We then iteratively fixed the test suite in Rails 3 while maintaining compatibility with Rails 2 — oftentimes switching out implementations by writing the Ruby equivalent of a C preprocessor flag. That resulted in 225 patches. At the same time, we added the
rails_xss plugin to Rails 2.3 and sprinkled
html_safes around liberally. While we were making all of those changes, every other developer who touches the core app was just doing his or her normal job.
All of the changes we’ve made for Rails 3 are on master, the branch we deploy to production. When we decide to deploy Rails 3, all we’ll have to do is enable the Rails 3 code at all times. The process has been painful at times, but the risk to the business was low. And most importantly, the deploy will be extremely low risk since rolling back is equally as small a change as moving forward.
Has your organization made the move to Rails 3? Tell us about your experiences in the comments below.