I started building a Rails application with my children today. We’ve been pretty obsessed with ponies lately, and finding good pony resources online can be difficult.
We decided to build our own application with a couple simple requirements. We need it to list all the ponies by type at unique urls, and we need it to perform well when we refresh the page many hundreds of times a second (children get excited).
Rails 4 might seem like the obvious choice for this application as it is the most recent version, but my youngest suspected a discordian plot and demanded that I show him the numbers.
Fortunately for me the New Relic Ruby agent is launching Ruby VM stats this week, so it’s easy for me to get a good look at our application’s performance across different Rails versions.
We all agreed that we would select the version with the lightest memory hoofprint and we were off to the races.
We decided to build the same application in Rails 2, Rails 3, Rails 4 and the recently announced and entirely adequate AdequateRecord branch of Rails 4 (thank you to Aaron Patterson for your adequacy).
We used siege to generate some load for our application with 1 user and the -b flag to remove the delay (`siege -c 1 -b`). We may have generated more realistic metrics using child labor but children are easily distracted from furious mouse clicking by Gogurt and cartoons.
I selected Rails 2.3.18 for our Rails 2 version because it was released on my birthday. If you’re thinking of getting me a birthday present please don’t make it an archaic command line interface I’ve forgotten how to use. We decided to name this application ‘s’.
Rails 2 (S)
You can see that Rails 2 is allocating about 1820 objects per request, which we found very surprising. My son guessed 4 (that’s how old he is; he mostly only thinks about 4) and my daughter estimated an even billion. I’m glad the billion was wrong but I actually expected a bit more as I’ve only seen the object allocations metrics for large applications, where object allocations are usually an order of magnitude higher.
Memory usage is 111MB on average. It reminds me of the olden days and automated slaying of unicorns (we’re naturally running all the apps with the unicorn server) but I don’t think this would be a good time to reminisce with the kids, given our subject matter.
Garbage collection in Rails 2 runs .685 times per hundred requests. Given this is our first application that seems like a pretty reasonable amount of GC to me and my daughter. My son votes to scratch Rails 2 off the list as he knows there’s a version 4 coming.
Moving up the chain we come to Rails 3. We chose Rails 3.2.17 as it was released exactly one month before my birthday, and we all agreed birthdays are pretty awesome.
Rails 3 (ZOMG Ponies)
“Rails 3.0 is ponies and rainbows!” – Ruby on Rails 3.0 Release Notes
Rails 3 bumps the memory usage a whopping 9MB to 120MB per instance. The heap size is the same but object allocations increased significantly to 3250, an increase of nearly 79%.
GC frequency also increased to 1.19, nearly 74% more than the .685 per 100 transactions we had for Rails 2.3.
My daughter seems convinced at this point that we should keep it old school and stick to Rails 2.3, I’m praying for some evidence that I won’t need to revert to a pre-Bundler version like a caveman.
My son is bouncing off the walls with anticipation for the Rails version that shares his ‘number’, so we better get on with it before he overheats.
Rails 4 (The 4 Ponies of The Apocalypse)
Rails 4 dropped our object allocations back down a bit to 2740. We’re not back to Rails 2 glory but I think I can talk my daughter into it.
Memory usage climbed to 131MB, confirming my hypothesis that a Rails version is worth approximately 7 floppy disks. I tried to take a moment and explain to my daughter what a floppy disk was so she could get a sense of scale for this change, she was not interested. Kids these days have no appreciation for the value of a megabyte.
The GC frequency for Rails 4 dropped slightly to 1.18 runs per 100 transactions (down from 1.19). Looks like I’m going to have to rely on object allocation improvements to make my case.
Finally we get to address the latest innovation from Adequate LLC, AdequateRecord.
Rails 4 with AdequateRecord (Adequestria)
AdequateRecord brought the object allocations back up to 3020, still way up from the Rails 2 days but certainly workable.
GC frequency is the lowest we’ve had yet with .5 runs per 100 transactions, and our overall time spent in GC is also the lowest we’ve seen with .324% of wall clock time.
Memory usage is up a bit to 183MB, crushing my clever 7 floppies worth of Rails hypothesis. To be honest I’d gladly pay the difference for the speed increases Tenderlove described here: AdequateRecord Pro™
The heap size has also increased noticeably to 897k from the otherwise consistent 501-2k range with Rails 2, 3 and stock 4.
Ponies are awesome. We’re quite pleased with our progress on the pony application so far. We’ve decided to proceed with the Rails 4 version including Adequaterecord, even though it won’t be released until Rails 4.2. Sometimes when you’re developing a cutting edge pony application you have to take some risks and run the latest hotness.
My daughter made a strong case for Rails 2.3 but I won her over with a picture of Gorbypuff, explaining that Gorby had likely contributed code to AdequateRecord (7 year olds are particularly susceptible to cats).
She would also like to point out that she is prepared to talk with any VCs who may be interested in a very performant list of ponies.
There are some interesting changes in VM stats as Rails has progressed over the years. We hope you’ll find as much insight into your own application internals as you get a chance to play with this feature.
If you’re interested in reviewing the various pony applications developed for these experiments you’ll find them here: S (Rails 2.3), ZOMG Ponies (Rails 3.2), The 4 Ponies of The Apocalypse (Rails 4.1), Adequestria (Rails 4.1 with AdequateRecord).