Recently, I started building a very thin JSON API service in Ruby. All it would need to do is perform a set of simple create / update / get operations on one database table; it wouldn’t even need a whole set of REST operations. The catch was that it’d need to handle as much traffic as some of our busiest Ruby applications, so one of the first decisions my team needed to make was whether to build this using Rails or Sinatra.
The trade-offs between Rails and Sinatra are generally well-known in our community: Rails offers you a lot of magic and convenience while keeping you from having to worry too much about the underpinnings of your web app, while Sinatra’s claim to fame is its speed and tiny footprint. But Ruby was built on the idea that processor time is cheap while developer time is expensive, and much of the language is optimized for the human element. In other words, since you’ve already chosen Ruby over C++, then Rails probably seems superior to Sinatra for similar reasons.
Many developers (including myself) tend to build all of their web apps in Rails, even these super-thin service apps. The justification is that “eventually, you’re going to need *XYZ Rails Feature* anyway, so you might as well use Rails from the start.” But I was curious just what kind of performance I was trading away for the sake of keeping my Rails magic. And boy was I surprised.
For this test, I wrote the same application in Rails (v 4.0.1), Rails-API, and Sinatra (v 1.4.4) with Sinatra-Activerecord (v 1.2.3). The application connects to an empty MySQL database, polls a count value from an empty table, and returns a body containing the result in an interpolated string, using Thin as its web server. In Rails-API, I removed Action Mailer and Sprockets. Finally, the apps were tested using “siege -r 1000 -b localhost:3000”. Siege defaults to 15 simulated users, so this test will run 15,000 GET requests. I repeated these tests a few times – not enough to fully eliminate noise, but enough that I’m confident the results are accurate performance indicators.
The Surprising Results
Rails handled the requests fairly well, finishing the test in about 40 seconds at 378 transactions per second. Rails API didn’t handle things much faster, finishing in a bit over 38 seconds, or 391 transactions per second. Sinatra blew away the competition at 1023 transactions per second, completing the benchmark with no errors in just 14.65 seconds.
The thing that surprised me most about these tests was how little performance difference there was between Rails and Rails-API. A 3.5% performance increase isn’t quite enough to impress me, but Sinatra’s nearly-tripled performance is.
Of course, even 378 transactions per second is a lot of requests — it would have to be a rather busy API to start running into performance problems at that rate. So going back to our original question, which are you “eventually” going to need in your API: Rails magic, or killer performance? For my APIs, I prefer to bet that the app will be wildly successful over betting that I’m going to be feeling lazy in six months. Sinatra wins, hands-down.
If you want to dig through the full test results, they’re available here.