Thread Safe APIs and Sidekiq Support for Your Threading

By Posted in Events, New Relic News, Tech Topics, Top Post 25 April 2013

RailsConf 2013 logoRailsConf 2013 is right around the corner! And to celebrate, we’re publishing a series of blog post that highlight what’s new and exciting in the world of New Relic’s Ruby support. Don’t forget to check out the entire series so far: Living on the Edge with Rails 4 and Ruby 2Cross Application Tracing and Thread Profiling.

A few years ago multi-threading support in Ruby was flaky or nonexistent. The Ruby agent came of age in that world, where the only real option for concurrency was running multiple processes. Recent versions of JRuby and MRI have changed that landscape. With Rubies providing OS-backed support for concurrency primitives, libraries and servers are taking advantage of threads like never before.

In light of this, we’ve done a significant rewrite – focused on thread safety – to how the Ruby agent accesses its metric data structures. All the agent internals now work through a set of methods which ensure concurrent reading and writing of metrics is safe. In the past highly threaded applications could encounter inaccurate monitoring or contention issues, but those should no longer be a problem for the Ruby agent.

Sidekiq logoAs an added bonus, revising the APIs also let us update them for consistency and usability. Users doing more sophisticated custom instrumentation will benefit from the cleaner API, along with the thread safety it provides. Taking advantage of the agent’s thread safe internals, we’ve responded to a common customer request by adding support for Sidekiq. Sidekiq is a background task framework similar to Resque. Where Resque uses a forking model, Sidekiq approaches this problem by executing jobs concurrently on multiple threads per worker process. Keeping fewer worker processes means less startup time, less memory consumption, and more sharing of resources between jobs. While it requires a keen eye toward concurrency issues in your code, for certain types of jobs Sidekiq’s multi-threaded approach can be a big performance win.

Sidekiq monitoring is enabled automatically by the New Relic Ruby agent starting with version 3.6.0.

Headed to RailsConf 2013? Stop by our booth to see the New Relic Ruby Agent in action, pick up your free Data Nerd t-shirt and more. You can even try New Relic Pro free for 30 days. For more information, go to newrelic.com/railsconf.

About the author

jclark@newrelic.com'

Tell us your thoughts Or Send us an internal high five

Talk to @newrelic