At New Relic, we have split personalities – we create tools to help our customers’ Rails applications scale and perform; we use our own tools to optimize our own tools; and finally, we use our tools to manage our own service which happens to be one of the largest and most heavily used Rails apps in existence. Say again?
The guy who keeps the New Relic RPM service running is Bayard Carlin. He is our Director of Technical Operations. That means he is our ops team. He is responsible for maintaining the ongoing growth, scalability, security and reliability of our RPM service. No small feat that, since RPM has grown from zero to 18,000 applications under management in less than a year. RPM now handles more than 1.3 Billion metrics per day!
Bayard thought RailsLab viewers might find it interesting to see how the RPM site has evolved to handle the load and some of the lessons he learned along the way. A big shout out to the team at Engine Yard who have helped us every day with this ongoing challenge.
Take a look at his episode called App Server Provisioning and Tuning on RailsLab. Bayard describes how our architecture evolved to enable our data collectors to balance the distribution of the ever increasing data load from our customers applications. He also provides some details about configuring HAProxy, mongrel queuing, and optimizing the number of mongrels. Bayard also gives a sneak preview at our Capacity Analysis feature which identifies how busy your mongrels are. This will enable you to continuously manage your server capacity. Capacity Analysis is coming to RPM soon.