Guest author Kit Reynolds is a project manager at Trainline, the leading online independent retailer of rail tickets in the United Kingdom.
Developer Paul Kiddie recently wrote a great blog post on Trainline about how we’ve made our customers—and our engineers—happier using New Relic. We were also given the opportunity to present some of these ideas at a recent New Relic User Group meetup in London.
Now that we’ve got New Relic ticking along nicely, with dashboards up next to our product teams so they can see metrics like error rate, revenue, and response times, we wanted to make sure we kept up to date with the latest features from New Relic. (Those updates have been arriving up to twice a month for the .NET APM agent we’re using.) As a cloud-hosted platform, New Relic has been continuously updated and improved, and we want to get the most out of our investment in it by keeping up to date with each release. It also makes our product teams happy when they get to play around with the latest new features.
A common question we were asked at the meetup was how we achieved this in a controlled manner.
Answering a complex question
We deploy our code using a mix of methods: Thougthworks Go, Microsoft SCCM (System Center Configuration Manager), and Chef. Different product teams have different methods, so procedures for testing and keeping the New Relic APM .NET agent up to date weren’t always obvious.
The approach we settled on is to let our Web frontend development team, known as Tango (Paul, who championed New Relic, is a member), trial the latest candidate New Relic APM agent. They have the benefit of already being a Continuous Delivery team; using Martin Fowler’s Blue-Green approach to offline deployment with fast switching—and rollback, if required. They can deploy the new agent to the Blue Slice, then perform the automated load balancing switch while monitoring how the new New Relic APM agent performs. If there are issues, it’s easy to switch back to the original Green Slice, running the older agent version that we know works.
Using SCCM to deploy New Relic updates
Once the Tango team is confident in the new New Relic agent, we combine it with work our server team does every month to update Windows with the regular patches. We use SCCM to deploy these patches, and after some debate we decided to use SCCM to do the same for New Relic. It makes sense to combine the New Relic agent update with the Windows patching because updating New Relic involves a Web server (for us, IIS) application pool recycle, which is almost as labor intensive as the server reboot we have to do when applying the patches (those have to be done outside of office hours).
To me this all sounds pretty straightforward, but New Relic vice president of customer success Bill Lapcevic assures me that it’s a model of best practice, and something we should tell the world about. We’re thrilled to do just that!
Be sure to read Keeping New Relic New on the Trainline engineering team blog.
RELATED CASE STUDY: Trainline Accelerates Release Cycles and Reduces Errors by 95% Using New Relic