Part of HealthCare.gov’s “tech surge,” New Relic software was “a key tool that led to several fixes.”
Four months ago, administrators of HealthCare.gov released details about the site’s technologically bumpy road to recovery. The March 10, 2014, issue of TIME magazine* offers a behind-the-scenes account of what exactly that journey looked like. And as you learn just a few paragraphs in, the ride wasn’t pretty. For the article’s full text, see this PDF:
According to TIME’s account, there were different understandings of what the site’s actual capacity was; there was no dashboard that gave engineers a quick way to view and measure what was going on; and perhaps worst of all, there wasn’t a single person taking ownership for anything.
White House chief of staff Dennis McDonough, who was tasked by President Obama to solve HealthCare.gov’s myriad problems, was at his wit’s end. He turned to Jeffrey Zients, former deputy director of the Office of Management and Budget, to lead the effort, and from there, the president’s fix-it team soon followed.
Coders and troubleshooters from across the country joined forces in Washington, D.C. to power-up the efforts to get the site back on track. Google site-reliability engineer and Obama campaign veteran, Mikey Dickerson, was responsible for stamping out bugs and speeding up response times. He and Jini Kim, a former Google product manager, relied heavily on the APM dashboard that was rolled out to everyone on the team to monitor the site and stay on the same page.
As the Wall Street Journal reported* back in December, the New Relic software was part of this “tech surge” and was “a key tool that led to several fixes, including a new feature that alerts consumers via email when the exchange is available to process their requests.”
The Journal article detailed New Relic’s role this way: “A software agent installed on applications used to operate HealthCare.gov sends data to New Relic’s own servers, which then generate analytics reports on issues such as the amount of time it takes Web pages to load, database errors, and database response times. The information contained in the reports are presented as a color-coded dashboard accessible via the Web…”
Of course, there were a number of other technologies and tech leaders, including New Relic partner and Rackspace CTO John Engates, that all contributed to the improvement of the site. Collectively, these tools and people helped Obama’s trauma team get the HealthCare.gov site working smoothly and in time to meet its six-week deadline.
The TIME article is a fascinating read and one that serves as a cautionary tale of all that could go wrong in what otherwise seemed like a relatively straightforward website launch. As Dickerson is quoted in the article as saying, “It’s just a website. We’re not going to the moon.”
But as the White House learned the hard way, when you’ve got multiple contractors all using different tools and lots of integration that has to happen with existing systems, building a website can sometimes be easier said than done. Getting the right tools in place was critical for getting HealthCare.gov on the right track.
For the article’s full text, see this PDF:
[Image credit: TIME]