As a developer, I love a really good engineering problem. But great problems don’t come along often, and big problems often turn out requiring a lot of effort for a relatively small overall benefit. On the other end of the difficulty spectrum, you can sometimes solve easier problems that bring a larger-than-anticipated benefit to your project. During the last few months I’ve experienced two examples of these small but important issues, and was able to solve both of them using what I call “digital duct tape.” Digital duct tape, like its real world counterpart, is a way of connecting systems together that may not be pretty, but is effective.
If you want to be data driven, you need to test your code against real data. Here at New Relic, we’re able to do just that with a set of test servers that point to our staging database. The problem is that the number of test servers is finite, and we need to be careful to not step on our coworkers’ use of them. To that end, each server has an endpoint that lists information about the code running on it: branch name deployed, Git commit metadata, etc.
In the interest of politeness, you would visit these pages when you need a test server, contact the person who last deployed, and ask if they’re still using that test server. The repetitive nature of going through all the servers to find an available one makes this a great task to automate. That would add visibility to who last used each test server, and when, so a developer can better target which test servers are most likely to be available. A perfect problem for fixing with some digital duct tape.
Fortunately, we also happen to have a dashboard using Shopify’s Dashing that we use to display the status of our site on our hallway big-screen–a great place to show test-server availability information. So I wrote a widget for the dashboard that periodically visits each server and gets the current deploy information in JSON, then extracts the person who was using the server and its last commit time (see below). It was a very simple widget to write, but my fellow developers were enthusiastic about the getting better visibility into test-server availability.
Bridging internal and customer-facing tools
Our engineering feature requests and bug tracking are handled by one tool; tickets with our customers are handled by a second. We tailored each tool to its specific task, but still need a bridge between the two when a customer provides a feature idea or identifies a bug. We can handle this with crosslinking, but that doesn’t provide visibility into the request’s priority, and there’s always the potential to have missing links. However, both tools have APIs which can be used to synthesize and summarize the links. Another excellent place to apply some digital duct tape!
To make our crosslinking more robust, I wrote a script in Ruby, which has gems to handle each of the APIs I’d be working with. First, the script queries each API to get a list of open issues, and scans their comments for crosslinks and priority information. Then it cross-references issues and calculates a value to indicate the impact of the issue itself.
It writes back the impact score to the engineering-facing issue, and prints a summary with a list of missing links (I later enhanced it with the graphviz visualization you see below). The impact score calculated by the script has helped us attack the problems that impact our customers the most, and the summary keeps issues from falling through the cracks by making sure their priority is visible across both systems.
Where to apply digital duct tape
Not all projects are right for digital duct tape, of course. Identifying where to apply it is an important part of making the approach work well. A good start is to make sure you’re working in a language you already feel comfortable in, or that has strong domain support for what you’re doing. Languages with REPL support can get your off the ground faster, and the language’s package repository (pip, rubygems, npm, etc.) may have modules for APIs you need to interact with.
It’s also important to start with a simple, clear goal that your digital duct tape can solve against a small data set (avoid big data). Some examples of areas that digital duct tape works well for include passing information between APIs, providing a summary or report, and adding visibility.
Whether you’re using digital duct tape to fill in a communication gap, or just automating a time-consuming task, always be on the lookout for opportunities where a small script can deliver a big reward.