You solve problems with programming. You pump out line after line of code during Scrum sprints. You dream in Ruby and Go. You’re a programmer, a software developer, a software engineer. It’s not just your job, it’s your life.

But here at New Relic, we had a simple question: How do you know if you’re really any good at it? (And, just as important, how do software organizations and software development managers determine who is doing a great job and who isn’t?)

“You don't,” says Mac Browning, engineering manager at DigitalOcean. “The best engineers I know are constantly questioning their own abilities—and I think that may be in part what makes them great.”

Humility is required

The truth is, if you think you know everything there is to know about software development, that’s not a positive sign. If you think you’re great, well, you think you’re great—regardless of whether that’s reflected in the products and services you build.

“Evaluating a developer is more akin to evaluating a psychologist than evaluating a steel worker,” says Lee Atchison, senior director of strategic architecture at New Relic. “It is a subjective profession with subjective measures and a strong need for soft skills to be truly excellent.” And with the rise of DevOps, full-stack engineers, and agile programming approaches that focus on how well the code works in the real world, appraising programming skills is getting even more complex.

Measuring software quality and productivity is extraordinarily difficult

That doesn’t mean the software industry hasn’t tried—repeatedly, but with little success—to come up with hard metrics of a programmer’s productivity and skill.

Techniques for measuring developer productivity in the past have included measuring SLOC (source lines of code) that were produced in a day, week, or month. Code complexity was often measured using Branches/SLOC (a branch was considered a change in direction in the code, such as in an if/while statement). Code quality was often measured using “Defects/SLOC” while QA coverage with “percent of SLOC touched” or “percent of branches touched.”

More recent attempts at quantifying and measuring programming quality include tracking the number of changes submitted to open source projects or even the number of followers a developer has on GitHub.

“But all of these approaches, old and new, were seriously flawed,” Lee says. “They measured and thus encouraged behavior that often—almost always—ended up at odds with the goals of efficiently creating great software.” For example, Lee points out that using SLOC to measure programming speed encouraged developers to write long, inefficient routines: quantity over quality.

Kal Vissa, senior product manager at CloudBees, points to static code analysis (also known as source code analysis) as another measuring stick for programming excellence. It certainly has its place and can be useful in identifying how well your code can handle edge conditions, how easy it is to debug, and whether it’s well documented.

But while all of those actions might be things good programmers do well, none of them were ever truly useful as an objective measurement of programming ability per se.

education iconEducation is not the bellwether

The same holds true for education: computer science training can be valuable, but even the fanciest degree is hardly a standalone indicator of programmer skill. “Some of the best programmers come from some of the best universities; others come out of local code school programs,” Lee notes. “And some of the worst programmers come from some of the best universities,” he adds, “or out of code school.” In fact, some of the world’s most in-demand programmers are self-taught or come from a completely different academic background.

The same holds true for certifications—they can be valuable professional indicators and tools for learning and self-assessment, but they’re certainly no guarantee of a software developer’s abilities.

“There has never been a single, reliable way of measuring how good a programmer is,” Lee concludes. Yet clearly there is such a thing as a good programmer, just as there are mediocre and even bad programmers.

So, again: How do you know if you’re any good? It starts with recognizing the subjectivity of the craft. The problem with many of the approaches above, according to Lee, is that they’re incorrectly applied as objective measures of programming ability.

“Much to the surprise of many non-programmers, even though computers are precise instruments, programming is not a precise science,” Lee states. “It is highly subjective and requires evaluation and trade-offs.”

Given all that, though, there are ways to get at the issue of software quality and productivity. Reflecting on these six questions can help programmers—and the managers and organizations they work for—better understand what it means to be really good at software development:

Question 1: Do you truly “get” and embrace programming’s inherent subjectivity?

“There is never, ever a case where there is a single right answer—there is always a spectrum of rightness and wrongness,” Lee says. “Understanding this, accepting this, and applying this to your job, your team, and your product is perhaps the biggest thing a programmer can do to become an excellent programmer.”

award iconQuestion 2: Are you humble?

There’s something to be said for confidence, but just like education and other variables, confidence does not automatically equal quality. In fact, no matter how good you may be, some dissatisfaction or even doubt is often healthy.

Imposter syndrome often prevents us from being able to recognize our own accomplishments, and [good] programmers are definitely not immune,” Mac Browning says. But “in small amounts and kept in context, it can be a sign that as an engineer you're constantly pushing yourself to grow.”

Question 3: Do your customers use and enjoy the products you develop?

Highlighted by the growing importance of DevOps and full-stack engineers, it’s increasingly clear that good programmers need to care about what their code does even after it’s in production. You care about how well it’s working, how fast it’s performing, if users find it intuitive to learn and use, and if it’s really helping them in some way. Beyond that, do they actually enjoy using the service you built?

“If your customers [or] end users use your product every day and have an enjoyable experience, that’s the best measure of your skills as a developer,” says Kal Vissa.

math iconQuestion 4: Do you obsessively revisit the fundamentals?

Good programmers are sticklers for detail and constantly return to the fundamentals: “They take great pride in ensuring their code is unit tested, well documented, follows the DRY [don’t repeat yourself] principles, and prioritizes simplicity over complexity,” Kal says. Put another way, are you building modular and reusable code that helps solve problems and offers value beyond the original implementation?

Question 5: Do you care about “soft” qualities, too?

According to Lee Atchison, many of the most important qualities that set apart a good programmer from a weak one fall into the “soft skills” category. Lee looks for a developer’s ability and willingness to

  • Work in a team environment with a wide variety of people
  • Handle stress that comes from unpredictable directions and change
  • Work through issues and find solutions, rather than complain about issues and avoid them
  • Understand that elegance of design is important but does not necessarily equate with good programs

ruler iconQuestion 6: Do you measure the value your code creates?

“I think most successful engineers don’t measure their success by lines of code [written] or tickets fixed, but in value delivered,” Mac Browning says.

Value, like programming itself, can be highly subjective, but that shouldn’t prevent you from measuring it in the context of your individual and organizational goals. Mac suggests a few sample scenarios: a new feature in the product your customers use, tests to cover previously unknown edge cases, or new functionality in an open source project. Performance, usability, and other factors are also increasingly important.

“You should be able to measure some key indicator, such as revenue for your product or adoption of your new feature, which we can correlate with our contributions,” Mac says. “By pulling our success criteria up to a level higher than the nuts and bolts of writing code, it makes it easier to measure whether or not we were in fact successful.”

So how did you do?

You unwittingly just took a test, and an appropriately subjective one. If you answered “yes” to each of the questions above, you might just be good at programming. Don’t let it go to your head, though. Software development is a work-in-progress that requires regular re-evaluation. There are always trade-offs, as Lee says, and change is constant. The best developers never lose sight of that.