By now, you’ve certainly heard that “every company is in the software business.” Well, it’s not a throwaway statement anymore. Today, companies of all kinds—not just technology firms—must rapidly deliver high-quality software in order to survive. Consider that the average age of an S&P 500 company is now less than 20 years, down from 60 years in the 1950s. A key reason is that technology is disrupting businesses across a wide variety of industries. These days, you can’t buy a cup of coffee, go to the movies, or visit the doctor without encountering an app or website.
That translates to significant new workloads for technology teams. A 2018 survey of IT professionals found that, on average, IT professionals faced a 27% increase in project requests last year—most of them related to creating new apps or modifying existing ones. The resulting workloads are daunting: most IT professionals are charged to deliver 10 or more new apps in 2018. And in a dramatic warning sign, two-thirds admitted they were not able to deliver on all of the projects asked of them.
In response, enterprises are significantly stepping up their software development efforts, turning fast app development into a strategic business weapon by adopting techniques such as Agile development, DevOps, cloud computing, and—increasingly—continuous integration, continuous deployment, and continuous delivery, commonly referred to as CI/CD. This post explains how CI/CD has changed the landscape of modern software.
Software development now a strategic priority
Modern software companies that understand the importance of quickly delivering high-quality employee-, customer- and partner-facing applications have an edge. They know that by providing better user experiences while protecting sensitive data, they will acquire more customers, make employees more productive, and grow both revenues and profitability.
But they have to do this extraordinarily quickly and efficiently. This means constantly updating and shipping new versions of production code.
So what are they developing? Two thirds (66%) are looking at customers and partners as priorities, and put apps that serve them at the top of their to-do lists. But that’s only part of the story. Internal efficiencies and effectiveness are also at stake, with more than half of companies developing apps that support internal operations. These are business-critical (but difficult) goals, and developers desperately need help to achieve them. Enter CI/CD.
CI/CD: software development practices go mainstream
The upshot is that the arcane subject of software-development practices is no longer of interest just to software developers, but is becoming a front-and-center topic in strategic business decisions, including CI/CD. But because these terms—continuous integration, continuous delivery, and continuous deployment are so similar, it can be difficult to tell them apart.
Let’s try to unpack them a bit.
On the most basic level, continuous integration happens when developers frequently test any new code commits to a project’s main repository to ensure the new code is compatible with existing code. Continuous integration is also an integral part of the other two “continuous” methodologies. Continuous delivery means making sure your code is always ready to deploy, although you might wait before putting it into production—often for business reasons. Continuous deployment is really just continuous delivery taken one step further, with releases happening automatically, without requiring human intervention.
With those basics in mind, let’s look at the origins of each term, and explore how they help software teams deliver better applications faster.
As noted, continuous integration refers to developers maintaining strict control over code compatibility. In practical terms, it’s the process of automating the build and testing of code every time a developer makes changes. Developers share their code and unit tests by merging their changes into a shared version-control repository after they make even the smallest update. Then the automated builds and tests are run. This helps developers avoid release-day hell by finding errors early in the development process, when they’re relatively easy to fix.
The first mentions of continuous integration began appearing in the 1990s. IBM’s Grady Booch first named and proposed the methodology in his Booch method of software development, which he presents in the early editions of his book, Object-Oriented Analysis and Design. In 1996, Steve McConnell described the “Daily Build and Smoke Test” technique in the IEEE Journal, recommending what was then the “extreme” practice of testing code on a daily basis. When Extreme Programming emerged in the late 1990s, it adopted the concept of continuous integration and the notion of integrating even more frequently than once per day.
Then, in 2000, Martin Fowler wrote the cornerstone description of continuous integration practices of that period:
- Establish a single repository for the source code so that everyone can get their hands on both current and previous sources.
- Automate builds so that anyone can build the system from the sources at any time with one command.
- Automate testing so that you can run a test suite at any time with a single command.
- Ensure everyone has access to a current executable that is the best executable to date.
Continuous integration, a critical aspect of Agile
Then came the Agile programming methodology. Continuous integration eventually became a critical part of the Agile philosophy.
Agile was a reaction to the once-dominant “waterfall” approach to building software, which enforced a sequential model of establishing user needs and then building a single, finished version of the software to meet those needs. Unfortunately, customers’ needs often evolved while development was underway. By the time the software was delivered under the conventional waterfall process, customers often wanted something completely different. This wasted a lot of time and resources and resulted in plenty of failed software projects.
Realizing that waterfall was no longer working, developers in the 1990s began experimenting with other ways to build software. A number of new approaches were tried, including Scrum and Kanban. In 2001, the Manifesto for Agile Software Development gathered many of these practices under the label of Agile software development.
When using the Agile method, developers and other stakeholders collaborate to continuously iterate requirements and solutions, encouraging rapid and flexible response to change. With this iterative approach there is no such thing as a finished software product.
Continuous integration is core to the Agile methodology because it minimizes risk and enables Agile teams to work at a rapid yet sustainable pace. Just as important, each iteration delivers new value as well as the opportunity for continuous feedback to developers.
Continuous integration, however, takes enterprises only part of the way to quickly putting high-quality software into users’ hands. The next step in that path: continuous delivery.
Developers who practice continuous delivery produce code that is always deployable and ready to go into production. Continuous delivery is a collection of software development practices and methodologies that speed up time to market while improving quality.
According to precepts laid down by Martin Fowler, the key points of continuous delivery can be captured in a few key questions:
- Is your software always deployable throughout its lifecycle?
- Can your team prioritize and keep the software deployable while working on new features?
- Can anyone receive fast, automated feedback on the production-readiness of their applications and infrastructure each time someone makes a change to them?
- Can you perform push-button deployments of any version of the software to any environment on demand?
A critical point regarding continuous delivery (more on this later) is that while teams have software that is ready to deploy, they don’t necessarily deploy it immediately. They may hold on to it—often for business reasons—until they decide to release it to production. Because every code change is delivered to a staging environment using automation, they can deploy it at the touch of a button.
The concept of continuous delivery was a major breakthrough. As Kurt Bittner, principal analyst at Forrester Research, is widely quoted as saying, “If Agile was the opening act, continuous delivery is the headliner.” However, some consider this misleading, because it implies that continuous delivery came from Agile. It didn’t, although it is frequently used in conjunction with Agile to speed up code development.
If continuous delivery is not part of Agile, where did it come from? A natural extension of continuous integration, it developed in parallel to another important new IT philosophy: DevOps. Although separate from DevOps, continuous delivery helps make DevOps possible. To fully understand continuous delivery, we need to delve into DevOps.
DevOps and continuous delivery
If Agile was one response to Waterfall’s deficiencies, DevOps was created to address a related problem: the unnatural separation of software development (dev) and IT operations (ops) teams.
By pulling in stakeholders from across the organization and sharing responsibility for software performance and reliability, DevOps balances the need for flexibility in development with the rigor required when applying software solutions to important business tasks and processes—rigor about security, compliance, and compatibility:
As companies began using Agile and DevOps, they realized significant synergies. Just as continuous integration is critical to Agile, continuous delivery is essential for DevOps. Agile was born as a philosophy for developers, while DevOps, with continuous delivery, adds IT operations to the mix and is intended to improve communications across multiple elements of the business.
Going from continuous integration and continuous delivery to continuous deployment takes yet another step toward automating the software development lifecycle. By practicing continuous deployment, you ensure that every change is robust and clean enough that it can be immediately released to users—or customers—without any manual intervention.
In other words, if continuous delivery inserts a “pause” button into the process of delivering software to your users, continuous deployment shines a permanent green light. Some experts argue that continuous deployment should be the ultimate goal of every organization that doesn’t have compliance or other constraints for automating software release.
Because continuous deployment does away with human safeguards against defective code, teams should use it for frequent, small, incremental updates, as opposed to wholesale changes to large systems. In addition, developers must safely and scrupulously follow rigorous continuous integration and automated testing methodologies, including deploying instrumentation to provide visibility and avoid incidents when new code is released to users. Finally, you need to be able to back out from updates that cause users to experience errors or crashes not caught by the automated tests.
The benefit of continuous deployment is that development projects become more predictable and routine. New code is deployed quickly and smoothly, as soon as possible, even when hundreds of developers are making multiple changes every day.
The end result: continuous software development
Continuous integration, delivery, and deployment are collectively referred to as continuous software development. Associated with both Agile and DevOps, they all work together to allow businesses to leverage automation in order to develop, build, test, and deploy higher-quality code more rapidly. Just as important, they allow software organizations to understand and respond to market changes faster and more easily, and have been shown to correlate to business success.
Although CI/CD might not be right for every business, frequency of software deployment is often cited as the best single measure of a high-functioning software organization and is increasingly correlated with business success. The best, most productive modern software companies update their sites multiple times per day, and changes to production code take seconds rather than months or years.
Agile practices and continuous integration are like peanut butter and jelly. You’re not using Agile correctly unless continuous integration is in the picture. Likewise, to credibly institute DevOps you need continuous delivery, which incorporates continuous integration. Finally, if you rigorously apply both continuous integration and continuous delivery methods while fully automating deployment, you can achieve continuous deployment—the pinnacle of end-to-end automated software development and today’s best path to software and business success.