The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. 2016. Authors: Gene Kim, Jez Humble, Patrick Debois, John Willis, and John Allspaw
An excellent book on DevOps which falls out of Agile-Scrum methodology. Some salient points made by the authors include the quotes below:
1 ) Myth—DevOps Replaces Agile: DevOps principles and practices are compatible with Agile, with many observing that DevOps is a logical continuation of the Agile journey that started in 2001. Agile often serves as an effective enabler of DevOps, because of its focus on small teams continually delivering high quality code to customers. Many DevOps practices emerge if we continue to manage our work beyond the goal of “potentially shippable code” at the end of each iteration, extending it to having our code always in a deployable state, with developers checking in to trunk daily, and that we demonstrate our features in production-like environments.
2 ) Why Agile and DevOps? – Productivity: This is highlighted in the famous book by Frederick P. Brooks, The Mythical Man-Month, where he explains that when projects are late adding more developers not only decreases individual developer productivity but also decreases overall productivity. (In other words, small teams, agile development, automation, use of SMEs will be the real enablers of project success).
3 ) Culture is King: DevOps shows us that when we have the right architecture, the right technical practices, and the right cultural norms, small teams of developers are able to quickly, safely, and independently develop, integrate, test, and deploy changes into production. As Randy Shoup, formerly a director of engineering at Google, observed, large organizations using DevOps “have thousands of developers, but their architecture and practices enable small teams to still be incredibly productive, as if they were a startup.”
4 ) DevOps builds for the market – not the Poliburo: Broadly speaking, to achieve DevOps outcomes, we need to reduce the effects of functional orientation (“optimizing for cost”) and enable market orientation (“optimizing for speed”) so we can have many small teams working safely and independently, quickly delivering value to the customer.
To achieve market orientation, we won’t do a large, top-down reorganization, which often creates large amounts of disruption, fear, and paralysis. Instead, we will embed the functional engineers and skills (e.g., Ops, QA, Infosec) into each service team, or provide their capabilities to teams through automated self-service platforms that provide production-like environments, initiate automated tests, or perform deployments.
5 ) DevOps means development and operations/support are on the same team: After the product is released to production, embedded Ops engineers may help with the production responsibilities of the Dev team. They will take part in all of the Dev team rituals, such as planning meetings, daily standups, and demonstrations where the team shows off new features and decides which ones to ship. As the need for Ops knowledge and capabilities decreases, Ops engineers may transition to different projects or engagements, following the general pattern that the composition within product.
6 ) Immutable infrastructure: Bill Baker, a distinguished engineer at Microsoft, quipped that we used to treat servers like pets: “You name them and when they get sick, you nurse them back to health. [Now] servers are [treated] like cattle. You number them and when they get sick, you shoot them.”
The latter pattern is what has become known as immutable infrastructure, where manual changes to the production environment are no longer allowed—the only way production changes can be made is to put the changes into version control and re-create the code and environments from scratch. By doing this, no variance is able to creep into production.
7 ) Automate code deployment: For instance, we may create a platform that provides a shared version control repository with pre-blessed security libraries, a deployment pipeline that automatically runs code quality and security scanning tools, which deploys our applications into known, good environments that already have production monitoring tools installed on them. Ideally, we make life so much easier for Dev teams that they will overwhelmingly decide that using our platform is the easiest, safest, and most secure means to get their applications into production.
8 ) Daily testing and automation leads to immutability: By having developers write, test, and run their own code in a production-like environment, the majority of the work to successfully integrate our code and environments happens during our daily work, instead of at the end of the release. By the end of our first interval, our application can be demonstrated to run correctly in a production-like environment, with the code and environment having been integrated together many times over, ideally with all the steps automated (no manual tinkering required).
9 ) DevOps at Google: By 2013, automated testing and continuous integration at Google enabled over four thousand small teams to work together and stay productive, all simultaneously developing, integrating, testing, and deploying their code into production. All their code is in a single, shared repository, made up of billions of files, all being continuously built and integrated, with 50% of their code being changed each month.
Some other impressive statistics on their performance include: 40,000 code commits/day 50,000 builds/day (on weekdays, this may exceed 90,000) 120,000 automated test suites 75 million test cases run daily 100+ engineers working on the test engineering, continuous integration, and release engineering tooling to increase developer productivity (making up 0.5% of the R&D workforce) In the remainder of this chapter, we will go through the continuous integration practices required to replicate these outcomes.
10 ) Automated and Tested Builds: Our deployment pipeline must create packages from version control that can be deployed to any environment, including production. Test: Anyone should be able to run any or all of our automated test suite on their workstation or on our test systems. Deploy: Anybody should be able to deploy these packages to any environment where they have access, executed by running scripts that are also checked in to version control. These are the practices that enable deployments to be performed successfully, regardless of who is performing the deployment.
11 ) Decouple the DB from the App: Decouple database changes from application changes: Instead of supporting two databases, we decouple the release of database changes from the release of application changes by doing two things: First, we make only additive changes to our database, we never mutate existing database objects, and second, we make no assumptions in our application about which database version will be in production. This is very different than how we’ve been traditionally trained to think about databases, where we avoid duplicating data. The process of decoupling database changes from application changes was used by IMVU (among others) around 2009, enabling them to do fifty deployments per day, some of which required database changes.
12 ) Telemetry — Pervasive production telemetry in both our code and production environments ensure that problems are detected and corrected quickly, confirming that everything is working as intended and customers are getting value from the software we create.
We also create pervasive telemetry so we can see how all our system components are operating in the production environment, so that we can quickly detect when they are not operating as expected. Telemetry also allows us to measure whether we are achieving our intended goals and, ideally, is radiated to the entire value stream so we can see how our actions affect other portions of the system as a whole.
13 ) Pay off your Technical Debt – by using DevOps: We will actively manage this technical debt by ensuring that we invest at least 20% of all Development and Operations cycles on refactoring, investing in automation work and architecture and non-functional requirements (NFRs, sometimes referred to as the “ilities”), such as maintainability, manageability, scalability, reliability, testability, deployability, and security.
The above are powerful, lucid and real world arguments why DevOps – which is a part of Agile – is mandatory.
==END