Magic DevOps overview on AWS

Abstract

DevSecOps on AWS can use many patterns, based on key principles.  This flexibility can create confusion with clients.  A common question is something like: ‘what is the best way to enable a CI-CD, DevSecOps pipeline, that we can understand and manage’?  A problem with AWS is its very flexibility leads to complexity, cost issues and governance problems.

This whitepaper outlines the key features and benefits of building a continuous integration, continuous delivery (CI/CD) pipeline as part of the DevSecOps process.  This whitepaper assumes that the firm in question is using Agile-Scrum properly and has already enable its Agile teams with proper engineering processes, tools, standards and most likely, the use of a Centre of Excellence to enforce compliance and monitor metrics.

Figure: AWS Pipeline using Kubernetes

Magic DevSecOps and Software Delivery on AWS

DevSecOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity, securely, minimizing vulnerabilities, and increasing quality.

Using DevSecOps principles, organizations can develop and improve products at a faster pace than organizations that use traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market. 

CI/CD is platform specific (and tied to the operating system).  It is the key to delivering application and digital software features rapidly and reliably.

AWS now offers a full range of DevOps and CI/CD capabilities as a set of developer services.  The table below is an overview of the many existing CI-CD services and related offerings in AWS.

CI-CD relatedOverview
AWS CodeCommitManaged Git-based source code repository and version control repository, for binaries, code, and documents.
AWS CodePipelineFully managed service which automates the build, test, and release processes within your code pipeline
AWS CodeBuildFully managed Continuous Integration service which automates the compiling, building, testing, and releasing of code into a delivery pipeline
AWS CodeDeployAutomates code deployments to any instance (EC2 or on-premise), across all environments (dev, test, prod)
AWS ArtifactFully managed artifact service (binary repository), to store, publish, share software packages and dependencies, can integrate with common package managers such as Maven
AWS CodeStarManaged service which integrates the Continuous Integration with Continuous Delivery by providing a unified template based pipeline using existing services such as Code Commit, CodeBuild, CodePipeline, Cloud9
AWS XrayUsed mainly with micro-services, deploys a daemon on the application which analyses how the application is used (data, services) and provides insights into improvements
Cloud FormationInfrastructure as Code (IaC) using JSON or YAML.  Use them.  AWS has many templates on github to help a client get started
AWS LambdaManaged serverless code deployment, can be used to run functions, in event-driven architectures, deploy applications, alerts and other state-change cycles
AWS System ManagerAWS Linux OS only – this is the major drawback.  Any other Linux OS and you will need to develop a security-patch-update runbook and model (a detailed example can be provided)
AWS XrayUsed mainly with micro-services, deploys a daemon on the application which analyses how the application is used (data, services) and provides insights into improvements
AWS AppConfigPay for use service, allows the user to validate an application’s configuration data against a JSON/YAML schema or Lambda function to ensure syntax and semantic correctness (part of System Manager)
AWS ConfigService which continuously monitors and audits the deployed assets against the configuration schema and provides alerts (SNS against state changes) and recommendations
AWS Cloud 9Browser based IDE to build, run, debug, test code in lieu of using a local IDE client
AWS AmplifyRapidly deploy React JS or Angular JS code and applications with a backend.  This complete stack significantly decreases deployment complexity, allows for IaC, and automates connecting the front-end of the application (UI-Presentation) with the Backend via CFT (Cloud Formation Templates)
ElasticBeanstalkCFT based service which provides an end-to-end-pipeline for the deployment of Web Sites and applications built in Java, Ruby, Node.js, Python, PHP, Docker and Go
AWS LightSailManaged service which allows simpler web, application deployments and provides the underlying infrastructure
Cloud Watch, Cloud Trail, VPC logsStandard services, can be customized, metrics provided on the application and related infra, along with API, network traffic

Magic DevSecOps Caveats and the Real Word

Figure: What is DevSecOps

DevSecOps is tightly integrated with Agile Teams and Engineering processes, and a defined Software Development Life Cycle (SDLC) process. Quite often neither of these concepts are well understood within firms.  Agile-DevSecOps entails cross-functional teams (Dev, Operations, Security, Testing, Business) and drives cultural, organizational, tooling, financial budgeting, and business development changes.

Figure: DevSecOps value stream

A screenshot of a cell phone

Description automatically generated

(See Docker on AWS: Running Containers in the Cloud)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.