What is CloudFormation
AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you. You don’t need to individually create and configure AWS resources and figure out what’s dependent on what; AWS CloudFormation handles all of that. The following scenarios demonstrate how AWS CloudFormation can help.
Source: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html
Deploy an application, example Agile-Scrum-Maturity-Model – managing Agile-Scrum maturity across projects.
https://qa.agile-scrum-maturity-model.com/#/faq
ASMM app components overview
ASMM application components:
- RESTful services – Java
- MySQL database
- Angular client
Application is deployed on Tomcat 9 with Open Java 11.
Note that some resources were placed on AWS S3 and used in CloudFormation template:
S3 resources (cloudformation-london):
- app folder
- ROOT.zip – Tomcat deployment package
- asmm.dmp – MySQL dmp file with database structure and initial data
- tomcat.service – Tomcat service configuration file specific to ASMM app
2. resources folder
- apache-tomcat-9.0.36.tar.gz – Tomcat 9.0.36 installation package
- mysql-connector-java-5.1.39-bin.jar – MySQL JDBC driver
- openjdk-11+28_linux-x64_bin.tar.gz – Open JDK 11
3. template folder
- asmm-amazon-linux-2.json – JSON CloudFormation template
CloudFormation ASMM template overview
The template is in JSON format.
It performs the following:
1.Creates Role, Instance Profile and Policy to connect to S3 from EC2.
2. Creates Amazon Linux 2 EC2 instance:
- Creates instance
- Installs MariaDB server
- Configures MariaDB service to start on EC2 restart
- Secures MariaDB
- Creates asmm database structure and inserts data
- Installs OpenJDK 11
- Installs and configures Tomcat, also Tomcat service
- Sets up redirection of port 80 to 8080
- Deploys ASMM Company Admin application on Tomcat
3. Creates security group to allow HTTP and SSH access
4. Creates and assigns Elastic IP to EC2.
5. Displays template Output: URLs (domain name assigned to EC2 and Elastic IP).
ASMM JSON template structure and walk-through
Full JSON template file: https://cloudformation-london.s3.eu-west-2.amazonaws.com/templates/asmm-amazon-linux-2.json
Sections
1. AWSTemplateFormatVersion
Identifies the capabilities of the template. The latest template format version is 2010-09-09 and is currently the only valid value.
2. Description
Text description of the template.
3. Parameters
Parameters which needs to be entered when creating a stack via AWS CloudFormation browser based console.
Note: Default values were provided in the template for the parameters in order to speed up testing and stack generation process. In production environment, default values for sensitive parameters such as password should not be provided.
ASMM template parameters:
3.1. KeyName
Name of an existing EC2 KeyPair to enable SSH access to the instance. EC2 KeyPair must be created prior creating a stack using the template. It must be generated in the same AWS Region where stack is being created.
3.2. DBName
MySQL (MariaDB) database name for the application. The template provides a default value.
3.3. DBUser
Username for MySQL database access. The template provides a default value.
3.4.DBPassword
Password for MySQL database access. The template provides a default value.
3.5.DBRootPassword
Root password for MySQL. The template provides a default value.
3.6.InstanceType
Web Server EC2 instance type. The template provides a default value.
3.7.SSHLocation
The IP address range that can be used to SSH to the EC2 instances. The template provide a default value.
4. Mappings
The optional Mappings section matches a key to a corresponding set of named values. For example, if you want to set values based on a region, you can create a mapping that uses the region name as a key and contains the values you want to specify for each specific region. You use the Fn::FindInMap intrinsic function to retrieve values in a map.
Source: AWS
In ASMM template Mappings section is specified but not used. It is included to show how it can be used.
AMI Image IDs
In production environment Mapping section could be used to specify AMI Image IDs for different instance types and AWS Regions. Note that Image IDs can change overtime and Mapping need to be updated manually in the template whenever it happens. An alternative is creating AWS Lambda functions which look up ids.
ASMM template uses hard-coded static Image ID for Amazon Linux 2 in Europe (London) Region. Image ID can be found by using “Launch” button from AWS EC2 Dashboard. List of available images for the current AWS Region lists Image ID which can be used in CloudFormation template.
5. Resources
Resources section list all AWS resources (i.e. EC2, security groups, Elastic IPs etc.) to be created by the template.
ASMM template resources:
5.1. EC2toS3Role
Type: AWS::IAM::Role
Creates IAM Role to allow EC2 to S3 connection. ASMM templates downloads resources required to configure EC2 instance and deploy ASMM application from S3.
"EC2toS3Role": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/"
}
}
Note: “EC2toS3Role” is a logical name for the resource assigned by template creator.
Each resource has “Type” property which must specify a valid AWS CloudFormation resource type.
AWS Reference: AWS Resource Type reference
5.2. EC2toS3InstanceProfile
Type: AWS::IAM::InstanceProfile
Creates a new instance profile. Used for EC2 to S3 connection.
"EC2toS3InstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [
{
"Ref": "EC2toS3Role"
}
]
}
}
5.3. EC2toS3RolePolicies
Type: AWS::IAM::Role
Creates a new instance profile. Used for EC2 to S3 connection.
"EC2toS3RolePolicies": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "EC2toS3RolePolicy",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::cloudformation-london/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
},
"Roles": [
{
"Ref": "EC2toS3Role"
}
]
}
}
5.4. WebServerInstance
Type: AWS::EC2::Instance
Creates EC2 instance.
Note:
The template uses CloudFormation Helper Scripts (cfn) to install and configure components. CFN scripts are available by default on Amazon Linux AMIs. They may need to be installed on other AMIs (i.e. Centos, Windows).
Errors displayed in CloudFormation portal are limited. In order to view detailed error messages the best way is to make sure that EC2 instance is not deleted on template execution error.
In ASMM temporarily removed CreationPolicy section. CreationPolicy expects notification via cfn-signal that EC2 creation process is finished. In case any error was thrown, cfn-signal was not sent, and as a result, stack created by the template for rolled back. Another way is to set Rollback policy for template resources to false. As a result, created resources will not be deleted.
Detailed logs can be viewed via SSH console: /var/logs/cfn-init.log
“Install” section specifies packages to be installed and files to be created
“Configure” section specifies command to be run after EC2 is initiated. All commands under “Configure” section have descriptive name i.e. “01_db_enable_mariadb.service”.
“UserData” section allows for running custom script after EC2 is initialize. In case of ASMM template, “UserData” sections calls cfn helper scripts to execute “Install” and “Configure” sections and send signal to CloudFormation engine that the process has been finished.
“ImageId” – in ASMM, a static value for for t2.small in eu-west-2 (London) region.
"WebServerInstance": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"configSets": {
"InstallAndRun": [
"Install",
"Configure"
]
},
"Install": {
"packages": {
"yum": {
"mariadb-server": []
}
},
"files": {
"/tmp/setup.mysql": {
"content": {
"Fn::Join": [
"",
[
"CREATE DATABASE ",
{
"Ref": "DBName"
},
";\n",
"GRANT ALL ON ",
{
"Ref": "DBName"
},
".* TO '",
{
"Ref": "DBUser"
},
"'@localhost IDENTIFIED BY '",
{
"Ref": "DBPassword"
},
"';\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
},
"/tmp/secure.mysql": {
"content": {
"Fn::Join": [
"",
[
"DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1')",
";\n",
"DELETE FROM mysql.user WHERE User=''",
";\n",
"DELETE FROM mysql.db WHERE Db='test' OR Db='test_%'",
";\n",
"FLUSH PRIVILEGES",
";\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
},
"/etc/cfn/cfn-hup.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\n",
"stack=",
{
"Ref": "AWS::StackId"
},
"\n",
"region=",
{
"Ref": "AWS::Region"
},
"\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
},
"/etc/cfn/hooks.d/cfn-auto-reloader.conf": {
"content": {
"Fn::Join": [
"",
[
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n",
"action=/opt/aws/bin/cfn-init -v ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource WebServerInstance ",
" --configsets InstallAndRun ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n",
"runas=root\n"
]
]
},
"mode": "000400",
"owner": "root",
"group": "root"
}
},
"services": {
"sysvinit": {
"cfn-hup": {
"enabled": "true",
"ensureRunning": "true",
"files": [
"/etc/cfn/cfn-hup.conf",
"/etc/cfn/hooks.d/cfn-auto-reloader.conf"
]
}
}
}
},
"Configure": {
"commands": {
"01_db_enable_mariadb.service" : {
"command" : "systemctl enable mariadb.service"
},
"02_db_start_mariadb,service" : {
"command" : "systemctl start mariadb.service"
},
"03_db_set_mysql_root_password": {
"command": {
"Fn::Join": [
"",
[
"mysqladmin -u root password '",
{
"Ref": "DBRootPassword"
},
"'"
]
]
},
"test": {
"Fn::Join": [
"",
[
"$(mysql ",
{
"Ref": "DBName"
},
" -u root --password='",
{
"Ref": "DBRootPassword"
},
"' >/dev/null 2>&1 </dev/null); (( $? != 0 ))"
]
]
}
},
"04_db_secure_database": {
"command": {
"Fn::Join": [
"",
[
"mysql -u root --password='",
{
"Ref": "DBRootPassword"
},
"' < /tmp/secure.mysql"
]
]
}
},
"05_db_create_database": {
"command": {
"Fn::Join": [
"",
[
"mysql -u root --password='",
{
"Ref": "DBRootPassword"
},
"' < /tmp/setup.mysql"
]
]
},
"test": {
"Fn::Join": [
"",
[
"$(mysql ",
{
"Ref": "DBName"
},
" -u root --password='",
{
"Ref": "DBRootPassword"
},
"' >/dev/null 2>&1 </dev/null); (( $? != 0 ))"
]
]
}
},
"06_db_download_database_dmp" : {
"command" : "aws s3 cp s3://cloudformation-london/apps/asmm.dmp /tmp/"
},
"07_db_populate_tables_and_data" : {
"command": {
"Fn::Join": [
"",
[
"mysql -u ",
{
"Ref": "DBUser"
},
" -p",
{
"Ref": "DBPassword"
},
" < /tmp/asmm.dmp"
]
]
}
},
"08_java_create_directory" : {
"command" : "mkdir /usr/java"
},
"09_java_download" : {
"command" : "aws s3 cp s3://cloudformation-london/resources/openjdk-11+28_linux-x64_bin.tar.gz /usr/java",
"cwd" : "/usr/java"
},
"10_java_unpackage" : {
"command" : "tar xvf openjdk-11+28_linux-x64_bin.tar.gz",
"cwd" : "/usr/java"
},
"11_java_remove_tar" : {
"command" : "rm -f openjdk-11+28_linux-x64_bin.tar.gz",
"cwd" : "/usr/java"
},
"12_tc_create_tomcat_directory" : {
"command" : "mkdir /opt/tomcat"
},
"13_tc_create_tomcat_group" : {
"command" : "sudo groupadd tomcat"
},
"14_tc_create_tomcat_user" : {
"command" : "sudo useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat"
},
"15_tc_download" : {
"command" : "aws s3 cp s3://cloudformation-london/resources/apache-tomcat-9.0.36.tar.gz /opt"
},
"16_tc_unpackage" : {
"command" : "sudo tar xzvf apache-tomcat-9.0.36.tar.gz -C /opt/tomcat --strip-components=1",
"cwd" : "/opt"
},
"17_tc_remove_tar" : {
"command" : "rm -f /opt/apache-tomcat-9.0.36.tar.gz",
"cwd" : "/opt"
},
"18_tc_download_mysql_driver" : {
"command" : "aws s3 cp s3://cloudformation-london/resources/mysql-connector-java-5.1.39-bin.jar /opt/tomcat/lib"
},
"19_tc_update_permissions_1" : {
"command" : "sudo chgrp -R tomcat /opt/tomcat"
},
"20_tc_update_permissions_2" : {
"command" : "sudo chmod -R g+r conf",
"cwd" : "/opt/tomcat"
},
"21_tc_update_permissions_3" : {
"command" : "sudo chmod g+x conf",
"cwd" : "/opt/tomcat"
},
"22_tc_update_permissions_4" : {
"command" : "sudo chown -R tomcat webapps/ work/ temp/ logs/",
"cwd" : "/opt/tomcat"
},
"23_tc_create_service" : {
"command" : "aws s3 cp s3://cloudformation-london/resources/tomcat.service /etc/systemd/system"
},
"24_tc_reload_deamon" : {
"command" : "sudo systemctl daemon-reload"
},
"25_tc_redirect_80_to_8080" : {
"command" : "sudo iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-ports 8080"
},
"26_app_deploy_download" : {
"command" : "aws s3 cp s3://cloudformation-london/apps/ROOT.zip /opt/tomcat/webapps"
},
"27_app_deploy_remove_root" : {
"command" : "rm -rf ROOT/*",
"cwd" : "/opt/tomcat/webapps"
},
"28_app_deploy_unzip_package" : {
"command" : "unzip -o ROOT.zip",
"cwd" : "/opt/tomcat/webapps"
},
"29_app_deploy_adjust_permissions" : {
"command" : "chown -R root.tomcat ROOT",
"cwd" : "/opt/tomcat/webapps"
},
"30_app_deploy_adjust_permissions" : {
"command" : "chmod -R 755 ROOT",
"cwd" : "/opt/tomcat/webapps"
},
"31_app_deploy_remove_package" : {
"command" : "rm -f ROOT.zip",
"cwd" : "/opt/tomcat/webapps"
},
"32_tc_enable_service_on_startup" : {
"command" : "sudo systemctl enable tomcat.service"
},
"33_tc_start_service" : {
"command" : "sudo systemctl start tomcat.service"
}
}
}
}
},
"Properties": {
"ImageId": "ami-032598fcc7e9d1c7a",
"InstanceType": {
"Ref": "InstanceType"
},
"SecurityGroups": [
{
"Ref": "WebServerSecurityGroup"
}
],
"KeyName": {
"Ref": "KeyName"
},
"IamInstanceProfile": {
"Ref": "EC2toS3InstanceProfile"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -xe\n",
"yum update -y aws-cfn-bootstrap\n",
"# Install the files and packages from the metadata\n",
"/opt/aws/bin/cfn-init -v ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource WebServerInstance ",
" --configsets InstallAndRun ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n",
"# Signal the status from cfn-init\n",
"/opt/aws/bin/cfn-signal -e $? ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource WebServerInstance ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n"
]
]
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Timeout": "PT5M"
}
}
}
5.5. WebServerSecurityGroup
Type: AWS::EC2::SecurityGroup
Enables HTTP access via port 80 and SSH via port 22.
"WebServerSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Enable HTTP access via port 80",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
},
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": {
"Ref": "SSHLocation"
}
}
]
}
}
5.6. ElasticIPForInstance
Type: AWS::EC2::EIP
Specifies an Elastic IP (EIP) address and associates it with an EC2 instance.
"ElasticIPForInstance" : {
"Type" : "AWS::EC2::EIP",
"Properties" : {
"InstanceId" : { "Ref" : "WebServerInstance" }
}
}
6. Outputs
The optional Outputs section declares output values that you can import into other stacks (to create cross-stack references), return in response (to describe stack calls), or view on the AWS CloudFormation console.
Source: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html
ASMM template displays two URLs to access ASMM app:
- Access via public DNS name
- Elastic IP address
Note: Stack should be deleted after creation.