AWS Cloud Formation Template example (deploying an app)

What is CloudFormation

AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you. You don’t need to individually create and configure AWS resources and figure out what’s dependent on what; AWS CloudFormation handles all of that. The following scenarios demonstrate how AWS CloudFormation can help.

Source: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html

Deploy an application, example Agile-Scrum-Maturity-Model – managing Agile-Scrum maturity across projects.

https://qa.agile-scrum-maturity-model.com/#/faq

ASMM app components overview

ASMM application components:

  1. RESTful services – Java
  2. MySQL database
  3. Angular client

Application is deployed on Tomcat 9 with Open Java 11.

Note that some resources were placed on AWS S3 and used in CloudFormation template:

S3 resources (cloudformation-london):

  1. app folder
  2. ROOT.zip – Tomcat deployment package
  3. asmm.dmp  – MySQL dmp file with database structure and initial data
  4. tomcat.service – Tomcat service configuration file specific to ASMM app

2. resources folder

  • apache-tomcat-9.0.36.tar.gz – Tomcat 9.0.36 installation package
  • mysql-connector-java-5.1.39-bin.jar – MySQL JDBC driver
  • openjdk-11+28_linux-x64_bin.tar.gz – Open JDK 11

3. template folder

  • asmm-amazon-linux-2.json – JSON CloudFormation template

CloudFormation ASMM template overview

The template is in JSON format.

It performs the following:

1.Creates Role, Instance Profile and Policy to connect to S3 from EC2.

2. Creates Amazon Linux 2 EC2 instance:

  • Creates instance
  • Installs MariaDB server
  • Configures MariaDB service to start on EC2 restart
  • Secures MariaDB
  • Creates asmm database structure and inserts data
  • Installs OpenJDK 11
  • Installs and configures Tomcat, also Tomcat service
  • Sets up redirection of port 80 to 8080
  • Deploys ASMM Company Admin application on Tomcat

3. Creates security group to allow HTTP and SSH access

4. Creates and assigns Elastic IP to EC2.

5. Displays template Output: URLs (domain name assigned to EC2 and Elastic IP).

ASMM JSON template structure and walk-through

Full JSON template file: https://cloudformation-london.s3.eu-west-2.amazonaws.com/templates/asmm-amazon-linux-2.json

Sections

1.      AWSTemplateFormatVersion

Identifies the capabilities of the template. The latest template format version is 2010-09-09 and is currently the only valid value.

2.      Description

Text description of the template.

3.      Parameters

Parameters which needs to be entered when creating a stack via AWS CloudFormation browser based console.

Note: Default values were provided in the template for the parameters in order to speed up testing and stack generation process. In production environment, default values for sensitive parameters such as password should not be provided.

ASMM template parameters:

3.1. KeyName

Name of an existing EC2 KeyPair to enable SSH access to the instance. EC2 KeyPair must be created prior creating a stack using the template. It must be generated in the same AWS Region where stack is being created.

3.2. DBName

MySQL (MariaDB) database name for the application. The template provides a default value.

3.3. DBUser

Username for MySQL database access. The template provides a default value.

3.4.DBPassword

Password for MySQL database access. The template provides a default value.

3.5.DBRootPassword

Root password for MySQL. The template provides a default value.

3.6.InstanceType

Web Server EC2 instance type. The template provides a default value.

3.7.SSHLocation

The IP address range that can be used to SSH to the EC2 instances. The template provide a default value.

 

4.      Mappings

The optional Mappings section matches a key to a corresponding set of named values. For example, if you want to set values based on a region, you can create a mapping that uses the region name as a key and contains the values you want to specify for each specific region. You use the Fn::FindInMap intrinsic function to retrieve values in a map.

Source: AWS

In ASMM template Mappings section is specified but not used. It is included to show how it can be used.

AMI Image IDs

In production environment Mapping section could be used to specify AMI Image IDs for different instance types and AWS Regions.  Note that Image IDs can change overtime and Mapping need to be updated manually in the template whenever it happens. An alternative is creating AWS Lambda functions which look up ids.

Info on it: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-custom-resources-lambda-lookup-amiids.html

ASMM template uses hard-coded static Image ID for Amazon Linux 2 in Europe (London) Region. Image ID can be found by using “Launch” button from AWS EC2 Dashboard. List of available images for the current AWS Region lists Image ID which can be used in CloudFormation template.

5. Resources

Resources section list all AWS resources (i.e. EC2, security groups, Elastic IPs etc.) to be created by the template.

ASMM template resources:

5.1. EC2toS3Role

Type: AWS::IAM::Role

Creates IAM Role to allow EC2 to S3 connection. ASMM templates downloads resources required to configure EC2 instance and deploy ASMM application from S3.

"EC2toS3Role": {
  "Type": "AWS::IAM::Role",
  "Properties": {
    "AssumeRolePolicyDocument": {
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": [
              "ec2.amazonaws.com"
            ]
          },
          "Action": [
            "sts:AssumeRole"
          ]
        }
      ]
    },
    "Path": "/"
  }
}

Note: “EC2toS3Role” is a logical name for the resource assigned by template creator.

Each resource has “Type” property which must specify a valid AWS CloudFormation resource type.

AWS Reference: AWS Resource Type reference

5.2. EC2toS3InstanceProfile

Type: AWS::IAM::InstanceProfile

Creates a new instance profile. Used for EC2  to S3 connection.

"EC2toS3InstanceProfile": {
  "Type": "AWS::IAM::InstanceProfile",
  "Properties": {
    "Path": "/",
    "Roles": [
      {
        "Ref": "EC2toS3Role"
      }
    ]
  }
}

5.3. EC2toS3RolePolicies

Type: AWS::IAM::Role

Creates a new instance profile. Used for EC2  to S3 connection.

"EC2toS3RolePolicies": {
  "Type": "AWS::IAM::Policy",
  "Properties": {
    "PolicyName": "EC2toS3RolePolicy",
    "PolicyDocument": {
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "s3:GetObject"
          ],
          "Resource": [
            "arn:aws:s3:::cloudformation-london/*"
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
            "s3:ListBucket"
          ],
          "Resource": [
            "arn:aws:s3:::*"
          ]
        }
      ]
    },
    "Roles": [
      {
        "Ref": "EC2toS3Role"
      }
    ]
  }
}

5.4. WebServerInstance

Type: AWS::EC2::Instance

Creates EC2 instance.

Note:

The template uses CloudFormation Helper Scripts (cfn) to install and configure components. CFN scripts are available by default on Amazon Linux AMIs. They may need to be installed on other AMIs (i.e. Centos, Windows).

Errors displayed in CloudFormation portal are limited. In order to view detailed error messages the best way is to make sure that EC2 instance is not deleted on template execution error.

In ASMM temporarily removed CreationPolicy section. CreationPolicy expects notification via cfn-signal that EC2 creation process is finished. In case any error was thrown, cfn-signal was not sent, and as a result, stack created by the template for rolled back. Another way is to set Rollback policy for template resources to false. As a result, created resources will not be deleted.

Detailed logs can be viewed via SSH console: /var/logs/cfn-init.log

“Install” section specifies packages to be installed and files to be created

“Configure” section specifies command to be run after EC2 is initiated. All commands under “Configure” section have descriptive name i.e. “01_db_enable_mariadb.service”.

“UserData” section allows for running custom script after EC2 is initialize. In case of ASMM template, “UserData” sections calls cfn helper scripts to execute “Install” and “Configure” sections and send signal to CloudFormation engine that the process has been finished.

“ImageId” – in ASMM, a static value for for t2.small in eu-west-2 (London) region.

"WebServerInstance": {
  "Type": "AWS::EC2::Instance",
  "Metadata": {
    "AWS::CloudFormation::Init": {
      "configSets": {
        "InstallAndRun": [
          "Install",
          "Configure"
        ]
      },
      "Install": {
        "packages": {
          "yum": {
            "mariadb-server": []
          }
        },
        "files": {
          "/tmp/setup.mysql": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "CREATE DATABASE ",
                  {
                    "Ref": "DBName"
                  },
                  ";\n",
                  "GRANT ALL ON ",
                  {
                    "Ref": "DBName"
                  },
                  ".* TO '",
                  {
                    "Ref": "DBUser"
                  },
                  "'@localhost IDENTIFIED BY '",
                  {
                    "Ref": "DBPassword"
                  },
                  "';\n"
                ]
              ]
            },
            "mode": "000400",
            "owner": "root",
            "group": "root"
          },
          "/tmp/secure.mysql": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1')",
                  ";\n",
                  "DELETE FROM mysql.user WHERE User=''",
                  ";\n",
                  "DELETE FROM mysql.db WHERE Db='test' OR Db='test_%'",
                  ";\n",
                  "FLUSH PRIVILEGES",
                  ";\n"
                ]
              ]
            },
            "mode": "000400",
            "owner": "root",
            "group": "root"
          },
          "/etc/cfn/cfn-hup.conf": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "[main]\n",
                  "stack=",
                  {
                    "Ref": "AWS::StackId"
                  },
                  "\n",
                  "region=",
                  {
                    "Ref": "AWS::Region"
                  },
                  "\n"
                ]
              ]
            },
            "mode": "000400",
            "owner": "root",
            "group": "root"
          },
          "/etc/cfn/hooks.d/cfn-auto-reloader.conf": {
            "content": {
              "Fn::Join": [
                "",
                [
                  "[cfn-auto-reloader-hook]\n",
                  "triggers=post.update\n",
                  "path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n",
                  "action=/opt/aws/bin/cfn-init -v ",
                  "         --stack ",
                  {
                    "Ref": "AWS::StackName"
                  },
                  "         --resource WebServerInstance ",
                  "         --configsets InstallAndRun ",
                  "         --region ",
                  {
                    "Ref": "AWS::Region"
                  },
                  "\n",
                  "runas=root\n"
                ]
              ]
            },
            "mode": "000400",
            "owner": "root",
            "group": "root"
          }
        },
        "services": {
          "sysvinit": {
            "cfn-hup": {
              "enabled": "true",
              "ensureRunning": "true",
              "files": [
                "/etc/cfn/cfn-hup.conf",
                "/etc/cfn/hooks.d/cfn-auto-reloader.conf"
              ]
            }
          }
        }
      },
      "Configure": {
        "commands": {
          "01_db_enable_mariadb.service" : {
            "command" : "systemctl enable mariadb.service"
          },
          "02_db_start_mariadb,service" : {
            "command" : "systemctl start mariadb.service"
          },
          "03_db_set_mysql_root_password": {
            "command": {
              "Fn::Join": [
                "",
                [
                  "mysqladmin -u root password '",
                  {
                    "Ref": "DBRootPassword"
                  },
                  "'"
                ]
              ]
            },
            "test": {
              "Fn::Join": [
                "",
                [
                  "$(mysql ",
                  {
                    "Ref": "DBName"
                  },
                  " -u root --password='",
                  {
                    "Ref": "DBRootPassword"
                  },
                  "' >/dev/null 2>&1 </dev/null); (( $? != 0 ))"
                ]
              ]
            }
          },
          "04_db_secure_database": {
            "command": {
              "Fn::Join": [
                "",
                [
                  "mysql -u root --password='",
                  {
                    "Ref": "DBRootPassword"
                  },
                  "' < /tmp/secure.mysql"
                ]
              ]
            }
          },
          "05_db_create_database": {
            "command": {
              "Fn::Join": [
                "",
                [
                  "mysql -u root --password='",
                  {
                    "Ref": "DBRootPassword"
                  },
                  "' < /tmp/setup.mysql"
                ]
              ]
            },
            "test": {
              "Fn::Join": [
                "",
                [
                  "$(mysql ",
                  {
                    "Ref": "DBName"
                  },
                  " -u root --password='",
                  {
                    "Ref": "DBRootPassword"
                  },
                  "' >/dev/null 2>&1 </dev/null); (( $? != 0 ))"
                ]
              ]
            }
          },
          "06_db_download_database_dmp" : {
            "command" : "aws s3 cp s3://cloudformation-london/apps/asmm.dmp /tmp/"
          },
          "07_db_populate_tables_and_data" : {
            "command": {
              "Fn::Join": [
                "",
                [
                  "mysql -u ",
                  {
                    "Ref": "DBUser"
                  },
                  " -p",
                  {
                    "Ref": "DBPassword"
                  },
                  " < /tmp/asmm.dmp"
                ]
              ]
            }
          },
          "08_java_create_directory" : {
            "command" : "mkdir /usr/java"
          },
          "09_java_download" : {
            "command" : "aws s3 cp s3://cloudformation-london/resources/openjdk-11+28_linux-x64_bin.tar.gz /usr/java",
            "cwd" : "/usr/java"
          },
          "10_java_unpackage" : {
            "command" : "tar xvf openjdk-11+28_linux-x64_bin.tar.gz",
            "cwd" : "/usr/java"
          },
          "11_java_remove_tar" : {
            "command" : "rm -f openjdk-11+28_linux-x64_bin.tar.gz",
            "cwd" : "/usr/java"
          },
          "12_tc_create_tomcat_directory" : {
            "command" : "mkdir /opt/tomcat"
          },
          "13_tc_create_tomcat_group" : {
            "command" : "sudo groupadd tomcat"
          },
          "14_tc_create_tomcat_user" : {
            "command" : "sudo useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat"
          },
          "15_tc_download" : {
            "command" : "aws s3 cp s3://cloudformation-london/resources/apache-tomcat-9.0.36.tar.gz /opt"
          },
          "16_tc_unpackage" : {
            "command" : "sudo tar xzvf apache-tomcat-9.0.36.tar.gz -C /opt/tomcat --strip-components=1",
            "cwd" : "/opt"
          },
          "17_tc_remove_tar" : {
            "command" : "rm -f /opt/apache-tomcat-9.0.36.tar.gz",
            "cwd" : "/opt"
          },
          "18_tc_download_mysql_driver" : {
            "command" : "aws s3 cp s3://cloudformation-london/resources/mysql-connector-java-5.1.39-bin.jar /opt/tomcat/lib"
          },
          "19_tc_update_permissions_1" : {
            "command" : "sudo chgrp -R tomcat /opt/tomcat"
          },
          "20_tc_update_permissions_2" : {
            "command" : "sudo chmod -R g+r conf",
            "cwd" : "/opt/tomcat"
          },
          "21_tc_update_permissions_3" : {
            "command" : "sudo chmod g+x conf",
            "cwd" : "/opt/tomcat"
          },
          "22_tc_update_permissions_4" : {
            "command" : "sudo chown -R tomcat webapps/ work/ temp/ logs/",
            "cwd" : "/opt/tomcat"
          },
          "23_tc_create_service" : {
            "command" : "aws s3 cp s3://cloudformation-london/resources/tomcat.service /etc/systemd/system"
          },
          "24_tc_reload_deamon" : {
            "command" : "sudo systemctl daemon-reload"
          },
          "25_tc_redirect_80_to_8080" : {
            "command" : "sudo iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-ports 8080"
          },
          "26_app_deploy_download" : {
            "command" : "aws s3 cp s3://cloudformation-london/apps/ROOT.zip /opt/tomcat/webapps"
          },
          "27_app_deploy_remove_root" : {
            "command" : "rm -rf ROOT/*",
            "cwd" : "/opt/tomcat/webapps"
          },
          "28_app_deploy_unzip_package" : {
            "command" : "unzip -o ROOT.zip",
            "cwd" : "/opt/tomcat/webapps"
          },
          "29_app_deploy_adjust_permissions" : {
            "command" : "chown -R root.tomcat ROOT",
            "cwd" : "/opt/tomcat/webapps"
          },
          "30_app_deploy_adjust_permissions" : {
            "command" : "chmod -R 755 ROOT",
            "cwd" : "/opt/tomcat/webapps"
          },
          "31_app_deploy_remove_package" : {
            "command" : "rm -f ROOT.zip",
            "cwd" : "/opt/tomcat/webapps"
          },
          "32_tc_enable_service_on_startup" : {
            "command" : "sudo systemctl enable tomcat.service"
          },
          "33_tc_start_service" : {
            "command" : "sudo systemctl start tomcat.service"
          }
        }
      }
    }
  },
  "Properties": {
    "ImageId": "ami-032598fcc7e9d1c7a",
    "InstanceType": {
      "Ref": "InstanceType"
    },
    "SecurityGroups": [
      {
        "Ref": "WebServerSecurityGroup"
      }
    ],
    "KeyName": {
      "Ref": "KeyName"
    },
    "IamInstanceProfile": {
      "Ref": "EC2toS3InstanceProfile"
    },
    "UserData": {
      "Fn::Base64": {
        "Fn::Join": [
          "",
          [
            "#!/bin/bash -xe\n",
            "yum update -y aws-cfn-bootstrap\n",
            "# Install the files and packages from the metadata\n",
            "/opt/aws/bin/cfn-init -v ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource WebServerInstance ",
            "         --configsets InstallAndRun ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n",
 
            "# Signal the status from cfn-init\n",
            "/opt/aws/bin/cfn-signal -e $? ",
            "         --stack ",
            {
              "Ref": "AWS::StackName"
            },
            "         --resource WebServerInstance ",
            "         --region ",
            {
              "Ref": "AWS::Region"
            },
            "\n"
          ]
        ]
      }
    }
  },
  "CreationPolicy": {
    "ResourceSignal": {
      "Timeout": "PT5M"
    }
  }
}
 
 
 
5.5.             WebServerSecurityGroup
                   Type: AWS::EC2::SecurityGroup 
                   Enables HTTP access via port 80 and SSH via port 22.
"WebServerSecurityGroup": {
  "Type": "AWS::EC2::SecurityGroup",
  "Properties": {
    "GroupDescription": "Enable HTTP access via port 80",
    "SecurityGroupIngress": [
      {
        "IpProtocol": "tcp",
        "FromPort": "80",
        "ToPort": "80",
        "CidrIp": "0.0.0.0/0"
      },
      {
        "IpProtocol": "tcp",
        "FromPort": "22",
        "ToPort": "22",
        "CidrIp": {
          "Ref": "SSHLocation"
        }
      }
    ]
  }
}

5.6. ElasticIPForInstance

Type: AWS::EC2::EIP

Specifies an Elastic IP (EIP) address and associates it with an EC2 instance.

"ElasticIPForInstance" : {
  "Type" : "AWS::EC2::EIP",
  "Properties" : {
    "InstanceId" : { "Ref" : "WebServerInstance" }
  }
}

6. Outputs

The optional Outputs section declares output values that you can import into other stacks (to create cross-stack references), return in response (to describe stack calls), or view on the AWS CloudFormation console.

Source: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html

ASMM template displays two URLs to access ASMM app:

  • Access via public DNS name
  • Elastic IP address

Note: Stack should be deleted after creation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.