Migration Best Practices
When migrating on-premise estate into the Cloud, some practical steps are followed. The following is an overview of how to take a database, or application, into the Cloud.
Step 1: Destination Server Architecture
We need a runway to land a plane. The first step is to provision the receiving, target platform (Azure, AWS etc). Whether we have a simple shared hosting account, or a complex environment involving load balancers, firewalls, multiple web-heads, physical and cloud servers, we must first properly architect the destination server environment before starting your migration.
This target platform will consider and be relevant for the type of application(s) being migrated and our own unique business requirements.
Requirements for a destination server architecture are based mainly upon the following:
-elasticity, redundancy, availability, network bandwidth, compliance standards, security, backing up data, perhaps disaster recovery (which is different than High Availability)
To architect the system accurately, you need to understand the current estate and details of your servers including; size, load, IOPS, type of code, concurrent usage, bandwidth requirements, access and authentication by role and location. Thus before the migration begins both ends of the process are understood: our current estate and our future target platform.
Step 2: Perform the Initial Migration
The following steps are performed when doing a migration:
1) Prepare the application and database data, for transfer to the target environment. This usually involves a fully compressed backup of files/folders requiring migration, including making a fully copy of the production database (Virtual Machine copies). The target platform will typically have a ‘Development and Pre Production Environment’ and a ‘Production’ environment. They should be the same (same AMIs, instances, OSes, environments), but separated and secured by role and access keys. Once Dev-Pre Prod is tested and passed you simply copy the same machine images and stack into Production.
2) Copy data and files to the target server. To do this properly, you need to setup the application at the destination server environment as similarly as it existed at the source server environment. Use the same folder paths (to avoid potential issues with hard-coded paths), as well as the same database names and usernames. You can use database migration tools or native tools provided by the Cloud platform vendors (IaaS or PaaS), including for example DMS by AWS.
3) Restore files on the target server. Once the files and databases are copied to the destination server, restore them to the appropriate home directory folders and restore the database to the respective database server.
4) Configuration. The next step will be reconfiguring the application to run in the new environment. For this step, you’ll need to identify any/all application configuration related files. If file paths or database login credentials have changed, you’ll need to update those paths and connection strings in the application at the destination server environment.
5) Specific server settings. This will depend on the type, version, build and complexity of the application, and it may require specific server settings (web, database, file system permissions, etc.) and dependencies (libraries, services), which must be configured in accordance with the requirements of the application.
To help mitigate server settings it is helpful to have a thorough deployment document which details all the configuration requirements. If this is not available then using an Agile or ‘fast-fail’ process will be necessary to do trial and error testing.
6) Web application specifics. If the migration is moving to a platform using a load balancer, we will need to clone web-heads to the additional web servers or repeat the process for the web/application files to all of the additional web-heads you plan to have in rotation. As well, we need to set up file replication across the web-heads (rsync, lsync, glusterfs, etc.). If the web application is being load balanced for the first time, it needs to be load balancer-ready.
7) Multi-DB servers. If we are using multiple database servers, we will have to setup a master-master or master-slave configuration. Depending on the destination server architecture, this process can become quite complex and could involve some time and expert DBA-DB SME work.
Step 3: From Dev to Production
8) Testing. Typically done in Dev-Pre-Production and confirmed before copying the same environments and stacks to Production. Comprehensive testing is needed prior to going live. In Dev-Pre-Prod configure the destination server environment, assign an IP to the application, restore the database(s) and reconfigure the application. Automated script testing along with manual testing is necessary. Once tested we would repeat the process in Production. Many products both open source and paid, exist to help test the system in the cloud.
9) Go live. A quick checklist before going live and cutting over to the new production system:
–Lower TTL Values
At least 48 hours before the go-live event is scheduled, you’ll want to lower the Time To Live (TTL) values on any domains that are going to need to be pointed to the new server environment during the go-live event. In most cases you can lower these values to just 5 minutes (360 seconds), although some registrars require higher values. This will reduce the time of the DNS propagation. More information on TTL can be found here: http://en.wikipedia.org/wiki/Time_to_live
–Downtime
There are two options to consider depending on the impact a brief period of site unavailability may have on your business:
- a) If the application simply cannot be down (i.e. in a maintenance mode) even for a short period of time, your only option is setting up database replication and reverse proxy between servers. This can significantly increase the complexity, cost and project lifecycle for any migration.
- b) If a temporary ‘maintenance mode’ is acceptable, this is a pretty simple step-by-step process:
- place the application into a maintenance mode
- synchronize the data
- switch the DNS
-Database and File Synchronization
We need to take a final copy of the database from the source server environment and restore it at the destination server environment for any recent changes. Or, if we want to avoid this we can use a migration tool (such as DMS) which replicates the changes in the DB in real time, from the source to the destination.
In either case, we will still want to search the source server environment for any/all files (usually images or documents) that have been added during the course of the initial migration, testing and troubleshooting, and restore them, or make sure they are present, in the destination server environment. We can do this in Dev-Pre-Prod before copying over to Production.
At the conclusion of the database and file synchronization, it’s always wise to perform a quick test of the application at the destination server environment.
-DNS Changes
Once all files/databases have been synchronized and you’ve performed your quick test, you’ll want to login to your registrar or DNS service provider and change your A record(s) to point to the new server IP(s). Since the TTL value is 48 hours the propagation (time it takes for the DNS services around the world to reflect your IP change) should be quite fast, usually 2-4 hours or even less.
10) Migration support
Obviously, issues can arise and a team must be on hand to deal with problems immediately. It typically takes 10 business days for the new platform to be declared ‘stable’. At this time, you can consider future improvements to the stable system and reduce direct support headcount.
The above are the main details in migrating database and applications to the Cloud.