Amazon S3 replication
Use Cases: Data immutability to Data compliance requirements, minimizing latency, or increasing operational efficiency.
Data Immutability: Data cannot be changed. It is related to but different than encryption.
Replication enables automatic and asynchronous copying of objects across Amazon S3 buckets, owned by the same AWS region/account or by different regions/accounts, and to a single or multiple destination bucket or buckets.
AWS S3
To create a replication rule, go to the Amazon S3 console, and select the S3 bucket (note that replication requires versioning to be enabled for both the source and destination buckets). Under the “Properties” tab of the S3 bucket, edit the “Bucket Versioning” field.
- Next, select “Enable” and save the changes.
- Next, to create bucket replication rules, under the “Management” tab, go to “Replication rules” and click “Create replication rule.”
- Provide a replication rule name and select a status for the rule during creation. Please note that for the batch job to work (we discuss it later in this article) the status has to be enabled first.
One can choose to apply the rule to all objects or filter objects by prefix, object tags, or a combination of both. In this example, let’s apply it to all objects in the bucket.
For the destination, you can choose the same or different AWS accounts. Ideally, especially for sensitive logs, a separate AWS account is recommended — in case the owner’s AWS account is compromised, you will have a backup copy in a safe location. It also allows the destination AWS account to be locked in security and compliance and is less susceptible to tampering in the same way the source account might be. It’s always a best practice to segregate AWS users or AWS IAM role access levels for the source and destination AWS accounts.
- Can use the same account but a bucket in a different region in this example.
- You will need to provide an IAM role. You can also simply select to “Create new role” automatically.
- Additionally, you have options to change the destination storage class and replicate objects encrypted with AWS KMS, if needed. Let’s use default options. Let’s also enable RTC as well as replication metrics and notifications and click “Save.”
You can enable a one-time batch operations job from the replication configuration to replicate objects that already exist in the bucket and synchronize the source and destination buckets. Let’s select the option for this example. If you do not select the option, only new objects will be replicated.
- To do this you will have to create a batch job, like in the example below, and click “Save.”
- Monitor the progress of the job under the “Batch Operations” tab in the Amazon S3 console (note that if you haven’t enabled replication from the Amazon S3 bucket’s “Management” tab yet, the job will fail).
When the batch job is ready, you will observe the status as “Awaiting your confirmation.” You will need to click “Run job” for the job to trigger. The status will then change to “Ready” and then “Active” during the progress, before “Completing” and finally “Completed.”
Download the completion report and verify all jobs:
Summary
Amazon S3 cross-region/account replication feature is useful for achieving data immutability to meet compliance requirements, minimize latency, or increase operational efficiency. In the next and final part of the series, we will discuss some additional security measures for your Amazon S3 data.