Steps to backup Amazon S3 bucket

May 16, 2023 actsupp-r0cks

backup Amazon S3 bucket Backing up your Amazon S3 bucket is an essential step to ensure the safety and security of your data. You can use the below steps to backup an Amazon S3 bucket.

Steps to Enable AWS S3 Versioning

Steps to Enable AWS S3 Versioning

  1. Sign in to the AWS Management Console.
  2. Click Services and select ‘S3’ from the storage category.
  3. Now click ‘Buckets’ in the navigation pane and select the S3 bucket you want to enable versioning. Then select the bucket name to open the bucket details (e.g., bucket 01)
  4. Next, open the ‘properties’ for the selected bucket and click ‘Edit’ in the bucket versioning.
  5. By default, the selected bucket version is disabled. Therefore, you can turn it on to enable bucket versioning.
  6. Save the changes.
  7. After completing the configuration changes, you will receive the message “Successfully edited bucket versioning”.

Lifecycle rules

Follow the below process to configure the lifecycle rules for Amazon s3 Versioning:

  • First, go to the Management tab from the selected bucket.
  • Click ‘Create Lifecycle rule’ in the Lifecycle rule.

Lifecycle Rule Configuration: 

  • Now enter the name in the lifecycle rule (e.g., Lifecycle 01)
  • You can choose any one scope.
  • Use filters to apply lifecycle rules to either particular objects or every object in the bucket.
  • Create an object tag for objects that need lifestyle activities. Then, click the Add or Delete tag buttons after entering the necessary information for the tag key and value.

Lifecycle Rule Actions: 

From the rule action, choose any of the processes you wish this rule to execute:

  • Transition current or previous versions of objects between storage classes.
  • Expire current versions of objects.
  • Permanently delete previous versions of objects.
  • Delete expired delete markers or incomplete multipart uploads.

Transition noncurrent versions of objects between storage classes:

Choose the storage class transitions and the number of days before objects become noncurrent.

Permanently delete previous versions of objects:

Set the number of days for the previous version to be deleted. Ensure the value entered is higher than the number of days before objects become noncurrent. To create a lifecycle rule, click ‘Create Rule’.

Bucket Replication

You can replicate the bucket between regions instead of using Amazon S3 automatic backup. First, you need to create a second bucket in another location that acts as the destination bucket, as well as create a replication rule. Every change made in the source bucket is immediately reflected in the destination after the replication rule is created.

  • In the sourced bucket, select the Replication rules from the Management tab and click Create ‘Replication rule’.
  • Once the Create Replication rule page open, enter the Replication rule name (e.g., AWS s3 replication)
  • Set the rule status as enabled or disabled when the rule is created.
  • Source bucket: The source bucket name has been selected (Lifecycle 01) already.
  • Now choose a rule scope: The replication rule can be applied to all objects in the bucket, or you may create filters and apply the rules to custom objects.
  • Destination: You can also choose the bucket from this account or another account. Enter the bucket name or select the bucket list by choosing the Browse S3 button. The designation region is based on the selected designation region.
  • The identity and access management (IAM) role should be configured before selecting a storage class and further replication parameters. To save the setup and create a replication rule for the bucket, click ‘Save’.

Backup in the AWS CLI

The command line interface in the AWS CLI are used to interact with different Amazon services like Amazon S3. You can back up Amazon S3 buckets to a Linux machine using the sync command, which copies files from the bucket to a local directory in Linux running on an EC2 instance. The feature of the ‘sync’ command in AWS CLI is important in AWS backup because in case if you accidentally delete some files in the S3 bucket, The file will not be deleted in the local directory of the Linux machine after synchronization.

Now let us see the command used for the Ubuntu server.

  • First, you want to install AWS CLI.
  • Now, update the tree of repositories:
sudo apt-get update
  • Install AWS CLI or Unzip:
sudo apt install awscli

or

sudo apt install unzip
curl"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscli-exe-linux-x86_64.zip
sudo./aws/install

  • Examine the AWS credentials in Linux on your EC2 instance.
aws configure list
  • If no credentials have set, add them to access AWS through AWS CLI from the Linux instance:
aws configure

Then, enter the following parameters: AWS Access Key ID, AWS Secret Access Key, Default region name, and Default output format.

  • Now create a directory to store Amazon S3 backup. Whereas the ~/S3/ directory is to store the backup file and the subdirectory will be the same as the bucket name & ‘~’ is the home directory. The files stored in the S3 bucket should be copied to the Linux machine local directory.
mkdir -p ~/s3/your_bucket_name

Note: Replace ‘your_bucket_name’ with the name created for your bucket.

mkdir -p ~/s3/ bucket 01
  • Synchronize the content in the bucket with your local directory on the Linux EC2 instance:
aws s3 sync s3:// bucket01 /home/ubuntu/s3/ bucket01/

The data will start downloading from the s3 bucket, if the credential configuration, bucket name, and destination path are accurate. The process duration will change based on the file size in the bucket and your network speed.

Automatic Amazon S3 backup

With AWS CLI sync, you can configure Amazon S3 automatic backup. To sync, create a sync.sh script file to execute AWS S3 backup and schedule the script to run.

nano /home/ubuntu/s3/sync.sh
#!/bin/sh
# Display the current date and time
echo '-----------------------------'
date
echo '-----------------------------'
echo ''
# Display the script initialization message
echo 'Syncing remote S3 bucket...'
# Running the sync command
/usr/bin/aws s3 sync s3://{BUCKET_NAME} /home/ubuntu/s3/{BUCKET_NAME}/
# Echo "Script execution is completed"
echo 'Sync complete'

Replace the BUCKET_NAME with the S3 bucket name you want to back up.

  • The full path is defined to ‘AWS CLI binaries’, to ensure that crontab executes the aws application properly in the shell environment.

Execute the script:

sudo chmod +x /home/ubuntu/s3/sync.sh

Check if the script works, and run it:

 /home/ubuntu/s3/sync.sh
  • To schedule Amazon S3 backup script execution edit the current user crontab.
crontab-e
  • The scheduling the task format is (m – minutes, h – hours, dom- day of the month, dow-day of the week):
m h dom mon dow command 
  • Add a line to the task configuration to perform synchronization every hour and save AWS S3 backup results to the log file. Include this line at the end of the crontab configuration.
0 * * * * /home/ubuntu/s3/sync.sh > /home/ubuntu/s3/sync.log

Automatic backup on Amazon S3 is enabled, and use the log file to verify the execution of the synchronization task.

If you need any direct assistance to backup an Amazon S3 bucket you can click here – Get Assistance

To get more updates you can follow us on Facebook, Twitter, LinkedIn

Subscribe to get free blog content to your Inbox

Written by actsupp-r0cks