masudhan / Migrate_From_DO_Spaces_To_AWS_S3

Migrate from Digital Ocean spaces to AWS S3 using rclone

Migrate_from_DO_Spaces_to_AWS_S3

Tools we need

1. Install rclone
   
   • For macOS, brew install rclone
     
   • For windows, https://rclone.org/downloads/
     
   • For other linux distributions, curl https://rclone.org/install.sh | sudo bash
     
2. AWS CLI - https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
3. Stable internet connection

Setup

1. Generate New Key - https://cloud.digitalocean.com/account/api/tokens (Don't Generate Token it's Key)
2. For your IAM account, better to have S3 full access and generate aws_access_key_id, aws_secret_access_key and aws_session_token (for organisations we'll have this)
   If you want to migrate 1000GB's of data then aws_session_token should definetly set to 12 hours, that can be done from Session Duration
3. Update the rclone.conf file accordinly and store it in
   • For linux distributions, ~/.config/rclone/rclone.conf
   • For Windows, C:\Users\<username>\AppData\Roaming\rclone\rclone.conf

How to run

rclone sync spaces:<DO Space Name> s3:<AWS S3 Name> --ignore-existing -Pv

Abbrevations

• DO - Digital Ocean
• AWS - Amazon Web Services