Daily Droplet Backups to DigitalOcean Spaces

In this article I explain how to setup daily backups of Droplets to Spaces at DigitalOcean.

More than Weekly

It should be noted that DigitalOcean provides a weekly backup system for Droplets. You may click a checkbox and get backups for the price of +20% of the droplet.

Personally I want to backup daily. With that in mind I came up with this backup system that backs up to DigitalOcean spaces.

There is however nothing wrong with redundancy. I’ll admit I actually use both. Meaning I use the weekly backups provided by DigitalOcean as well as my daily backup solution.

Cheaper than Volumes

It should be noted that DigitalOcean spaces are cheaper than volumes.

At the time of writing you get 250GB of spaces storage for 5$ monthly. A volume of that size is 25$. If we look at a 1TB volume that would cost 100$ monthly. The same storage in Spaces would cost 20$.

It would seem the price is 1/5.

Choosing the Right Data Center

Spaces are not yet available in all data centers. I had a droplet in London (LON1) that I moved to Amsterdam (AMS3) since Spaces were not yet available in London. If you want to move your droplet you can follow this guide.

While it’s probably not required to have the Spaces in the same datacenter as the Droplet I would imagine it improves backup performance a bit.

An Important Security Note

When you create the Spaces it’s important you use the default “Restrict File Listing”. You wouldn’t want people having web access to your backups and all sensitive configuration files therein would you?

Installing RClone

For a previous backup system I have used rsync. However rsync won’t work this time since it does not support Object storage (such as Spaces).

There are multiple projects out there that could work as alternatives. I finally decided to go for rclone. I reasoned it had the most stars on GitHub and the best looking documentation.

You should install RClone following the guide in their documentation. Personally I used this command to install the beta version:

Next I followed these two guides to configure the rclone remote “spaces”:

  1. https://rclone.org/s3/#digitalocean-spaces
  2. https://www.digitalocean.com/community/tutorials/how-to-migrate-from-amazon-s3-to-digitalocean-spaces-with-rclone

The Backup Script

My backup script is really quite simple:

A stripped down version could be as simple as this:

The main idea is that we backup daily by running this script using cron. The $(date +%d) evaluates to the current day of month such as 01, 15 or 30. This way we get a simple and straight forward “rotation of files”. If we have many droplets backing up using the same script we avoid clashes by adding ${HOSTNAME} in there. We add –delete-excluded to remove files and not just add during the sync. The –skip-links silences a warning about symlinks not being possible to be backed up. Finally we specify what directories to include in the backup.

The time before rclone means we get a message saying how long it all took at the end.

To get progress information we add –stats 60s and –stats-log-level NOTICE to the mix.

We speed up the whole backup process by adding –transfers=32 and –checkers=128 in there. The default values are way to low and better performance can be reached by increasing the values like so.

Add to Crontab

Time to add this to the crontab! Feel free to read my article about cron jobs first.

Logged in as root I get started by ensuring the script is actually executable:

Then I open up the crontab …

… and add the entry so that the crontab looks like this:

This makes the backup run at 04:12 in the morning. To look at the results you may simply:

And that’s that!

By | 2018-02-02T15:03:41+00:00 February 2nd, 2018|Uncategorized|0 Comments

Leave A Comment