In this guide I describe how to setup backups for your Linux server. We will backup both locally and remotely for extra security. We will backup…
- MySQL-databases using automysqlbackup.
- Normal files using rdiff-backup.
- Special large folders per weekday using rsync.
- To a remote server using rsync.
Setting up the folder structure
We start out with creating a folder structure like this:
mkdir /backup chmod 755 /backup mkdir /backup/local mkdir /backup/local/mysql mkdir /backup/local/files |
The new root-level folder “backup” is the storage location we will use for any kind of backups. The folder “local” inside it is where we will save the backup-files for that computer itself. Later on we will add sibling folders that contain backups from other computers.
Note that the folder “/backup” not necessarily need to be a folder on your main storage device. It could for example be a mount point or a symlink to a folder on another storage device. This makes a lot of sense if you main storage device is an SSD. Do not backup to an SSD.
Backup MySQL-databases
For MySQL-databases we will use the tool automysqlbackup. This tool is in the Ubuntu repositories. The versions are Ubuntu 12.04 with automysqlbackup 2.5 right now when I write this.
apt-get install automysqlbackup |
We need to make two changes in the config file “/etc/default/automysqlbackup”:
DBNAMES=`mysql --defaults-file=/etc/mysql/debian.cnf --execute="SHOW DATABASES" | awk '{print $1}' | grep -Ev '^phpmyadmin|test|information_schema|performance_schema|mysql|Database$' | tr \\\r\\\n ,\ ` ... BACKUPDIR="/backup/local/mysql" |
The grep part in DBNAMES is to exclude databases. You may want to add additional databases there that you don’t want to backup. You should now run it once to make sure it works:
automysqlbackup |
Look through the contents in “/backup/local/mysql” to verify it looks the way you want it to.
Question: When will these backups run?
Answer: When installing automysqlbackup the file “/etc/cron.daily/automysqlbackup” was created. Check the content of “/etc/crontab” to see when daily cron runs. In my case it looks like this:
# m h dom mon dow user command 17 * * * * root cd / && run-parts --report /etc/cron.hourly 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
So it seems daily cron-tasks run at 06:25 on my system (Ubuntu 12.04).
Backup normal files
For backing up normal files we are going to use a tool called rdiff-backup that creates incremental backups.
apt-get install rdiff-backup |
Question: How often should I backup?
Answer: I personally enjoy backing up once a day.
Question: How long should I save the backups?
Answer: I personally enjoy saving the backups for three months.
Question: What files should I backup?
Answer: A good start is “/home”, “/etc” and “/root”. You will however often need to setup some special exclude and include rules.
With this in mind I should create the file “/etc/cron.daily/backupfiles” with the following content:
#!/bin/bash rdiff-backup \ --include /home \ --include /etc \ --include /root \ --exclude / \ / /backup/local/files rdiff-backup --force --remove-older-than 3M /backup/local/files |
Ensure that file has the right permissions:
sudo chmod 755 /etc/cron.daily/backupfiles |
Say however that I would like to ignore all home-files for the user “derp” and only want the folders “a” and “b” from the user “herp”.
The file would then look like this instead:
#!/bin/bash rdiff-backup \ --include /home/herp/b \ --include /home/herp/a \ --exclude /home/herp \ --exclude /home/derp \ --include /home \ --include /etc \ --include /root \ --exclude / \ / /backup/local/files rdiff-backup --force --remove-older-than 3M /backup/local/files |
It is important to consider that:
Multiple include and exclude options take precedence in the order they are given.
- http://www.nongnu.org/rdiff-backup/examples.html#exclude
I prefer looking at the include/exclude order like this:
- The deeper the directory paths – the higher up in that cron-file they should be written.
- We can visualize the end result by imagining we apply the includes/excludes from bottom up.
Backup normal files using crontab
I actually prefer having more fine grained control over when backups are run. For this we will use crontab. Start out with creating a folder for your cron-scripts:
mkdir /root/cron |
And then create the file “/root/cron/backupfiles.sh” instead of the file “/etc/cron.daily/backupfiles”. Next we add a crontab entry as root:
crontab -e |
If we want it to run the same time as automysqlbackup (06:25) the crontab entry would look like this:
#m h dom mon dow command 25 6 * * * /root/cron/backupfiles.sh
Backup special large folders per weekday
To be more precise I run a few minecraft servers. The map files are huge and change often because they are compressed natively. If we took incremental backups (rdiff-backup) those increments would be very large due to the compression. I prefer creating this setup instead:
mkdir /backup/local/server mkdir /backup/local/server/1 mkdir /backup/local/server/2 mkdir /backup/local/server/3 mkdir /backup/local/server/4 mkdir /backup/local/server/5 mkdir /backup/local/server/6 mkdir /backup/local/server/7 |
Monday is “1″ and Sunday is “7″. Next we create the script “/root/cron/backupserver.sh”.
#!/bin/bash rsync -avz --delete /home/mc/files/server/ /backup/local/server/$(date '+%u') |
Note that after the first week the old backups will start getting overwritten.
Lets add it to the root crontab as well:
#m h dom mon dow command 25 6 * * * /root/cron/backupfiles.sh 35 6 * * * /root/cron/backupserver.sh
Backup to a remote server
For extra security it makes sense to backup to a remote computer as well. Lets call the local computer “torkel” and the remote computer “berit”. Let’s also assume entries for those are added to the file “/etc/hosts” so they resolve into actual ip-addresses.
For this we could simply rsync the folder “torkel:/backup/local” to “berit:/backup/torkel” but in our case the folder “torkel:/backup/local/server” will be huge. We thus only synchronize “torkel:/backup/local/1″ (the monday server backup).
Create a user for torkel@berit and create the backup folder:
ssh root@berit adduser torkel * give it a password for now * mkdir /backup/torkel chown torkel.torkel /backup/torkel |
Create a private key for root@torkel and transfer it to torkel@berit:
ssh root@berit ssh-keygen ssh-copy-id torkel@berit |
Remove the password from torkel@berit:
ssh root@berit passwd -d torkel |
Create the script “/root/cron/backupremote.sh”.
#!/bin/bash rsync -avz --delete \ --include server/1 \ --exclude server/* \ /backup/local/ torkel@berit:/backup/torkel |
Note that the include/exclude syntaxes are different for rsync and rdiff-backup.
Lets add it to the root crontab as well:
#m h dom mon dow command 25 6 * * * /root/cron/backupfiles.sh 35 6 * * * /root/cron/backupserver.sh 0 7 * * * /root/cron/backupremote.sh
Restoring MySQL-databases
TODO
Restoring normal files
TODO
Restoring special large folders
TODO
That way you won’t need to mess with symbolic links at all. If you are using the dynmap plugin you should configure so the rendered tiles are stored elsewhere. Because all those images can easily reach over 10 gigabytes.