cpmr – full server cpanel rsync backup script

by brian on August 21, 2010

cpmr produces a full file-based backup of a cPanel server and is lightning fast and puts little load on the system.  It is based on a number of forum posts from WHT and cPanel forums and about 2 years of live use.

cpmr can be used to produce a minimal full backup of a system and then keep it up to date either on a second machine, or on a spare disk; very useful as an extra backup to protect against disk failure, machine failure (or even the very unlikely possibility of datacentre failure!). It’s wise to keep cPanel account-based tarball backups as well, perhaps run less often if you really need the space.

Overview

Cpmr was invented for two reasons – because cPanel’s cpbackup had some performance issues at the time (mostly fixed now in 2010), and to assist with making an image that could be used to do a fast restore in the unlikely event of a complete machine failure.  Restoring from cpbackup tarballs could take 12-24 hours for even a medium-size host with 400 accounts; restoring from an rsync backup is 2 hours or less (2 hours machine-to-machine, 45 mins on same machine).

When we say minimal full backup above, we mean that cpmr is intended to just copy the minimum of files necessary to enable emergency restore. Currently it takes all of cpanel including the bits that are compiled, so could be optimised to take a little less with some careful work. If you do that work, please comment below or email us.  Secondly, rsync only ever copies changes, so the amount transferred is minimal.  This bears emphasizing.

Using cpmr to migrate between servers

We’ve used cpmr a few times in migrating from one server to another. This works brilliantly; you can run it several times on the day of migration to copy the files over (say 2 hours for first run); and then shut everything down on both servers at the time of transfer and run it once more, and it should complete within 15 minutes and you’ve got the latest copies of everything. Remember, as mentioned above, rsync is itself optimized and will only copy files that have changed, making the process as fast and bandwidth efficient as possible.

If using this for migrating to a new server — one very important point to note is that the basic system UIDs must be the same. That is, the UIDs created by an empty cPanel install on the target machine must match the UIDs for these accounts on the source system before the move. You can rectify that fairly easily if they’re not; but you need to check and fix it on the new server (make the UIDs the same as the source server) before you start the transfer as it can cause complications otherwise. (this includes the cpanel uids as well as anything else you see on an empty installation). We haven’t covered how to do this here; but the simplest way is to either renumber the system uids (like cpanel etc) before the cpanel install, or run a “find -user 501 -print | xargs chown cpanel” (where 501 is the old cpanel userid).

If migrating, some skill is needed in using this tool; it is blindingly fast, but you will be overwriting system files so you need to know that in using it, you are on your own! We will happily incorporate any experience you have in using it back into these instructions; please do supply feedback on how your migration went.

Setup from crontab

You can and should read and tune the cpmr script before use. For instance, it allows you to add extra directories to the backup list easily via extras=”…” cpmr …

You’ll need to understand ssh and keys to use this script from cron to go to a remote system; to set a cron version, simply use a target of machine: as the argument. Directions on setting up a remote client are around the net; we strongly recommend you set up the remote key to only allow execution of cpmr to ensure you don’t create a root loophole. You might also use IP limitations to make things even tighter.

Example 1:
50 23,7 * * * cd /backup && /usr/local/sbin/cpmr > cpmr.local.out 2>&1
– this creates or updates a backup in /backup/recovery/host (which should be on a separate backup disk only used for backup)

Example 2:
30 1 * * * cd /backup && (mv cpmr.host2.out cpmr.host2.last; /backup/cpmr host2-backup: > cpmr.host2.out 2>&1)
– this sends a backup to host2, putting it in /backup/recovery/hostname, where hostname is your hostname. host2-backup is a special 10.x.x.x IP used for the local internal network, if you don’t need that simply use host2’s name.

Download

Your feedback and improvements are welcomed!

Download cpmr.sh [2806 downloads] txt

Previous post:

Next post: