postcpbackup – auto cpbackup rotation

by brian on January 31, 2010

This script that rotates cpanel backups after they are completed. You can set it to keep N copies of your daily, weekly and monthly backups. It has been in production use since Sept 2008 on multiple servers, and has been in use around the net since it was published early in 2009.

The script is run after a cpbackup run completes successfully. The script rotates daily, weekly and monthly copies keeping N copies of each. You can specify N separately for daily, weekly and monthly through keepdaily/keepweekly/keepmonthly variables, and no copies are made if those variables are set to empty or not set. For instance, the following would keep two copies of daily backups and 3 weeks of weekly and no monthly backups:

keepdaily=2
keepweekly=3
keepmonthly=

The settings shown here keep 2 copies of the daily backup, 3 copies of the weekly backup, and no monthly backups (ie just the last one).

You can put these settings in the script itself (not recommended) or in a file /backup/cpbackup/.postcpbackup – we recommend putting the settings in the setting file so you can keep a single copy of the script the same across multiple servers.

The script is run from /scripts by cpbackup (the cPanel backup program) itself when it completes, so there is little risk of it rotating a backup unless backups have completed. [One day I should add a further sanity check that the backups were done OK to prevent false rotations, but haven’t yet]

To install the script, simply copy it to /scripts/postcpbackup and make it mode 755 – it will then be called when cPanel’s cpbackup process completes. Since it is run from cpbackup, adding it to crontab explicitly will cause a world of problems – just putting it in /scripts is sufficient for it to run.  Note – there is no need to edit cpanel scripts, as the backup script looks for a file called  “/scripts/postcpbackup” and, if it finds it exists, runs it at completion of the backup.

cPanel now requires you to go into WHM –> “Configure Backups” and down the bottom under “Execute Pre/Post Backup Script” tick the “postcpbackup” checkbox.

 

Note: This solution rotates your backups and thus won’t keep old account backup data, as cPanel does.  cPanel’s backup strategy only keeps one monthly, weekly and daily backup but doesn’t delete old backups, in case you need them.  We recommend you use our prekillacct script in tandem with this script, as that backs up an account as it is deleted and keeps the backup indefinitely in a separate directory away from backup rotation.

This has been posted at forums.cpanel.net a few times (first being early 2009):  post1  post2

Download postcpbackup.sh [2013 downloads] txt

This script works nicely with the cprevert command (one command file restore).

Originally released: 12 Feb 2009

  • LDelgado

    I’ve been looking for something like this, will it work nowadays? are there any updates?
    Also… are the copies tar zipped?

    Cheers.

  • The script works just as it is, doesn’t need any updates. When you say are the copies tar zipped, do you mean the backup tarballs? They stay in the format that they are generated in, which is always a gzipped tarball (ie .tar.gz) as far as I know, the script doesn’t change them at all …

    You’ll see the script is pretty short, so it really doesn’t do much other than some simple rotation.

  • Will this work on FTP’ed backups as well?

  • Nevermind – I see it’s looking for backups in a folder locally. Would be awesome if it would do that too, though!!

    • Absolutely correct; it’s all about a local folder. You’re welcome to adapt it, not sure how it would work for ftp’ed backups! (You could run the script on the FTP server perhaps, but it would need a way to pick up that (1) there was a new backup there; and, (2) the backup had completed. The current script knows that the backup has completed as it is run from postcpbackup, which cPanel only runs on backup completion.

  • Michael

    Is there a way to know I setup the script properly? I ran test=echo /scripts/postcpbackup” as noted in the scrpt and it said was was going to mv daily to daily [date] etc. but while that seems to show it will move the backups into a particular folder to be rotated later is there a way to test the rotations without having to wait for the entire sequence to complete a few weeks, months down the road?

    • I guess you could create daily/files (“mkdir /backup/cpbackup/daily/files”) and watch it rotate the daily backups as it runs, and likewise for the weekly and monthly – it only rotates when it sees the “files” directory in there, so you could simulate it that way.

      But the bottom line is, the script does work and has been in use on production servers for years, so you can be fairly confident! (and if it doesn’t work, please come back here with details and error messages).

      Here’s what the /backup/cpbackup directory looks like on one of our servers:

      daily
      daily_20111201
      daily_20111202
      daily_20111203
      daily_20111204
      daily_20111205
      daily_20111206
      daily_20111207
      daily_20111208
      daily_20111209
      daily_20111210
      monthly
      monthly_201110
      monthly_201111
      monthly_201112
      weekly
      weekly_20111120
      weekly_20111127

      # cat /backup/cpbackup/.postcpbackup
      keepmonthly=4
      keepweekly=3
      keepdaily=10

      Note the way it works is that it relies on cpbackup to make the weekly and monthly directories.

      • Michael

        Looks to be doing exactly as you said it would on my system as well. Is it possible to have different rotation settings for different accounts so my more important accounts can be have more extensive restore points and less urgent/infrequently changing accounts can have less restore points

  • At this point we haven’t added that sort of functionality though that idea’s occurred to me as well …

  • Hey there, great script, I’m using it and works great.

    I do have one small problem, trying to use the configuration file (.postcpbackup), whenever I have the file there (/backup/cpbackup/.postcpbackup) the script returns with this error:

    /scripts/postcpbackup: line 45: .: .postcpbackup: file not found

    yet, the file is there:

    # cat /backup/cpbackup/.postcpbackup
    keepmonthly=2
    keepweekly=3
    keepdaily=7

    If I remove the file then it works fine (I guess since it is picking up the configuration from the script itself).

    Any thoughts?

    • It looks almost as if there is an extra line in there, causing the error! Are there any lines other than the ones reported? This works as is on our servers, for instance one of the files is:

      # cat /backup/cpbackup/.postcpbackup
      keepmonthly=5
      keepweekly=4
      keepdaily=10
      #

      Sorry to reply so late!

  • 4u123

    Wouldn’t mind some advice on this problem we are having…

    We want to backup a server only on a monday each week and retain three weekly backups and three monthly ones. Unfortunately, neither the cpanel backup, or your script allows for this.

    The problem seems to be with the backup schedule. In cpanel, the only way to actually get a weekly backup when running the backup once a week, is to set the backup frequency to daily and choose the selected day of the week in the cpanel backup config – but this fails to save a monthly. So your script wont rotate it.

    Here is a post on the cpanel forums about this…

    http://forums.cpanel.net/f402/retaining-monthy-backups-when-backup-run-specific-day-277352.html

    • Chris,

      I’m afraid I can’t help here; I haven’t really run with a config like that.

      At a guess, you could write something based on find(1) (to test the directory age) and do the move yourself?

      Something like:
      if [ "$(find monthly -mtime +29 -prune)" != "" ]
      then
      mv monthly monthly.last
      mv daily monthly
      fi
      ...

      I’m afraid I have no idea how to tie that into postcpbackup, but that’s where you should put it, rather than modifying cpbackup.

  • Michiel

    Hi,

    I’m having a strange issue when trying this script on an NFS mount. I’ve tried it on a test server with a locally mounted backup directory and the script works like a charm there. When i try to use it on one of our production servers i get a strange issue. The script is actually called, the date tagged directories are created, the directories show up in the restore a backup interface in the cpanel WHM, however when i click a date tagged directory, lets say daily_20120706 for example, no accounts are shown. When i copy the contents of the date tagged directory to the normal daily directory the accounts do show up there. I tried asking cPanel what could cause the restore backups interface to not recognize the backups (it works fine on the testing server), but so far they havent been able to produce any helpful suggestions. Personally i have no clue how i can troubleshoot this. Any hints/directions would be appreciated!

    • Michiel

      And as always shortly after posting i get a solution. I had changed the date formatting in the script a little and it turns out that cpanel’s restore functionality does not support hyphens in file names.

  • Carlo Santoni

    Hi I have some issue with the script.
    It’s been working for a week then after I did some change to the backups (I installed http://applications.cpanel.net/restore-manager/ and change some setting in cpanel backup configuration page) the script stopped rotating backups.
    I did everything to restore previous settings and even deleted old backups but the script has no effect at all.
    I am using it as it is (I only changed rotation to
    keepmonthly=
    keepweekly=
    keepdaily=5
    but I tested it with other settings as well)

    The script is executed and I don’t see any error but no extra backup folder IS CREATED ANYMORE.
    This is the final part of the cpbackup log:
    homesize is: 318955520
    [cpbackup] The following backups were already up to date: weekly,monthly
    [cpbackup] Executing user defined post backup script (/usr/local/cpanel/scripts/postcpbackup).
    [cpbackup] Completed at Wed Jul 25 12:29:54 2012

    ANY HELP IS APPRECIATED.
    Thanks Carlo

    • Hi Carlo,

      I would try putting this line in the top of the script, to see what is failing:

      exec > /var/log/postcpbackup 2>&1
      set -x

      This will send log output from the last run of postcpbackup to the /var/log/postcpbackup file where you can inspect it and see where the error lies.

      I suspect one of the arithmetic lines is failing ie (( keepweekly++ ))
      For those to work the value has to either be completely empty (in which case the code block is skipped) or it has to be an integer.

      Please let us know what you find!

      • Carlo Santoni

        Hi Brian, thanks a LOT.
        I followed your instructions and then I watched at the log file /var/log/postcpbackup, this is the full result: http://d.pr/n/UMLg
        I hope it makes more sense for you than it does for me….
        Let me know if you can give any help, in case you need it, this is the full postcpbackup code http://d.pr/n/fLs4
        THANKS A LOT, Carlo

        • Carlo – very late reply, sorry … looks like cpanel is not creating the backup in the first place. The script only rotates a backup if it sees one has been created by cpanel.

  • Ioan

    Hi

    I have copied the script to the correct folder, gave it chmod 755.

    I imagine that this has to be called from the cpbackup script for it to start its work? How do I call it and at what line should the call be made?

    Thanks

    Ioan

    • If you check the install instructions, or if you check cpbackup itself, it’s sufficiently simply to name it /scripts/postcpbackup with mode 755. The cpbackup script will pick up and run /scripts/postcpbackup by name if it exists (line 624 or near). Altering cpbackup will cause it to be run twice.

      • Ioan

        Hi Brian

        Thanks for the info.

        I’ve located the issue. The instructions above say download “postcpbackup.sh” the instructions say /scripts/postcpbackup. This is slightly ambiguous.

        I created the file as postcpbackup.sh, then if you check line 624 or near, then it calls it without the .sh. Once that was renamed then it reported in the log:

        [cpbackup] Executing user defined post backup script (/usr/local/cpanel/scripts/postcpbackup).

        So its been called. I imagine that i wont see the different backup directories until tomorrow as the daily ones are up to date.

        Thanks for you help

        Regards

        Ioan

        • Thanks Ioan, I added a clarification line just to simplify this for future readers.

          • Ioan

            Hi Brian

            Ive let it run for a bit, but the /backup/cpbackup does not contain any rotation folders. Is this the location where they are created or is it somewhere else?

            As mentioned before the backup log states that it has called the script, but either the files are not created or im looking at the wrong place.

            Thanks

            Ioan

  • Brian,

    I use your script to make daily backups and weekly backups. I keep 6 daily backups and 5 weekly backups.

    How can I keep WHM from making a daily backup on the day that it makes the weekly backup?

    Thanks.

    Jim

    • Hi Jim,

      Bear in mind that cpanel is just making daily backups here that the postcpbackup script is rotating.

      You should be able to stop the rotation of the daily backup in the event a monthly backup is made. You could simply delete it (save the name in a variable and then delete it in the monthly case). A little messy but you should be able to work out a more elegant solution with a little thought. Or you could hire us to do the mods if you are from a larger host?

      Cheers,

      Brian

      • Brian,

        I see your point. I guess that I would like to turn off weekly (and monthly) backups and have the postcpbackup script move the daily folder to a weekly folder once per week, then the script would rotate the weekly scripts as normal. I don’t know enough coding to do this.

        i want to replace:
        $test mv daily daily_$cdate

        with:
        if weekly is older than 6.5 days or if date is Tuesday
        $test mv -f daily weekly
        else
        $test mv daily daily_$cdate
        fi

        Any help would be appreciated.

  • Ioan

    Having the same issue as carlo santoni i think, output of the log:

    + keepmonthly=2
    + keepweekly=4
    + keepdaily=6
    + export PATH=/usr/bin:/bin:/sbin:/usr/local/sbin:/usr/local/bin
    + PATH=/usr/bin:/bin:/sbin:/usr/local/sbin:/usr/local/bin
    + export POSTCPBACKUP=13742
    + POSTCPBACKUP=13742
    + testing=echo
    + system.backup
    + cd /backup/cpbackup
    + test -s .postcpbackup
    + ‘[‘ 6 ‘!=’ ” -a -d daily/files ‘]’
    + ‘[‘ 4 ‘!=’ ” -a -d weekly/files ‘]’
    + ‘[‘ 2 ‘!=’ ” -a -d monthly/files ‘]’
    + exit 0

    • Ioan, that looks like the backup hadn’t been created. The script relies on cpbackup to have run and created the backup before it runs. (Remember, of course, you don’t run this script directly, you simply put it in place as explained (ie as /scripts/postcpbackup) and the system will call it.)

      The other possibility is that the backup files are going somewhere else; worth checking the config to see where they are being directed.

  • Ioan

    Hi Brian

    Thanks for your patience.

    Before placing this script in the cpbackup was running fine, and being placed in /backup/cpbackup/ then daily, weekly and monthly folders. It still does that, but the rotation doesnt happen.

    The file is placed in /scripts/postcpbackup, and this file is called and noted in the log file of cpbackup. I dont call it independently, i just check the logs after the nightly backups. So it is calling it, and the backup is being placed in the default location.

    Dont know what else to do now, but give up!

    • Ioan

      when i go to /scripts, i get redirected to /usr/local/cpanel/scripts, this is where the postcpbackup file is located.

      I note in the script it says:

      export PATH=$PATH:/usr/local/sbin:/usr/local/bin
      export POSTCPBACKUP=$$

      Does this need to be modified to point at the above place? Cant see where else a reference to a folder might be incorrect.

      Thanks

  • So long as you can get output from “ls -l /scripts/postcpbackup” the redirection is fine. Normally /scripts is actually a symlink pointing somewhere else.

    The folder for backups is set in WHM somewhere, should be .backup – I think the cpbackup folder is created automatically.

  • Also, worth noting that cPanel now requires you to go into WHM –> Configure Backup
    – down the bottom there is an area “Execute Pre/Post Backup Script”. You will now need to tick postcpbackup for this system to work.

  • Michael

    Looks like the pre and post backup scripts feature is being depreciated in WHM and a new backup feature is coming in 11.38

    http://forums.cpanel.net/f49/how-get-checkboxes-precpbackup-postcpbackup-show-whm-327641.html

    http://forums.cpanel.net/f49/any-news-new-backup-feature-339951.html

Next post: