Jump to content
Ancan

Basic brtfs snapshot and backup script: snapback.sh

2 posts in this topic Last Reply

Recommended Posts

Hi all,

Just spent the day creating a somewhat simple script for creating snaphots and transferring them to another location, and thought I'd throw it in here as well if someone can use it or improve on it.

Note that it's user-at-your-own risk. Could probably need more fail-checks and certainly more error checking, but it's a good start I think. I'm new to btrfs as well, so I hope I've not missed anything fundamental about how these snapshots works.

 

The background is that I wanted something that performs hot backups on my VM's that lives on the cache disk, and then moves the snapshots to the safety of the array, so that's more or less what this does, with a few more bells and whistles.

 

- It optionally handles retention on both the primary and secondary storage, deleting expired snapshots.

- Snapshots can be "tagged" with a label, and the purging of expired snapshots only affects the snapshots with this tag, so you can have different retention for daily, weekly and so on.

- The default location for the snapshots created on the primary storage is a ".snapshots" directory alongside the subvolume you are protecting. This can however be changed, but no check is currenlty performed that it's on the same volume as the source subvolume.

 

To use it there's some prerequisites:

- Naturally both the source and destination volumes must be brtfs.

- Also, all things you want to protect must be converted to a brtfs subvolume if they are not.

- Since there's way to manage btrfs subvolumes that span multiple disks in unRAID, the source and destinations must be specified by disk path (/mnt/cache/..., /mnt/diskN/...).

 

Note that this is a very abrubt way to protect VM's, with no VSS integration or other means of flushing guest OS file system. It's however not worse than what I've been doing at work with NetApp/vmware for years, and I've yet to see a rollback that didn't work out just fine there.

 

Below is the usage header quoted, and the actual script is attached.

 

Example of usage:

 

./snapback.sh --source /mnt/cache/domains/pengu --destination /mnt/disk6/backup/domains --purge-source 48h --purge-destination 2w -tag daily

 

This will create a snapshot of the virtual machine "pengu" under /mnt/cache/domains/.snapshots, named something like pengu.2019-10-27@1828.daily.

It will then transfer this snapshot to /mnt/disk6/backup/domains/pengu.2019-10-27@1828.daily.

The transfer will be incremental or full depending on if a symbolic link called "pengu.last" exists in the snapshot-directory. This link always points to the latest snapshot created for this subvolume.

Any "daily" snapshots on the source will be deleted if they are older than 48 hours, and any older than two weeks will be deleted from the destination.

 

# snapback.sh
#
# A.Candell 2019
#
# Mandatory arguments
#       --source | -s
#               Subvolume that should be backed up
#
#       --destination | -d
#               Where the snapshots should be backed up to.
#
# Optional arguments:
#
#       --snapshot-location | -s
#               Override primary storage snapshot location. Default is a directory called ".snapshots" that is located beside the source subvolume.
#
#       --tag | -t
#               Add a "tag" on the snapshot names (for example for separating daily, weekly).
#               This string is appended to the end of the snapshot name (after the timestamp), so make it easy to parse and reduce the risk of
#               mixing it up with the subvolume name.
#
#       --create-destination | -c
#               Create destination directory if missing
#
#       --force-full | -f
#               Force a full transfer even if a ".last" snapshot is found
#
#       --purge-source <maxage> | -ps <maxage>
#               Remove all snapshots older than maxage (see below) from snapshot directory. Only snapshots with specified tag is affected.
#
#       --purge-destination <maxage> | -pd <maxage>
#               Remove all snapshots older than maxage (see below) from destination directory. Only snapshots with specified tag is affected.
#
#       --verbose | -v
#               Verbose mode
#
#       --whatif | -w
#               Only echoes commands, not executing them.
#
# Age format:
#       A single letter suffix can be added to the <maxage> arguments to specify the unit used.
#       NOTE: If no suffix is specified, hours are assumed.
#               s = seconds (5s = 5 seconds)
#               m = minutes (5m = 5 minutes)
#               h = hours (5m = 5 hours)
#               d = days (5d = 5 days)
#               w = weeks (5w = 5 weeks)

 

 

snapback.sh

Edited by Ancan

Share this post


Link to post

...and here's an example on how I'm using it.

 

This is my "daily-backup.sh", that is scheduled daily at 01:00. It snapshots and backs up all VM's and all directories under appdata.

 

#!/bin/sh

# VMs

KEEP_SRC=48h
KEEP_DST=14d

SRC_ROOT=/mnt/cache/domains
DST_ROOT=/mnt/disk6/backup/domains

cd /mnt/disk6/backup

virsh list --all --name | sed '/^$/d' | while read VM; do
        /mnt/disk6/backup/snapback.sh -c -s $SRC_ROOT/$VM -d $DST_ROOT -ps $KEEP_SRC -pd $KEEP_DST -t daily
done


# AppData

KEEP_SRC=48h
KEEP_DST=14d

SRC_ROOT=/mnt/cache/appdata
DST_ROOT=/mnt/disk6/backup/appdata

for APP in $SRC_ROOT/*/; do
        /mnt/disk6/backup/snapback.sh -c -s $APP -d $DST_ROOT -ps $KEEP_SRC -pd $KEEP_DST -t daily
done

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.