btrfs incremental snapshots


Recommended Posts

1 minute ago, pyrater said:

anyone using this? Have you had issues? regrets? lessons learned?

 

Been using this since I got unraid setup.

 

Takes daily snapshots and no issues at all.

 

Only real lesson is that you have to clear snapshots to regain space from deleted files but that should go without saying.

Link to comment
  • 1 month later...

I found myself here looking for answers after watching @SpaceInvaderOne's most recent video "Ransomware vs Snapshots with VMs". This is probably the biggest issue I have with Unraid. Not having native snapshot ability in the GUI is saddening. Thank you for making this script and I would also love to see this turned into a plugin. I hope a new video is on setting this up is on @SpaceInvaderOne's short list since his most recent one was snapshot/ransomware related.

Link to comment
  • 2 years later...

As I'm using this script within the "User scripts"-plugin I just want to post some improvements I have implemented:

 

- the script can now process a directory with spaces in its name

- the script checks if it is really processing a directory (the former script had some trouble with unwanted files under /mnt/diskX/)

 

Feel free to check out but keep in mind: Always make a backup of your data first!

 

Note: The snapshotted Unraid share must be on a BTRFS-disk and this disk must be configured as a included "Primary" or "Secondary" storage for this share (see configuration on Unraid web panel -> "Shares" ... click on share ... check "Share setting")

 

#!/bin/bash
#description=This script implements snapshots on btrfs array drives.
#arrayStarted=true

## Credits
# catapultam_habeo - Initial script
# Tomr - Modified version with SNAPSHOT_TYPE retention policy
# studmw - Modified script for spaces in directory names and a directory/file check (and German console output)

#if you change the type you'll have to delete the old snapshots manually
#valid values are: hourly, daily, weekly, monthly
SNAPSHOT_TYPE=hourly

#how many snapshots should be kept
MAX_SNAPS=2

#name of the shares to exclude stored as an array: EXCLUDE=('share 1' 'share 2')
EXCLUDE=('tm macbook air' 'tm macbook pro')

#name of the snapshot folder and delimeter. Do not change.
#https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html
SNAPSHOT_DELIMITER="_UTC_"
SNAPSHOT_FORMAT="$(TZ=UTC date +${SNAPSHOT_TYPE}${SNAPSHOT_DELIMITER}%Y.%m.%d-%H.%M.%S)"

#make empty directories not freak out
shopt -s nullglob

#btrfs check
is_btrfs_subvolume() {
    local dir=$1
    [ "$(stat -f --format="%T" "$dir")" == "btrfs" ] || return 1
    inode="$(stat --format="%i" "$dir")"
    case "$inode" in
        2|256)
            return 0;;
        *)
            return 1;;
    esac
}

#part from original script (not used)
#POSITIONAL=()
#while [[ $# -gt 0 ]]
#do
#key="$1"

#case $key in
#    -n|--number)
#    MAX_SNAPS="$2"
#    shift # past argument
#    shift # past value
#    ;;
#    -e|--exclude)
#    EXCLUDE="$2"
#    shift # past argument
#    shift # past value
#    ;;
#    *)
#    POSITIONAL+=("$1") # save it in an array for later
#    shift # past argument
#    ;;
#esac
#done
#set -- "${POSITIONAL[@]}" # restore positional parameters

#adjust MAX_SNAPS to prevent off-by-1
MAX_SNAPS=$((MAX_SNAPS+1))

#tokenize exclude list
declare -A excludes
for token in "${EXCLUDE[@]}"; do
    excludes[$token]=1
	#debug echo line
    echo "Bearbeite das EXCLUDE-Verzeichnis \"$token\" und habe hier den Wert \"${excludes[$token]}\" gesetzt"
done

#iterate over all disks on array
for disk in /mnt/disk*[0-9]* ; do
    #examine disk for btrfs-formatting
    if is_btrfs_subvolume $disk ; then 
        echo "\"$(basename "${disk}")\" ist ein BTRFS-Laufwerk und wird bearbeitet:"
        #iterate over shares present on disk
        for share in ${disk}/* ; do
            #test for exclusion
            if [ ! -n "${excludes[$(basename "${share}")]}" ]; then
				#check for .snapshots directory prior to generating actual snapshot
				if [ -d "$disk" ]; then
					if [ ! -d "$disk/.snapshots/" ] ; then
						echo "Erstelle ein neues BTRFS-Subvolume \"$disk/.snapshots/\" als Snapshot-Hauptverzeichnis"
						btrfs subvolume create "${disk}/.snapshots"
					fi
					if [ ! -d "$disk/.snapshots/$SNAPSHOT_FORMAT/" ] ; then
						echo "Erstelle neues BTRFS-Subvolume \"$disk/.snapshots/$SNAPSHOT_FORMAT/\" als Snapshot-Unterverzeichnis"
						btrfs subvolume create "$disk/.snapshots/$SNAPSHOT_FORMAT"
					fi
				fi
                echo "Bearbeite \"$share\":"
                    #check if it is a directory
                    if [ ! -d "$share" ]; then
                        echo "$share ist kein Verzeichnis und wird übersprungen..."
                    else
                        is_btrfs_subvolume "$share"
                        if [ ! "$?" -eq 0 ]; then
                            echo "\"$share\" ist kein BTRFS-Subvolume und wird nun in ein neues BTRFS-Subvolume verschoben..."
                            mv -v "${share}" "${share}_TEMP"
                            btrfs subvolume create "$share"
                            cp -axvT --reflink=always "${share}_TEMP" "$share"
                            rm -vrf "${share}_TEMP"
                        fi
                        btrfs subvolume snap "${share}" "${disk}/.snapshots/${SNAPSHOT_FORMAT}/$(basename "${share}")" #read only use: -r
                    fi
			else
				echo "\"$share\" ist auf der EXCLUDE-Liste und wird übersprungen..."
			fi
		done
		#find old snaps to delete
		echo "Ich habe $(find "${disk}/.snapshots/${SNAPSHOT_TYPE}${SNAPSHOT_DELIMITER}"*/ -maxdepth 0 -mindepth 0 | sort -nr | tail -n +$MAX_SNAPS | wc -l) überzählige(n) \"$SNAPSHOT_TYPE\" Snapshot(s) gefunden"
		for snap in $(find "${disk}/.snapshots/${SNAPSHOT_TYPE}${SNAPSHOT_DELIMETER}"*/ -maxdepth 0 -mindepth 0 | sort -nr | tail -n +$MAX_SNAPS); do
			for share_snap in ${snap}/*; do
				echo ""$share_snap" wird gelöscht"     
				btrfs subvolume delete -c "$share_snap"
			done
			btrfs subvolume delete -c "$snap"
		done
	fi
done

 

Edited by studmw
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.