[btrfs incremental snapshots]

I've modified @Tomr's snapshot with retention policy script to merge in @catapultam_habeo's original snapshot pruning method in order to improve the reliability of snapsot pruning. I've changed it back to a number of snapshots being retained instead of retention minutes. I found that going off minutes was fairly unreliable and often failed to auto-prune.

EDIT: Forgot I also added back the exclusion list instead of an inclusion list.

Instructions for shadow copy remains unchanged:

On 10/13/2020 at 7:51 PM, Tomr said:

(Optional) Add this to your Settings->SMB->SMB Extra. This enables file versioning for smb clients.

vfs objects = shadow_copy2
shadow:sort = desc
shadow:format = _UTC_%Y.%m.%d-%H.%M.%S
shadow:localtime = no
shadow:snapprefix = ^\(monthly\)\{0,1\}\(weekly\)\{0,1\}\(daily\)\{0,1\}\(hourly\)\{0,1\}$
shadow:delimiter = _UTC_

Here's the script:

#!/bin/bash
#description=This script implements incremental snapshots on btrfs array drives.
#arrayStarted=true

## Credits
# catapultam_habeo - Initial script
# Tomr - Modified version with SNAPSHOT_TYPE retention policy

#If you change the type you'll have to delete the old snapshots manually
#valid values are: hourly, daily, weekly, monthly
SNAPSHOT_TYPE=hourly
#How many snapshots should be kept. 
MAX_SNAPS=
#Name of the shares to exclude, can be comma separated like "medias,valuables"
EXCLUDE=
#name of the snapshot folder and delimeter. Do not change.
#https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html
SNAPSHOT_DELIMETER="_UTC_"
SNAPSHOT_FORMAT="$(TZ=UTC date +${SNAPSHOT_TYPE}${SNAPSHOT_DELIMETER}%Y.%m.%d-%H.%M.%S)"


shopt -s nullglob #make empty directories not freak out

is_btrfs_subvolume() {
    local dir=$1
    [ "$(stat -f --format="%T" "$dir")" == "btrfs" ] || return 1
    inode="$(stat --format="%i" "$dir")"
    case "$inode" in
        2|256)
            return 0;;
        *)
            return 1;;
    esac
}

#ADJUST MAX_SNAPS to prevent off-by-1
MAX_SNAPS=$((MAX_SNAPS+1))

#Tokenize exclude list
declare -A excludes
for token in ${EXCLUDE//,/ }; do
	excludes[$token]=1
done

#iterate over all disks on array
for disk in /mnt/disk*[0-9]* ; do
#examine disk for btrfs-formatting (MOSTLY UNTESTED)
 if is_btrfs_subvolume $disk ; then 
 #iterate over shares present on disk
  for share in ${disk}/* ; do
  declare baseShare=$(basename $share)
   #test for exclusion
   if [ -n "${excludes[$baseShare]}" ]; then
       echo "$share is on the exclusion list. Skipping..."
   else
     #check for .snapshots directory prior to generating snapshot
	 #only before we make the actual snapshot 
	 #so we don't create empty folders 
	  if [ -d "$disk" ]; then
	   if [ ! -d "$disk/.snapshots/$SNAPSHOT_FORMAT/" ] ; then
		#echo "new"
		mkdir -v -p $disk/.snapshots/$SNAPSHOT_FORMAT
	   fi
	  fi
  
    #echo "Examining $share on $disk"
    is_btrfs_subvolume $share
    if [ ! "$?" -eq 0 ]; then
     echo "$share is likely not a subvolume"
     mv -v ${share} ${share}_TEMP
     btrfs subvolume create $share
     cp -avT --reflink=always ${share}_TEMP $share
     rm -vrf ${share}_TEMP
    fi
    #make new snap
    btrfs subvolume snap -r ${share} $disk/.snapshots/${SNAPSHOT_FORMAT}/$baseShare
  fi
 done
	#find old snaps
    echo "Found $(find ${disk}/.snapshots/${SNAPSHOT_TYPE}${SNAPSHOT_DELIMETER}*/ -maxdepth 0 -mindepth 0 | sort -nr | tail -n +$MAX_SNAPS | wc -l) old snaps"
    for snap in $(find ${disk}/.snapshots/${SNAPSHOT_TYPE}${SNAPSHOT_DELIMETER}*/ -maxdepth 0 -mindepth 0 | sort -nr | tail -n +$MAX_SNAPS); do
        for share_snap in ${snap}/*; do
         btrfs subvolume delete $share_snap
        done
     rmdir $snap --ignore-fail-on-non-empty
    done

 fi
done

Hope it's useful!

[[SOLVED] Parity and BTRFS Snapshots]

Ok so that means that it shouldn't affect parity at all since it's just in the filesystem right? It's just as if you deleted a bunch of files and created new ones.

 

Thank you!

[[SOLVED] Parity and BTRFS Snapshots]

Hi,

I have a question concerning parity and BTRFS snapshots. 
Let’s say all my data disks are BTRFS, and I snapshotted every single one. Now suppose something catastrophic happened and I need to rollback the snapshot on all the disks. Wouldn’t the parity be out of sync and if so, how would I fix that?

 

Thank you so much!