Script - ZFS - simple-snapshot-zfs - Automate Snapshot, Avoid 0 Change Snapshots, Prune Old Snapshots

[samsausages]

I didn't like that too many snapshots were created with 0 file changes.  So I wrote a script that creates the snapshot only if data has been written to the dataset since the last snapshot, using the "written" property.  It also prunes snapshots afterwards, keeping only the 100 most recent.  Number to KEEP can be adjusted in the script.

 

Use User-Scripts Plug-in to setup a cron schedule. 

Hope it helps someone!

For the most current version, go to the simple-snapshot-zfs github repository.  I will not update the code in this post.   
https://github.com/samssausages/simple-snapshot-zfs

Sample output:
 

Starting Snapshot 20221221-2349
_______________________________________
No changes detected in pool/dataset1. No snapshot created.
Total snapshots for pool/dataset1: 1
Space used by snapshots for pool/dataset1: 0B
_______________________________________
Snapshot created: pool/dataset2@20221221-2349
Total snapshots for pool/dataset2: 6
Space used by snapshots for pool/dataset2: 8.06M
_______________________________________
No changes detected in pool/dataset3. No snapshot created.
Total snapshots for pool/dataset3: 3
Space used by snapshots for pool/dataset3: 947K
_______________________________________
No changes detected in pool/dataset4. No snapshot created.
Total snapshots for pool/dataset4: 4
Space used by snapshots for pool/dataset4: 17.5G
_______________________________________
No changes detected in pool/dataset4/nextcloud. No snapshot created.
Total snapshots for pool/dataset4/nextcloud: 2
Space used by snapshots for pool/dataset4/nextcloud: 20.0G
_______________________________________
-----------------Done!-----------------

 

 

Code:

#!/bin/bash

#v0.9
########################simple-snapshot-zfs#######################
###################### User Defined Options ######################
# List your ZFS datasets.  You can add/remove sets
# readarray -t DATASETS < <(zfs list -o name -H)  # when replacing DATASETS below, should return all pools/Datasets, not thoroughly tested yet.  If you test this let me know!
DATASETS=("workpool/admin" "workpool/archive")

# Set Number of Snapshots to Keep
SHANPSHOT_QTY=100

# Snapshot Name Format
SNAPSHOTNAME=$(date "+simplesnap_%Y-%m-%d-%H:%M:%S")

###### Don't change below unless you know what you're doing ######
##################################################################

echo "Starting Snapshot ${SNAPSHOTNAME}"
echo "_____________________________________________________________"

## Validation Steps
# Validate SHANPSHOT_QTY
if ! [[ $SHANPSHOT_QTY =~ ^[0-9]+$ ]]; then
  echo "Error: SHANPSHOT_QTY must be a positive integer."
  exit 1
fi

# Validate DATASETS
for dataset in "${DATASETS[@]}"; do
  if ! zfs list "$dataset" &>/dev/null; then
    echo "Error: Dataset '$dataset' does not exist."
    exit 1
  fi
done

# Function to handle errors
handle_error() {
  local error_message="$1"
  echo "Error: $error_message"
  exit 1
}

# Function to create snapshot if there is changed data
create_snapshot_if_changed() {
  local DATASET="$1"
  local WRITTEN
  WRITTEN=$(zfs get -H -o value written "${DATASET}")

  if [[ "${WRITTEN}" != "0" ]]; then
    if ! zfs snapshot "${DATASET}@${SNAPSHOTNAME}"; then
      handle_error "Failed to create snapshot for ${DATASET}"
    fi
    echo "Snapshot created: ${DATASET}@${SNAPSHOTNAME}"
  else
    echo "No changes detected in ${DATASET}. No snapshot created."
  fi
}

# Function to prune snapshots
prune_snapshots() {
  local DATASET="$1"
  local KEEP="${SHANPSHOT_QTY}"

  # Declare the SNAPSHOTS array
  declare -a SNAPSHOTS
  # Use mapfile to split the output into the SNAPSHOTS array
  if ! mapfile -t SNAPSHOTS < <(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@"); then
    echo "Error: Failed to list snapshots for ${DATASET}"
    return 1
  fi

  local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]}
  echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}"

  local SNAPSHOTS_SPACE
  if ! SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}"); then
    echo "Error: Failed to get space used by snapshots for ${DATASET}"
    return 1
  fi
  echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}"

  if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then
    local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP))
    for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do
      if ! zfs destroy "${i}"; then
        echo "Error: Failed to delete snapshot: ${i}"
        return 1
      fi
      echo "Deleted snapshot: ${i}"
      echo "_____________________________________________________________"
    done
  else
   echo "_____________________________________________________________"
  fi
}

# Iterate over each dataset and call the functions
for dataset in "${DATASETS[@]}"; do
  create_snapshot_if_changed "${dataset}"
  prune_snapshots "${dataset}"
done

echo "----------------------------Done!----------------------------"

 

Edited October 17, 2023 by samsausages

[TheLinuxGuy]

hey would this script automatically show "Shadow Copies" on windows for those unraid shares whose disks are zfs formatted individually (not a zpool of many disks)?

[samsausages]

On 7/20/2023 at 6:08 PM, TheLinuxGuy said:

hey would this script automatically show "Shadow Copies" on windows for those unraid shares whose disks are zfs formatted individually (not a zpool of many disks)?

You need to configure that, part of the config is in Samba.  Instructions are here:
https://forum.level1techs.com/t/zfs-on-unraid-lets-do-it-bonus-shadowcopy-setup-guide-project/148764

I didn't add anything to my script that would deal with shadowcopies on windows.  But it may still work for that purpose, I haven't reviewed that article in a few years.  But if I recall the change is mainly about the naming convention the script uses matching the Samba settings, so you would most likely only need to make that part match.
EDIT:
I recently did this config for windows and it's all done in the SMB config file on unraid.  After proper config in that file, it will show shadow copies when browsing that share.

Edited October 17, 2023 by samsausages

[samsausages]

Updated to v0.9.  Lots of upgrades to make it more durable and robust.  Mainly error checks and validation checks.

[ShivalWolf]

This script works quite well. I am running it with userscripts as a cron task to do ZFS snapshots.

Thankyou for your efforts with it.

 

Given the current way the script handles purging old snapshots based on the number of snapshots

would it be possible to purge by age.

eg. only keep snapshots less than 30 days old?

I have been looking how i may do this alteration myself and found an easy way to get the creation time. so if its not something you have considered then fingers crossed

 

>zfs list -t snapshot -p -o name,creation -s creation -r "${DATASET}"
Returns Snapshot Name and the creation time (as a unix timestamp)