Is this ZFS snapshot cleanup script a good solution?

December 6, 20241 yr

Hi everyone,

I’m running my drives in a ZFS RAID on my Unraid server, and I’ve noticed that the number of snapshots keeps growing. To manage this, I want to periodically clean up old snapshots while keeping a specified number of recent ones.

I found the following script, which seems to address the problem by pruning snapshots and limiting their quantity to a defined number (24 in this case).

Could you let me know if this is a good approach or if there’s a better alternative for managing ZFS snapshots on Unraid?

Here’s the full script:

#!/bin/bash

#v0.10.2

########################unraid-snapshot-zfs#######################

###################### User Defined Options ######################

# List your ZFS datasets. You can add/remove sets

# readarray -t DATASETS < <(zfs list -o name -H) # when replacing DATASETS below, should return all pools/Datasets, not thoroughly tested yet. If you test this let me know!

DATASET=("zfsbackup/cachebackup")

mapfile -t DATASETS < <(zfs list -r -o name -H "${DATASET}")

# Set Number of Snapshots to Keep

SHANPSHOT_QTY=24

# Snapshot Name Format

SNAPSHOTNAME=$(date "+simplesnap_%Y-%m-%d-%H:%M:%S")

###### Don't change below unless you know what you're doing ######

##################################################################

echo "Starting Snapshot ${SNAPSHOTNAME}"

echo "_____________________________________________________________"

## Validation Steps

# Validate SHANPSHOT_QTY

if ! [[ $SHANPSHOT_QTY =~ ^[0-9]+$ ]]; then

echo "Error: SHANPSHOT_QTY must be a positive integer."

exit 1

fi

# Validate DATASETS

for dataset in "${DATASETS[@]}"; do

if ! zfs list "$dataset" &>/dev/null; then

echo "Error: Dataset '$dataset' does not exist."

exit 1

fi

done

# Function to handle errors

handle_error() {

local error_message="$1"

echo "Error: $error_message"

exit 1

}

# Function to create snapshot if there is changed data

create_snapshot_if_changed() {

local DATASET="$1"

local WRITTEN

WRITTEN=$(zfs get -H -o value written "${DATASET}")

if [[ "${WRITTEN}" != "0" ]]; then

if ! zfs snapshot "${DATASET}@${SNAPSHOTNAME}"; then

handle_error "Failed to create snapshot for ${DATASET}"

fi

echo "Snapshot created: ${DATASET}@${SNAPSHOTNAME}"

else

echo "No changes detected in ${DATASET}. No snapshot created."

fi

}

# Function to prune snapshots

prune_snapshots() {

local DATASET="$1"

local KEEP="${SHANPSHOT_QTY}"

# Declare the SNAPSHOTS array

declare -a SNAPSHOTS

# Use mapfile to split the output into the SNAPSHOTS array

if ! mapfile -t SNAPSHOTS < <(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@"); then

echo "Error: Failed to list snapshots for ${DATASET}"

return 1

fi

local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]}

echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}"

local SNAPSHOTS_SPACE

if ! SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}"); then

echo "Error: Failed to get space used by snapshots for ${DATASET}"

return 1

fi

echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}"

if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then

local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP))

for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do

if ! zfs destroy "${i}"; then

echo "Error: Failed to delete snapshot: ${i}"

return 1

fi

echo "Deleted snapshot: ${i}"

echo "_____________________________________________________________"

done

else

echo "_____________________________________________________________"

fi

}

# Iterate over each dataset and call the functions

for dataset in "${DATASETS[@]}"; do

# create_snapshot_if_changed "${dataset}"

prune_snapshots "${dataset}"

done

echo "----------------------------Done!----------------------------"

Do you think this script is safe and efficient? Can it potentially cause any issues?

Or is there a more reliable/better way to automate snapshot management in a ZFS setup on Unraid?

Thanks for your help!

Quote

1

January 11, 20251 yr

Thank you for the script - unfortunately, I can give you the answer you've asked for.
Did you get any feedback on the script or do you experienced any issues while using it?

Chat-GPT told me that there is maybe a typo:
The variable SHANPSHOT_QTY contains a typo ("SHANPSHOT" instead of "SNAPSHOT"):

## Validation Steps

# Validate SHANPSHOT_QTY

if ! [[ $SHANPSHOT_QTY =~ ^[0-9]+$ ]]; then

  echo "Error: SHANPSHOT_QTY must be a positive integer."

  exit 1

Quote

January 11, 20251 yr

I played around a bit with what ChatGPT gave me and tested it using the dry run function (to be honest, I'm a novice when it comes to these scripts).

Added:

Dry Run Mode
Age-Based Pruning (optional)
Error Handling

Quote

Key Features

Dataset Selection:
    Operates on the ZFS datasets specified in the DATASET array and dynamically generates a list of snapshots for each dataset.



Snapshot Retention:
    Retains the most recent snapshots, as defined by SNAPSHOT_QTY (e.g., 24 snapshots). Optionally deletes snapshots older than a specified number of months (DELETE_OLDER_THAN_MONTHS).



Dry Run Mode:
    When DRY_RUN=true, the script simulates deletions and lists snapshots it would delete without actually removing them.



Age-Based Pruning:
    Identifies snapshots older than DELETE_OLDER_THAN_MONTHS for deletion, provided the total number exceeds SNAPSHOT_QTY. Extracts dates from snapshot names (e.g., after @autosnap_YYYY-MM-DD) and calculates the snapshot age.



Error Handling:
    Skips snapshots with invalid or unparseable dates and provides warnings for any issues (e.g., missing datasets or invalid snapshot names).

----------------------------------------------------------------------------------------------------------------------------------------------------------------

How It Works



Dataset Validation:
    Checks that the datasets listed in the DATASET array exist and are valid ZFS datasets.

    Snapshot Listing:
    Uses zfs list to fetch snapshots for each dataset, filtering snapshots matching the @autosnap_YYYY-MM-DD pattern.



Age Calculation:
        Extracts the date from each snapshot name (e.g., after @autosnap_).
        Converts the date to a timestamp and calculates its age in months relative to the current date.



Pruning Logic:
    Snapshots are deleted if they meet both conditions:
        Age exceeds DELETE_OLDER_THAN_MONTHS.
        Total number of snapshots exceeds SNAPSHOT_QTY.



Dry Run or Deletion:
        If DRY_RUN=true, the script lists snapshots that would be deleted without actually deleting them.
        If DRY_RUN=false, the script deletes the qualifying snapshots using zfs destroy.

----------------------------------------------------------------------------------------------------------------------------------------------------------------

Usage Instructions



Set Configuration Variables:
        Update the DATASET array with your ZFS datasets.
        Adjust SNAPSHOT_QTY (number of snapshots to retain) and DELETE_OLDER_THAN_MONTHS (age threshold) to match your retention policy.
        Set DRY_RUN to true (for testing) or false (for live execution).



Run the Script:
    Use the User Scripts add-on to test the script and let it run manual or via cronjob

    Review Output: Check the output to see which snapshots are pruned (or would be pruned in dry run mode).

----------------------------------------------------------------------------------------------------------------------------------------------------------------

Example Snapshot Naming Convention

This script assumes snapshot names follow this format:
@autosnap_YYYY-MM-DD

Examples:

    cache_nvme_2tb/zfs_domains_backup/Windows@autosnap_2024-12-30
    cache_nvme_2tb/zfs_domains_backup/Windows@autosnap_2025-01-11

#!/bin/bash

# v0.10.4

########################unraid-snapshot-zfs#######################

###################### User Defined Options ######################

# Enable or disable dry-run mode (set to true for testing without making changes)
DRY_RUN=true  # Set to false to perform actual operations

# Enable or disable deletion of snapshots older than a specified number of months
DELETE_OLD_SNAPSHOTS=false  # Set to false to skip deleting old snapshots

# Set the age threshold for old snapshot deletion (in months)
DELETE_OLDER_THAN_MONTHS=12  # Change this to the desired number of months

# List your ZFS datasets. You can add/remove sets
DATASET=("/mnt/my_path_to_DATASET")
mapfile -t DATASETS < <(zfs list -r -o name -H "${DATASET}")

# Set Number of Snapshots to Keep
SNAPSHOT_QTY=24

# Snapshot Name Format
SNAPSHOTNAME=$(date "+simplesnap_%Y-%m-%d-%H:%M:%S")

###### Don't change below unless you know what you're doing ######
##################################################################

echo "Starting Snapshot ${SNAPSHOTNAME}"
echo "Dry-run mode: ${DRY_RUN}"
echo "Delete old snapshots: ${DELETE_OLD_SNAPSHOTS}"
echo "Delete snapshots older than: ${DELETE_OLDER_THAN_MONTHS} months"
echo "_____________________________________________________________"

## Validation Steps
# Validate SNAPSHOT_QTY
if ! [[ $SNAPSHOT_QTY =~ ^[0-9]+$ ]]; then
  echo "Error: SNAPSHOT_QTY must be a positive integer."
  exit 1
fi

# Validate DATASETS
for dataset in "${DATASETS[@]}"; do
  if ! zfs list "$dataset" &>/dev/null; then
    echo "Error: Dataset '$dataset' does not exist."
    exit 1
  fi
done

# Function to handle errors
handle_error() {
  local error_message="$1"
  echo "Error: $error_message"
  exit 1
}

# Function to create snapshot if there is changed data
create_snapshot_if_changed() {
  local DATASET="$1"
  local WRITTEN
  WRITTEN=$(zfs get -H -o value written "${DATASET}")

  if [[ "${WRITTEN}" != "0" ]]; then
    if [[ "$DRY_RUN" == "true" ]]; then
      echo "[DRY RUN] Would create snapshot: ${DATASET}@${SNAPSHOTNAME}"
    else
      if ! zfs snapshot "${DATASET}@${SNAPSHOTNAME}"; then
        handle_error "Failed to create snapshot for ${DATASET}"
      fi
      echo "Snapshot created: ${DATASET}@${SNAPSHOTNAME}"
    fi
  else
    echo "No changes detected in ${DATASET}. No snapshot created."
  fi
}

# Function to prune snapshots
prune_snapshots() {
  local DATASET="$1"
  local KEEP="${SNAPSHOT_QTY}"

  # Declare the SNAPSHOTS array
  declare -a SNAPSHOTS
  # Use mapfile to split the output into the SNAPSHOTS array
  if ! mapfile -t SNAPSHOTS < <(zfs list -t snapshot -o name,creation -s creation -r "${DATASET}" | grep "^${DATASET}@"); then
    echo "Error: Failed to list snapshots for ${DATASET}"
    return 1
  fi

  local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]}
  echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}"

  local SNAPSHOTS_SPACE
  if ! SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}"); then
    echo "Error: Failed to get space used by snapshots for ${DATASET}"
    return 1
  fi
  echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}"

  local NOW=$(date +%s)

  # Delete snapshots older than the specified number of months
  if [[ "$DELETE_OLD_SNAPSHOTS" == "true" ]]; then
    echo "Checking for snapshots older than ${DELETE_OLDER_THAN_MONTHS} months..."
    for snapshot_info in "${SNAPSHOTS[@]}"; do
      # Extract the snapshot name and creation date
      local snapshot_name=$(echo "$snapshot_info" | awk '{print $1}')
      local snapshot_creation_date=$(echo "$snapshot_info" | awk '{print $2}')
      local snapshot_timestamp=$(date -d "$snapshot_creation_date" +%s)

      # Calculate age in months
      local age_in_months=$(( (NOW - snapshot_timestamp) / (60 * 60 * 24 * 30) ))  # Approximate 30 days per month

      if [[ $age_in_months -ge $DELETE_OLDER_THAN_MONTHS ]]; then
        if [[ "$DRY_RUN" == "true" ]]; then
          echo "[DRY RUN] Would delete old snapshot: ${snapshot_name} (Age: ${age_in_months} months)"
        else
          if ! zfs destroy "${snapshot_name}"; then
            echo "Error: Failed to delete snapshot: ${snapshot_name}"
            return 1
          fi
          echo "Deleted old snapshot: ${snapshot_name} (Age: ${age_in_months} months)"
        fi
      fi
    done
  fi

  # Prune snapshots to retain only the desired quantity
  if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then
    local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP))
    for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do
      local snapshot_name=$(echo "$i" | awk '{print $1}')
      if [[ "$DRY_RUN" == "true" ]]; then
        echo "[DRY RUN] Would delete snapshot: ${snapshot_name}"
      else
        if ! zfs destroy "${snapshot_name}"; then
          echo "Error: Failed to delete snapshot: ${snapshot_name}"
          return 1
        fi
        echo "Deleted snapshot: ${snapshot_name}"
        echo "_____________________________________________________________"
      fi
    done
  else
    echo "_____________________________________________________________"
  fi
}

# Iterate over each dataset and call the functions
for dataset in "${DATASETS[@]}"; do
  create_snapshot_if_changed "${dataset}"  # Uncommented to enable snapshot creation
  prune_snapshots "${dataset}"
done

echo "----------------------------Done!----------------------------"

Edited January 11, 20251 yr by gilladur

Quote

Is this ZFS snapshot cleanup script a good solution?

Featured Replies

Join the conversation

Account

Navigation

Search

Configure browser push notifications

Chrome (Android)

Chrome (Desktop)

Safari (iOS 16.4+)

Safari (macOS)

Edge (Android)

Edge (Desktop)

Firefox (Android)

Firefox (Desktop)