March 22, 20233 yr I didn't like that too many snapshots were created with 0 file changes. So I wrote a script that creates the snapshot only if data has been written to the dataset since the last snapshot, using the "written" property. It also prunes snapshots afterwards, keeping only the 100 most recent. Number to KEEP can be adjusted in the script. Use User-Scripts Plug-in to setup a cron schedule. Hope it helps someone! For the most current version, go to the simple-snapshot-zfs github repository. I will not update the code in this post. https://github.com/samssausages/simple-snapshot-zfs Sample output: Starting Snapshot 20221221-2349 _______________________________________ No changes detected in pool/dataset1. No snapshot created. Total snapshots for pool/dataset1: 1 Space used by snapshots for pool/dataset1: 0B _______________________________________ Snapshot created: pool/dataset2@20221221-2349 Total snapshots for pool/dataset2: 6 Space used by snapshots for pool/dataset2: 8.06M _______________________________________ No changes detected in pool/dataset3. No snapshot created. Total snapshots for pool/dataset3: 3 Space used by snapshots for pool/dataset3: 947K _______________________________________ No changes detected in pool/dataset4. No snapshot created. Total snapshots for pool/dataset4: 4 Space used by snapshots for pool/dataset4: 17.5G _______________________________________ No changes detected in pool/dataset4/nextcloud. No snapshot created. Total snapshots for pool/dataset4/nextcloud: 2 Space used by snapshots for pool/dataset4/nextcloud: 20.0G _______________________________________ -----------------Done!----------------- Code: #!/bin/bash #v0.9 ########################simple-snapshot-zfs####################### ###################### User Defined Options ###################### # List your ZFS datasets. You can add/remove sets # readarray -t DATASETS < <(zfs list -o name -H) # when replacing DATASETS below, should return all pools/Datasets, not thoroughly tested yet. If you test this let me know! DATASETS=("workpool/admin" "workpool/archive") # Set Number of Snapshots to Keep SHANPSHOT_QTY=100 # Snapshot Name Format SNAPSHOTNAME=$(date "+simplesnap_%Y-%m-%d-%H:%M:%S") ###### Don't change below unless you know what you're doing ###### ################################################################## echo "Starting Snapshot ${SNAPSHOTNAME}" echo "_____________________________________________________________" ## Validation Steps # Validate SHANPSHOT_QTY if ! [[ $SHANPSHOT_QTY =~ ^[0-9]+$ ]]; then echo "Error: SHANPSHOT_QTY must be a positive integer." exit 1 fi # Validate DATASETS for dataset in "${DATASETS[@]}"; do if ! zfs list "$dataset" &>/dev/null; then echo "Error: Dataset '$dataset' does not exist." exit 1 fi done # Function to handle errors handle_error() { local error_message="$1" echo "Error: $error_message" exit 1 } # Function to create snapshot if there is changed data create_snapshot_if_changed() { local DATASET="$1" local WRITTEN WRITTEN=$(zfs get -H -o value written "${DATASET}") if [[ "${WRITTEN}" != "0" ]]; then if ! zfs snapshot "${DATASET}@${SNAPSHOTNAME}"; then handle_error "Failed to create snapshot for ${DATASET}" fi echo "Snapshot created: ${DATASET}@${SNAPSHOTNAME}" else echo "No changes detected in ${DATASET}. No snapshot created." fi } # Function to prune snapshots prune_snapshots() { local DATASET="$1" local KEEP="${SHANPSHOT_QTY}" # Declare the SNAPSHOTS array declare -a SNAPSHOTS # Use mapfile to split the output into the SNAPSHOTS array if ! mapfile -t SNAPSHOTS < <(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@"); then echo "Error: Failed to list snapshots for ${DATASET}" return 1 fi local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]} echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}" local SNAPSHOTS_SPACE if ! SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}"); then echo "Error: Failed to get space used by snapshots for ${DATASET}" return 1 fi echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}" if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP)) for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do if ! zfs destroy "${i}"; then echo "Error: Failed to delete snapshot: ${i}" return 1 fi echo "Deleted snapshot: ${i}" echo "_____________________________________________________________" done else echo "_____________________________________________________________" fi } # Iterate over each dataset and call the functions for dataset in "${DATASETS[@]}"; do create_snapshot_if_changed "${dataset}" prune_snapshots "${dataset}" done echo "----------------------------Done!----------------------------" Edited October 17, 20232 yr by samsausages
July 20, 20232 yr hey would this script automatically show "Shadow Copies" on windows for those unraid shares whose disks are zfs formatted individually (not a zpool of many disks)?
July 21, 20232 yr Author On 7/20/2023 at 6:08 PM, TheLinuxGuy said: hey would this script automatically show "Shadow Copies" on windows for those unraid shares whose disks are zfs formatted individually (not a zpool of many disks)? You need to configure that, part of the config is in Samba. Instructions are here: https://forum.level1techs.com/t/zfs-on-unraid-lets-do-it-bonus-shadowcopy-setup-guide-project/148764 I didn't add anything to my script that would deal with shadowcopies on windows. But it may still work for that purpose, I haven't reviewed that article in a few years. But if I recall the change is mainly about the naming convention the script uses matching the Samba settings, so you would most likely only need to make that part match. EDIT: I recently did this config for windows and it's all done in the SMB config file on unraid. After proper config in that file, it will show shadow copies when browsing that share. Edited October 17, 20232 yr by samsausages
October 17, 20232 yr Author Updated to v0.9. Lots of upgrades to make it more durable and robust. Mainly error checks and validation checks.
December 25, 20232 yr This script works quite well. I am running it with userscripts as a cron task to do ZFS snapshots. Thankyou for your efforts with it. Given the current way the script handles purging old snapshots based on the number of snapshots would it be possible to purge by age. eg. only keep snapshots less than 30 days old? I have been looking how i may do this alteration myself and found an easy way to get the creation time. so if its not something you have considered then fingers crossed >zfs list -t snapshot -p -o name,creation -s creation -r "${DATASET}" Returns Snapshot Name and the creation time (as a unix timestamp)
June 7, 20242 yr I amended this script to do a recursive look at all datasets in a given parent. For me I just need the purge function so I have commented out the snapshot part for my purposes: #!/bin/bash #v0.9.1 ########################simple-snapshot-zfs####################### ###################### User Defined Options ###################### # List your ZFS datasets. You can add/remove sets # readarray -t DATASETS < <(zfs list -o name -H) # when replacing DATASETS below, should return all pools/Datasets, not thoroughly tested yet. If you test this let me know! DATASET=("zfsbackup/cachebackup") mapfile -t DATASETS < <(zfs list -r -o name -H "${DATASET}") # Set Number of Snapshots to Keep SHANPSHOT_QTY=24 # Snapshot Name Format SNAPSHOTNAME=$(date "+simplesnap_%Y-%m-%d-%H:%M:%S") ###### Don't change below unless you know what you're doing ###### ################################################################## echo "Starting Snapshot ${SNAPSHOTNAME}" echo "_____________________________________________________________" ## Validation Steps # Validate SHANPSHOT_QTY if ! [[ $SHANPSHOT_QTY =~ ^[0-9]+$ ]]; then echo "Error: SHANPSHOT_QTY must be a positive integer." exit 1 fi # Validate DATASETS for dataset in "${DATASETS[@]}"; do if ! zfs list "$dataset" &>/dev/null; then echo "Error: Dataset '$dataset' does not exist." exit 1 fi done # Function to handle errors handle_error() { local error_message="$1" echo "Error: $error_message" exit 1 } # Function to create snapshot if there is changed data create_snapshot_if_changed() { local DATASET="$1" local WRITTEN WRITTEN=$(zfs get -H -o value written "${DATASET}") if [[ "${WRITTEN}" != "0" ]]; then if ! zfs snapshot "${DATASET}@${SNAPSHOTNAME}"; then handle_error "Failed to create snapshot for ${DATASET}" fi echo "Snapshot created: ${DATASET}@${SNAPSHOTNAME}" else echo "No changes detected in ${DATASET}. No snapshot created." fi } # Function to prune snapshots prune_snapshots() { local DATASET="$1" local KEEP="${SHANPSHOT_QTY}" # Declare the SNAPSHOTS array declare -a SNAPSHOTS # Use mapfile to split the output into the SNAPSHOTS array if ! mapfile -t SNAPSHOTS < <(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@"); then echo "Error: Failed to list snapshots for ${DATASET}" return 1 fi local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]} echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}" local SNAPSHOTS_SPACE if ! SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}"); then echo "Error: Failed to get space used by snapshots for ${DATASET}" return 1 fi echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}" if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP)) for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do if ! zfs destroy "${i}"; then echo "Error: Failed to delete snapshot: ${i}" return 1 fi echo "Deleted snapshot: ${i}" echo "_____________________________________________________________" done else echo "_____________________________________________________________" fi } # Iterate over each dataset and call the functions for dataset in "${DATASETS[@]}"; do # create_snapshot_if_changed "${dataset}" prune_snapshots "${dataset}" done echo "----------------------------Done!----------------------------" Hope this helps someone.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.