I would like to backup my cloud server on my local Unraid, eliminating the recurrent cost of backups provided by my cloud provider (which is Hetzner, btw).

 

A simple rsync script initiated by Unraid using a sudo user would probably be sufficient, but I would like to improve the solution a little:

 

1) Ideally, it would require no installation on the server, or a minimal client app.

2) Some retention would be nice, like keeping snapshots for the last six months and 10 days.

3) The ability to filter data folders, since I already have them on my local system and can recreate them very easily. Also, data is what takes more space on the server.

4) Finally, I'd like to minimize transfers over the network, just the changes since the last backup.

 

I'd appreciate any suggestions.

Edited April 21, 20251 yr by monate

there is no one application That I know of on Unraid that does this.

From a few unraid plugins to dockers applications this is achievable.

This also requires some prenotion of the unraid disk sizes and setups.

In example snapshots. I would use and prioritizes zfs and the built-in zfs feature.
-This would mean you would need a min of 3 disk the same size to make a pool disk array to use zfs.

 

With unraid 7 you can ditch the array and use pool only as example above.
cache disk used for unraid system .img files for dockers and vms and potentaily other plugins..

 

A few plugin to mention...

 

appdata backup plugin

user script plugin

Fix Common Problems plugin

ZFS Master Plugin

 

optional but helpful

SwapFile Plugin (adds a swap file on a btrfs partition)

Docker Compose Plugin (adds docker compose support unraid)

 

The main plugin/docker setup your looking at is rclone...

 

Then you may need additional services for webdav / web file / sftp/ftp access to connect to your unraid system...

I would use the rclone Washe plugin.

-Other options exist in the CA (sythinthing, seafile....)

 

My Reccomend option for unriad front facing secure file access
SFTP/FTP : 

SFTP-Fail2ban Docker

-Other options exist in the CA

Webdav / Website file access:

Rejetto HFS

-Other options exist in the CA

then with user script and appdata backup you can maintain backup of your current running docker applications and OS flash drive.

a user script to backup and make zfs snapshots.
 

#!/bin/bash

#v0.2
########################simple-snapshot-zfs#######################
###################### User Defined Options ######################

# List your ZFS datasets.  You can add/remove sets
DATASETS=("vm-zfs/Backups" "vm-zfs/Cloud" "vm-zfs/Users")
# readarray -t DATASETS < <(zfs list -o name -H)  # when replacing DATASETS above, should return all pools/Datasets, but not thouroughly tested yet

# Set Number of Snapshots to Keep
SHANPSHOT_QTY=12

###### Don't change below unless you know what you're doing ######
##################################################################

timestamp=$(date "+%Y-%m-%d-%H:%M")
echo "Starting Snapshot ${timestamp}"
echo "_____________________________________________________________"

# Function to create snapshot if there is changed data
create_snapshot_if_changed() {
  local DATASET="$1"
  local WRITTEN
  WRITTEN=$(zfs get -H -o value written "${DATASET}")

  if [[ "${WRITTEN}" != "0" ]]; then
    local TIMESTAMP
    TIMESTAMP="$(date '+%Y-%m-%d-%H%M')"
    zfs snapshot "${DATASET}@${TIMESTAMP}"
    echo "Snapshot created: ${DATASET}@${TIMESTAMP}"
  else
    echo "No changes detected in ${DATASET}. No snapshot created."
  fi
}

# Function to prune snapshots
prune_snapshots() {
  local DATASET="$1"
  local KEEP="${SHANPSHOT_QTY}"
  
  local SNAPSHOTS=( $(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@") )
  local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]}

  echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}"

  local SNAPSHOTS_SPACE
  SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}")
  echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}"

  if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then
    local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP))
    for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do
      zfs destroy "${i}"
      echo "Deleted snapshot: ${i}"
      echo "_____________________________________________________________"
    done
  else
   echo "_____________________________________________________________"
  fi
}

# Iterate over each dataset and call the functions
for dataset in "${DATASETS[@]}"; do
  create_snapshot_if_changed "${dataset}"
  prune_snapshots "${dataset}"
done

echo "----------------------------Done!----------------------------"

 

To back up your Hetzner cloud server to your local Unraid system while minimizing costs and maintaining control, here's a tailored solution addressing your requirements:

 

Key Requirements & Solutions

 

Minimal Installation on the Server

Utilize rsync over SSH, which is typically pre-installed on most Linux distributions. This approach requires no additional software on your Hetzner server. Set up SSH key-based authentication from your Unraid server to the cloud server to enable secure, passwordless access.

 

Retention with Snapshots

Implement the incbackup.sh script by mgutt, designed for Unraid. This script creates incremental backups using hardlinks, allowing for efficient storage. It supports customizable retention policies, such as keeping backups for the last 10 days and monthly snapshots for six months. The script integrates with Unraid's User Scripts plugin for scheduling and management.

 

see trash guides on hardlink setups:

https://trash-guides.info/File-and-Folder-Structure/

 

Selective Data Backup

Configure rsync to exclude directories that are easily re-creatable or already present on your local system. Use the --exclude option in your rsync command or specify exclusions in the incbackup.sh script to prevent unnecessary data transfer and storage.

 

Efficient Data Transfer

rsync inherently transfers only the differences between source and destination files, ensuring efficient use of bandwidth. The incbackup.sh script leverages this feature to maintain up-to-date backups with minimal data transfer.

 

So, How to implement:

So, I would highly recommend using the sftp-fail2ban docker. Unraid does have sftp via ssh, but the fail2ban is important to have to protect unraid from threats and attacks...

I also recommend Install and Configure incbackup.sh on Unraid:

Mgutt repository and forum post have some good script that we can use..

https://codeberg.org/mgutt/rsync-incremental-backup

 

User script plugin is helpful here. user script may need to copy the file off the usb which is located at /boot
Download the script from mgutt's repository and Place the script in a suitable directory on your Unraid server, e.g., /usr/local/bin/incbackup.sh
mark it as executable.... 

 

Configure the script with your desired source and destination paths, retention settings, and exclusion patterns.

cd /boot/config
wget https://codeberg.org/mgutt/rsync-incremental-backup/raw/branch/main/incbackup.sh
cp incbackup.sh /usr/local/bin/incbackup.sh
chmod +x /usr/local/bin/incbackup.sh

 

 

Security Considerations:

Ensure that SSH access to your Hetzner server is secured, preferably allowing only key-based authentication and disabling password login.

Regularly update your Unraid and Hetzner servers to patch any security vulnerabilities.

 

See other forum topics on this:

 

 

 

Edited April 21, 20251 yr by bmartino1
Data - typo

Hey bmartino,

 

Thank you very much for your detailed answer. The incremental backup script looks very promising.

 

I don't allow password access for SSH, only keys, and actually the root user is not even allowed to log in through SSH.

 

I plan to use a sudo user to do the backups. I guess that I should modify the sudoers file to allow this user to access root files without a password, and maybe modify the script to invoke rsync with sudo privileges. I will have to investigate if this is possible, but the script looks like a very good place to start.