October 31, 2025Oct 31 Looking for advice on the concept of running Cold and Hot Storage on a single Unraid box.I currently have a Dell XC630 (R630 for HCI) head sever running Unraid, connected to a 60 bay Dell MD3060e with about 24 HDD’s installed. I’m currently running Unraid with no array, but solely running 3 zpools (2 zpools on HDD on the 60 bay JBOD and 1 NVME Zpool).The energy cost and heat output of the MD3060e is not longer acceptable, so I’m looking to deploy 2 NetApp NAJ-1502 12 bay SAS3 units and would like to migrate the 2 HDD zpools from the Dell MD3060e to the NetApp units and run those as the primary (HOT) Storage JBODS and run the MD3060e as a auxiliary (Cold) JBOD for say on-site replica’s once a month or so.My question is in regards to running Unraid with the Cold Storage unit with minimal downtime. My theory is the Hot Storage 24x7 operation with the Cold Storage powered on at time of sync. Can I within Unraid stop the “array” and poweroff the Cold Storage unit, then power up the “Array” and run solely the Hot Storage or would I have to power off the entire set?Hope this makes to anyone. Thanks in advance.
October 31, 2025Oct 31 Community Expert Solution unasigned disk. zfs via terminal. use of zfs send recieve... This is iondeed posible. I find ti easier to use teh zfs send recvie when zfs shares the same pool name...You can do this cleanly without taking the Unraid host down. The key is to treat the “cold” shelf as a ZFS pool that you export before you power it off and import only when you need it. With Unraid 6.12+ ZFS integration, you don’t need to stop the whole “array” to do that—just don’t let the cold pool auto-mount at array start.Define the layout (If I'm reading this corectly...)Host: Dell XC630 (HBA → external SAS) running Unraid (no parity array).Hot storage: Two NetApp NAJ-1502 12-bay SAS3 JBODs → your two HDD ZFS pools (24/7).Fast tier: NVMe ZFS pool (24/7).Cold storage: Dell MD3060e (SAS2 60-bay) → one ZFS pool used as a monthly on-site replica (powered OFF except during sync windows).Unraid behavior you’ll rely on?ZFS pools can be managed independently of the Unraid “array.” You can:Disable automount on array start for the cold pool.zpool export the cold pool, power the shelf off, and keep the server up.Later power the shelf on and zpool import the pool, so no host reboot needed.Docker/VMs: ensure nothing points to the cold pool path when it’s offline (disable any shares that include that pool, or use separate share names).FYI: Unraid doesn't support sas hotplug...Hardware handling (SAS JBOD etiquette)Before power-off: make sure the pool on that shelf is exported (not merely unmounted). Export tells ZFS “these vdevs will disappear,” avoiding error spam and pool faults.Power sequencing: generally power JBOD(s) on first, wait 30–60s for link up, then import (host can already be running). For power-off, export → power down the shelf.you will need a onetime migration to send the data form your current setup to the new ready to go system...One-time migration from MD3060e → NetApp shelvesAssume:Old hot pools on MD3060e: tankA_old, tankB_old.New hot pools on NetApp: tankA, tankB.You’ll move datasets via snapshot + zfs send | zfs recv.Example scripts (using zfs send receive)https://openzfs.github.io/openzfs-docs/man/master/8/zfs-send.8.html https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html Create the new pools (example raidz2 vdevs; adjust to your layout):# Example only — pick your desired vdev geometry zpool create -o ashift=12 \ -O compression=zstd-3 -O atime=off -O xattr=sa -O acltype=posixacl \ -O normalization=formD -O relatime=on -O dnodesize=auto \ tankA raidz2 /dev/disk/by-id/wwn-disk1 ... /dev/disk/by-id/wwn-disk8 zpool create -o ashift=12 \ -O compression=zstd-3 -O atime=off -O xattr=sa -O acltype=posixacl \ -O normalization=formD -O relatime=on -O dnodesize=auto \ tankB raidz2 /dev/disk/by-id/wwn-disk9 ... /dev/disk/by-id/wwn-disk16 Initial mirror copy:# Snapshot source zfs snapshot -r tankA_old@seed zfs snapshot -r tankB_old@seed # Full send (use -Lec for large/sparse/embedded, and pv if you want progress) zfs send -LecR tankA_old@seed | zfs recv -uF tankA # -u leaves datasets unmounted for review zfs send -LecR tankB_old@seed | zfs recv -uF tankB # Review properties/mountpoints, then mount zfs mount -a Incremental catch-up right before cutover:zfs snapshot -r tankA_old@cutover zfs snapshot -r tankB_old@cutover zfs send -LecR -I @seed tankA_old@cutover | zfs recv -F tankA zfs send -LecR -I @seed tankB_old@cutover | zfs recv -F tankB Point shares to the new pools (update Unraid Shares to include/exclude pools as you like), then export the old pools and repurpose those disks if desired.Now with data on the system...Standing up the cold pool on the MD3060eCreate a single cold mirror/raidz pool (e.g., cold), tuned for capacity and resiliency (often raidz2/3). Same property set as above (zstd-3, atime=off, xattr=sa, acltype=posixacl). Consider setting:zpool set autoreplace=on cold zpool set autoexpand=on cold Monthly cold-sync workflow (no host downtime)Setup once (recommended tooling):Use Sanoid/Syncoid (Community Apps plugin on Unraid) or plain ZFS send/recv scripts. Syncoid is great for one-liner replications and pruning.Make the cold pool not automount at array start in Unraid (ZFS pool settings) so the host boots cleanly with the shelf off.Each month:Power ON MD3060e → wait for SAS links.On Unraid: import the pool:zpool import zpool import cold Replicate hot → cold (examples with syncoid):# replicate whole pools or specific datasets; --no-sync-snap if you manage snaps yourself syncoid --recursive --compress=mbuffer tankA cold/tankA syncoid --recursive --compress=mbuffer tankB cold/tankB Or raw ZFS (per dataset):SNAP=$(date +%Y%m%d) zfs snapshot -r tankA@${SNAP} zfs snapshot -r tankB@${SNAP} zfs send -LecR -I @prev tankA@${SNAP} | zfs recv -F cold/tankA zfs send -LecR -I @prev tankB@${SNAP} | zfs recv -F cold/tankB # (Keep track of the last snapshot name per dataset; you can use bookmarks too.) Scrub the cold pool occasionally:zpool scrub cold Export and power off:zpool export cold # now safe to power down the MD3060e (Export first, then power the shelf off. The Unraid host stays up the whole time.)^ -- Most can be automated!...Share & path hygieneKeep hot shares and cold replicas separated (e.g., /mnt/tankA/..., /mnt/tankB/... vs /mnt/cold/...).Do not let Docker/VMs or SMB/NFS exports reference /mnt/cold/... paths (or mark those shares as Export: No in Unraid) so nothing breaks when the shelf is offline.For SMB, you can expose read-only replicas on demand when the shelf is online.Health & monitoring tipsEnable and review smartd on the HBAs/shelves; NetApp SAS3 shelves usually expose SMART through the expander (check that your HBA is in IT mode).Schedule zpool status, zpool scrub, and alerting.Consider zfs set reservation on cold datasets if you want to prevent accidental over-filling during replication.Q/A: Direct questions:Question: Can I stop the “array” and power off only the cold shelf?You don’t need to stop the host or the whole Unraid array. Just:Ensure the cold pool is not set to automount on array start.When finished syncing: zpool export cold → power OFF the MD3060e.When ready to sync again: power ON the MD3060e → zpool import cold → replicate → zpool export cold → power OFF.The hot NetApp shelves and the NVMe pool keep running 24×7—minimal to zero downtime for the Unraid server and your hot storage....That is my understanding...
October 31, 2025Oct 31 Community Expert zfs send/recieve script for automation (As i do something similar...):Here’s a copy-paste Bash script you can drop on your Unraid box. It supports either syncoid (preferred, if installed) or a pure ZFS send/recv fallback. It’s built for your “hot → cold” workflow, with variables at the top to switch pools/datasets, safe checks, logging, optional e-mail, and an optional auto-export of the cold pool when done.*Requires user script plugin to automate...cold_sync.sh#!/usr/bin/env bash set -euo pipefail # =========================[ CONFIGURE ME ]========================= # Which pool is the COLD replica pool (lives on the MD3060e) COLD_POOL="cold" # Your HOT sources. You can replicate whole pools or specific datasets. # - If REPL_DATASETS is empty, we replicate each dataset in REPL_POOLS (root datasets). # - If REPL_DATASETS is non-empty, we replicate only those datasets. REPL_POOLS=("tankA" "tankB") # e.g., whole pools on NetApp shelves REPL_DATASETS=() # e.g., ("tankA/media" "tankB/projects"); leave empty to do pools # Destination layout: cold/<source-root> # Example: tankA -> cold/tankA, tankB/media -> cold/tankB/media DEST_ROOT_PREFIX="${COLD_POOL}" # Tooling preferences PREFER_SYNCOID=true # Use syncoid if present; fallback to raw zfs if not USE_MBUFFER_IF_PRESENT=true # Try to use mbuffer if available (benefits long pipes) # Snapshot naming (used by the raw ZFS fallback) SNAP_PREFIX="sync" DATE_FMT="+%Y%m%d-%H%M%S" # Logging & notifications LOG_DIR="/var/log/cold-sync" LOG_FILE="${LOG_DIR}/cold-sync-$(date +%F).log" ROTATE_KEEP=14 # Keep N most-recent log files EMAIL_TO="" # e.g., "[email protected]" (requires /usr/bin/mailx or /bin/mail) EMAIL_SUBJ_PREFIX="[Unraid Cold Sync]" # Safety / behavior AUTO_IMPORT_IF_SEEN=true # If cold pool is visible but not imported, import it AUTO_EXPORT_WHEN_DONE=false # Export cold pool after replication completes successfully ABORT_IF_COLD_OFFLINE=true # If cold pool not present, bail out (instead of continuing on hot only) DRY_RUN=false # If true, show plan, don't replicate # Lock file to avoid concurrent runs LOCK_FILE="/var/run/cold-sync.lock" # ================================================================ # ---- helpers ---------------------------------------------------- _ts(){ date "+%Y-%m-%d %H:%M:%S"; } log(){ echo "[$(_ts)] $*" | tee -a "$LOG_FILE"; } die(){ log "ERROR: $*"; exit 1; } have(){ command -v "$1" >/dev/null 2>&1; } normalize_list(){ # Build a unique list of datasets to replicate from REPL_POOLS and REPL_DATASETS local out=() if ((${#REPL_DATASETS[@]}==0)); then # replicate root of each pool for p in "${REPL_POOLS[@]}"; do out+=("$p") done else out=("${REPL_DATASETS[@]}") fi printf "%s\n" "${out[@]}" | awk 'NF' | sort -u } find_latest_snap(){ # Find the latest snapshot on $1 that starts with ${SNAP_PREFIX}- local ds="$1" zfs list -H -t snapshot -o name -S creation -r "$ds" 2>/dev/null \ | awk -v pfx="@${SNAP_PREFIX}-" -F@ '$2 ~ "^"pfx {print $0; exit}' || true } ensure_dirs(){ mkdir -p "$LOG_DIR" } rotate_logs(){ ls -1t "$LOG_DIR"/cold-sync-*.log 2>/dev/null | tail -n +$((ROTATE_KEEP+1)) | xargs -r rm -f } email_summary(){ local subject="${EMAIL_SUBJ_PREFIX} $(hostname) $(date +%F)" [[ -z "$EMAIL_TO" ]] && return 0 if have mailx; then mailx -s "$subject" "$EMAIL_TO" < "$LOG_FILE" || true elif have mail; then mail -s "$subject" "$EMAIL_TO" < "$LOG_FILE" || true else log "NOTE: No mailx/mail; skipping email to $EMAIL_TO" fi } auto_import_cold_if_possible(){ # If cold pool not imported but visible, import it with -N (don’t mount) then mount if ! zpool list -H -o name | grep -qx "$COLD_POOL"; then if zpool import | grep -q "pool: ${COLD_POOL}"; then $AUTO_IMPORT_IF_SEEN || return 0 log "Cold pool '$COLD_POOL' visible but not imported; importing..." zpool import -N "$COLD_POOL" zfs mount -a fi fi } mbuffer_flag(){ if $USE_MBUFFER_IF_PRESENT && have mbuffer; then echo "--compress=mbuffer" else echo "" fi } syncoid_replicate(){ local src="$1" local dst="${DEST_ROOT_PREFIX}/${src}" local mbflag; mbflag="$(mbuffer_flag)" # syncoid creates sync snapshots and handles incrementals automatically # -r recursive, -f continues on errors (optional), -C do a resume if supported local cmd=(syncoid -r -C) [[ -n "$mbflag" ]] && cmd+=("$mbflag") cmd+=("$src" "$dst") if $DRY_RUN; then log "[DRY] ${cmd[*]}" return 0 fi log "syncoid: $src -> $dst" "${cmd[@]}" } raw_send_recv(){ local src="$1" local dst="${DEST_ROOT_PREFIX}/${src}" local stamp snap curr latest stamp="$(date "$DATE_FMT")" snap="${src}@${SNAP_PREFIX}-${stamp}" # create a new recursive snapshot if $DRY_RUN; then log "[DRY] zfs snapshot -r $snap" else zfs snapshot -r "$snap" fi # Make sure destination parent exists if ! zfs list -H -o name "$DEST_ROOT_PREFIX" >/dev/null 2>&1; then $DRY_RUN || zfs create -o mountpoint=legacy "$DEST_ROOT_PREFIX" log "Created destination root dataset: $DEST_ROOT_PREFIX" fi # If dst does not exist, create placeholder (recv will -uF anyway) if ! zfs list -H -o name "$dst" >/dev/null 2>&1; then if $DRY_RUN; then log "[DRY] zfs create -p -o mountpoint=legacy $dst" else zfs create -p -o mountpoint=legacy "$dst" fi fi # Find latest previous sync snapshot on source that also exists on destination (by name) latest="$(find_latest_snap "$src")" local send_cmd recv_cmd if [[ -n "$latest" ]]; then # Incremental from latest to new send_cmd=(zfs send -LecR -I "${latest##*@}" "$snap") else # Full send send_cmd=(zfs send -LecR "$snap") fi recv_cmd=(zfs recv -uF "$dst") if $DRY_RUN; then log "[DRY] ${send_cmd[*]} | ${recv_cmd[*]}" return 0 fi log "zfs send/recv: $src -> $dst (from: ${latest:-FULL})" # Use mbuffer if available if $USE_MBUFFER_IF_PRESENT && have mbuffer; then "${send_cmd[@]}" | mbuffer | "${recv_cmd[@]}" else "${send_cmd[@]}" | "${recv_cmd[@]}" fi # Optionally mount new datasets zfs mount -a } main(){ ensure_dirs rotate_logs # Lock exec 9>"$LOCK_FILE" if ! flock -n 9; then die "Another cold-sync is running (lock: $LOCK_FILE)." fi log "=== Cold sync start ===" log "Host: $(hostname) | Cold pool: $COLD_POOL | DRY_RUN=$DRY_RUN" # Verify hot sources exist local src_list; mapfile -t src_list < <(normalize_list) ((${#src_list[@]})) || die "No sources defined (REPL_POOLS/REPL_DATASETS)." for s in "${src_list[@]}"; do zfs list -H -o name "$s" >/dev/null 2>&1 || die "Source dataset not found: $s" done # Ensure cold pool is online or importable auto_import_cold_if_possible if ! zpool list -H -o name | grep -qx "$COLD_POOL"; then if $ABORT_IF_COLD_OFFLINE; then die "Cold pool '$COLD_POOL' is not imported. Power on the shelf and import it first." else log "WARNING: Cold pool '$COLD_POOL' not imported; continuing (no replication will be done)." email_summary exit 0 fi fi # Choose engine local use_syncoid=false if $PREFER_SYNCOID && have syncoid; then use_syncoid=true log "Using syncoid for replication." else log "Using raw zfs send/recv fallback." fi # Replicate each source local ok=0 for src in "${src_list[@]}"; do if $use_syncoid; then if syncoid_replicate "$src"; then log "OK: $src" else log "FAIL: $src" ok=1 fi else if raw_send_recv "$src"; then log "OK: $src" else log "FAIL: $src" ok=1 fi fi done # Optionally export the cold pool if $AUTO_EXPORT_WHEN_DONE && ! $DRY_RUN; then log "Exporting cold pool '$COLD_POOL' as requested..." zpool export "$COLD_POOL" || log "WARN: export of $COLD_POOL failed." fi if (( ok == 0 )); then log "=== Cold sync completed successfully ===" else log "=== Cold sync completed with errors (see above) ===" fi email_summary exit "$ok" } main "$@" How it behaves (quick notes)Configure at top: set COLD_POOL, your REPL_POOLS (e.g., tankA, tankB) or list specific REPL_DATASETS, pick PREFER_SYNCOID=true/false, and adjust logging/email.Safety checks: verifies sources exist; ensures the cold pool is imported (can auto-import if visible).Syncoid first: if syncoid exists, it handles snapshots/incrementals automatically. If not, the script falls back to raw zfs send/recv with its own snapshot naming (sync-YYYYmmdd-HHMMSS) and incrementals.mbuffer: used automatically if present.Locking: prevents overlapping runs with flock.Optional auto-export: set AUTO_EXPORT_WHEN_DONE=true if you want the script to export the cold pool when done (so you can safely power the shelf down immediately after).
November 1, 2025Nov 1 Author On 10/31/2025 at 11:35 AM, bmartino1 said:unasigned disk. zfs via terminal. use of zfs send recieve... This is iondeed posible. I find ti easier to use teh zfs send recvie when zfs shares the same pool name...You can do this cleanly without taking the Unraid host down. The key is to treat the “cold” shelf as a ZFS pool that you export before you power it off and import only when you need it. With Unraid 6.12+ ZFS integration, you don’t need to stop the whole “array” to do that—just don’t let the cold pool auto-mount at array start.Define the layout (If I'm reading this corectly...)Host: Dell XC630 (HBA → external SAS) running Unraid (no parity array).Hot storage: Two NetApp NAJ-1502 12-bay SAS3 JBODs → your two HDD ZFS pools (24/7).Fast tier: NVMe ZFS pool (24/7).Cold storage: Dell MD3060e (SAS2 60-bay) → one ZFS pool used as a monthly on-site replica (powered OFF except during sync windows).Unraid behavior you’ll rely on?ZFS pools can be managed independently of the Unraid “array.” You can:Disable automount on array start for the cold pool.zpool export the cold pool, power the shelf off, and keep the server up.Later power the shelf on and zpool import the pool, so no host reboot needed.Docker/VMs: ensure nothing points to the cold pool path when it’s offline (disable any shares that include that pool, or use separate share names).FYI: Unraid doesn't support sas hotplug...Hardware handling (SAS JBOD etiquette)Before power-off: make sure the pool on that shelf is exported (not merely unmounted). Export tells ZFS “these vdevs will disappear,” avoiding error spam and pool faults.Power sequencing: generally power JBOD(s) on first, wait 30–60s for link up, then import (host can already be running). For power-off, export → power down the shelf.you will need a onetime migration to send the data form your current setup to the new ready to go system...One-time migration from MD3060e → NetApp shelvesAssume:Old hot pools on MD3060e: tankA_old, tankB_old.New hot pools on NetApp: tankA, tankB.You’ll move datasets via snapshot + zfs send | zfs recv.Example scripts (using zfs send receive)https://openzfs.github.io/openzfs-docs/man/master/8/zfs-send.8.htmlhttps://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html Create the new pools (example raidz2 vdevs; adjust to your layout):# Example only — pick your desired vdev geometry zpool create -o ashift=12 \ -O compression=zstd-3 -O atime=off -O xattr=sa -O acltype=posixacl \ -O normalization=formD -O relatime=on -O dnodesize=auto \ tankA raidz2 /dev/disk/by-id/wwn-disk1 ... /dev/disk/by-id/wwn-disk8 zpool create -o ashift=12 \ -O compression=zstd-3 -O atime=off -O xattr=sa -O acltype=posixacl \ -O normalization=formD -O relatime=on -O dnodesize=auto \ tankB raidz2 /dev/disk/by-id/wwn-disk9 ... /dev/disk/by-id/wwn-disk16 Initial mirror copy:# Snapshot source zfs snapshot -r tankA_old@seed zfs snapshot -r tankB_old@seed # Full send (use -Lec for large/sparse/embedded, and pv if you want progress) zfs send -LecR tankA_old@seed | zfs recv -uF tankA # -u leaves datasets unmounted for review zfs send -LecR tankB_old@seed | zfs recv -uF tankB # Review properties/mountpoints, then mount zfs mount -a Incremental catch-up right before cutover:zfs snapshot -r tankA_old@cutover zfs snapshot -r tankB_old@cutover zfs send -LecR -I @seed tankA_old@cutover | zfs recv -F tankA zfs send -LecR -I @seed tankB_old@cutover | zfs recv -F tankB Point shares to the new pools (update Unraid Shares to include/exclude pools as you like), then export the old pools and repurpose those disks if desired.Now with data on the system...Standing up the cold pool on the MD3060eCreate a single cold mirror/raidz pool (e.g., cold), tuned for capacity and resiliency (often raidz2/3). Same property set as above (zstd-3, atime=off, xattr=sa, acltype=posixacl). Consider setting:zpool set autoreplace=on cold zpool set autoexpand=on cold Monthly cold-sync workflow (no host downtime)Setup once (recommended tooling):Use Sanoid/Syncoid (Community Apps plugin on Unraid) or plain ZFS send/recv scripts. Syncoid is great for one-liner replications and pruning.Make the cold pool not automount at array start in Unraid (ZFS pool settings) so the host boots cleanly with the shelf off.Each month:Power ON MD3060e → wait for SAS links.On Unraid: import the pool:zpool import zpool import cold Replicate hot → cold (examples with syncoid):# replicate whole pools or specific datasets; --no-sync-snap if you manage snaps yourself syncoid --recursive --compress=mbuffer tankA cold/tankA syncoid --recursive --compress=mbuffer tankB cold/tankB Or raw ZFS (per dataset):SNAP=$(date +%Y%m%d) zfs snapshot -r tankA@${SNAP} zfs snapshot -r tankB@${SNAP} zfs send -LecR -I @prev tankA@${SNAP} | zfs recv -F cold/tankA zfs send -LecR -I @prev tankB@${SNAP} | zfs recv -F cold/tankB # (Keep track of the last snapshot name per dataset; you can use bookmarks too.)Scrub the cold pool occasionally:zpool scrub cold Export and power off:zpool export cold # now safe to power down the MD3060e(Export first, then power the shelf off. The Unraid host stays up the whole time.)^ -- Most can be automated!...Share & path hygieneKeep hot shares and cold replicas separated (e.g., /mnt/tankA/..., /mnt/tankB/... vs /mnt/cold/...).Do not let Docker/VMs or SMB/NFS exports reference /mnt/cold/... paths (or mark those shares as Export: No in Unraid) so nothing breaks when the shelf is offline.For SMB, you can expose read-only replicas on demand when the shelf is online.Health & monitoring tipsEnable and review smartd on the HBAs/shelves; NetApp SAS3 shelves usually expose SMART through the expander (check that your HBA is in IT mode).Schedule zpool status, zpool scrub, and alerting.Consider zfs set reservation on cold datasets if you want to prevent accidental over-filling during replication.Q/A: Direct questions:Question: Can I stop the “array” and power off only the cold shelf?You don’t need to stop the host or the whole Unraid array. Just:Ensure the cold pool is not set to automount on array start.When finished syncing: zpool export cold → power OFF the MD3060e.When ready to sync again: power ON the MD3060e → zpool import cold → replicate → zpool export cold → power OFF.The hot NetApp shelves and the NVMe pool keep running 24×7—minimal to zero downtime for the Unraid server and your hot storage....That is my understanding...This is perfect! And thanks for the script to automate the tasks. Long term this will come in handy as things get upgraded and expanded.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.