Backup whole Storage

[HannesDi]

Hello,

after setting up my complete system with 6 hard disks + ssd, and installed all stuff the question: "how to backup ALL data?" comes up.

I have 4 hard disks (2x 8tb + 2x 3tb) in a separate housing that can be connected via usb3 or sata. 

All of these are recognized as single hard disks and appears in the area of unassigned devices and of course i can mount them manually so that they are available
as single backup storage targets.

But i want them as "one big" storage target with 22 tb. call it 1 big logical volume... 

Any Idea how this could be managed? Is UNRAID cappable to do this? 

Sure from nerdpack i could install unionfs and try this.. but is there not a easy way i could not see ?

Or is it needed to install a Linux VM and setup there a lvm?

 

Best Regards
Hannes

[testdasi]

1 hour ago, HannesDi said:

I have 4 hard disks (2x 8tb + 2x 3tb) in a separate housing that can be connected via usb3 or sata. 

If the drives can be connected via SATA (and show up as individual drives), why don't you just add all of them to the array?

Then set the shares up such that backup share will use these 4 drives exclusively and the other shares will exclude these 4.

That essentially sections your array into two and you don't need anything more than the GUI and built-in Unraid functionalities.

 

If they are connected via USB then it's a different story since it's a terrible idea to add USB drives into the array due to their tendency to drop offline for no reason.

In that case, you can have no choice but to use unionfs / mergerfs to combine the UD mounts into a single mount point. You can even set it up with User Script plugin to run the script to mount it at array start.

 

I would use a Linux VM only as a last resort most critically because creating a pool can be so easily done without the additional VM overhead.

[HannesDi]

8 minutes ago, testdasi said:

why don't you just add all of them to the array?

The Answer is Simple:

1. having the two hard disk sets for "main data" and the "backup data" in the same enclosure is not a backup for me.

 

In case of fire or water damage, burglary or vandalism all data is lost.

 

I want a real Backup because after the backup is finished, the hard drives will be detached and taken out of the office.

[testdasi]

2 minutes ago, HannesDi said:

The Answer is Simple:

1. having the two hard disk sets for "main data" and the "backup data" in the same enclosure is not a backup for me.

In case of fire or water damage, burglary or vandalism all data is lost.

I want a real Backup because after the backup is finished, the hard drives will be detached and taken out of the office.

Same as me then.

All you need is Unassigned Devices to automount the external and even trigger bash script + mergerfs to create a pool of the mounted UD drives + a bash script to install mergerfs and run commands.

It's quite straight-forward actually. That's how I utilised by external USB SSD's for offline backup.

[HannesDi]

6 hours ago, testdasi said:

+ a bash script to install mergerfs and run commands.

It's quite straight-forward actually. That's how I utilised by external USB SSD's for offline backup.

would and could you share your script with me? that would be really nice

[testdasi]

Something like the one below. I wrote it with inspiration from @DZMM rclone mount script so there's plenty of similarity.

Assumptions

• You have a share called mergerfs (which is where things will be mounted) that has cache = only.
• This was designed for 2 UD drives mounted under ud01 and ud02
• Under each UD drive, there's a folder called mergerfs, the content of which will be merged into the mergerfs mount
• Each UD drive also has a mountcheck file which is used to determine whether the drive has mounted

#!/bin/bash
mkdir -p /mnt/cache/mergerfs/build
mkdir -p /mnt/cache/mergerfs/mount
# check if mergerfs already installed
if [[ -f "/bin/mergerfs" ]]; then
	echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed"
else
# Build mergerfs binary
	echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now."
	mkdir -p /mnt/cache/mergerfs/build
	docker run -v /mnt/cache/mergerfs/build:/build --rm trapexit/mergerfs-static-build
	mv /mnt/cache/mergerfs/build/mergerfs /bin
fi
	
echo "INFO: Checking drives are mounted" | logger

if [[ -f "/mnt/cache/mergerfs/mount/mountdisk1" ]]; then
	echo "INFO: Mergerfs mount already created, exiting." | logger
	exit
else
	if [[ ! -f "/mnt/disks/ud01/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: UD disk 1 not mounted, exiting."
		fusermount -uz /mnt/cache/mergerfs/mount
		exit
	elif [[ ! -f "/mnt/disks/ud02/mountcheck" ]]; then
		echo "$(date "+%d.%m.%Y %T") INFO: UD disk 2 not mounted, exiting."
		fusermount -uz /mnt/cache/mergerfs/mount
		exit
	else
		echo "INFO: All USB drives mounted, creating Mergerfs mountpoint" | logger
		mkdir -p /mnt/cache/mergerfs/mount
		mergerfs /mnt/disks/ud01/mergerfs:/mnt/disks/ud02/mergerfs /mnt/cache/mergerfs/mount -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=mfs,cache.files=partial,dropcacheonclose=true,minfreespace=32G
		echo "INFO: mergerfs mountpoint created" | logger
	fi
fi

 

[sota]

See my sig.

[HannesDi]

Thank you for all the Help , i started made my solution on base of testdasi´s script and now i prepare my borg backup scripts to write on them