Backup application request

[Senord]

Hi everyone, 

 

Been using unraid on a basic fujitsu tower server, 6 x 1 tb disks i got from work(am IT person, mostly windows) for a good few years now, I find it really useful for storing stuff, running things overnight on vms etc. Most of the stuff on here or r/homelab makes me extra jealous, but that's another story. Most of the data on my array is either pictures, music or videos, so there are a variety of big files and lots of smaller ones if that impacts any recommendations.

 

Previously i had been using the duplicati docker to backup to a big usb drive(unassigned devices), but as i check the forums around i see a lot of people making recommendations to not use it, tbh i haven't had to use it in anger yet, nor have i done a test restore, so i'm not sure on how that would go. But i also know what people can be like when they take against something. Basically i'm looking for a relatively unbiased opinion on what to do moving forward.

 

So, since all i really need/have space for on the backup drive is a standard copy of whats on the array. Which is absolutely fine with me, no compression surely means it will be a bit easier on the processing power to do said compression(ofcourse trading off with an increased file size to transfer). I tried using user scripts plugin and several rsync lines(rsync -avzh /mnt/user/share /mnt/disks/SGT/rsync_backup/share ), which seemed fine, takes forever but that's usb writing for you.

 

The issue arises when i need to cancel the backup, i abort the user script, which suggests it has been aborted but i can see the (meagre)processor is getting hammered still, htop shows several of the rync lines still running, even using the kill command on the processes only calmed the processor down a bit and unraid wouldnt even restart. So this doesn't seem like its hugely fit for my purposes at least, i would prefer a docker app or something that wouldn't have these issues - unless someone can help with me with the script and i did something stupid.

 

TLDR:

 

i was using duplicati, then moved to rync user script, but trying to cancel it while running is a giant ballache. Recommend me a backup application to transfer to a local usb drive? or troubleshoot my crappy rsync script

 

 

 

Cheers everyone

 

 

[cameronconner]

I've noticed the same thing with that plugin. I wasn't sure if I was doing something wrong, or it was.

 

You need to call a

killall rsync

Maybe set up another user script to just be that?

 

Personally, here's my script. I'm not sure it's the best way to go, but it's working for now. I figure if the two parity disk rebuild fails me, I can clear the array and copy and merge everything together if needed... Maybe someone has a better way than our rsync ideas?

 

#!/bin/bash
#BKP DISK to BKP DISK.

rsync -avhP /mnt/disk3/ /mnt/disks/BKP_DISK3/bkp_3/ 2>&1 | tee /mnt/user/Storage/UNRAIDbkpSCRIPTS/log_disk3.txt

d=$(date +%Y-%m-%d)

echo $d >> /mnt/user/Storage/UNRAIDbkpSCRIPTS/lastrun-DISK3.txt
cp /mnt/user/Storage/UNRAIDbkpSCRIPTS/lastrun-DISK3.txt /mnt/disks/BKP_DISK3/lastrun-DISK3.txt

 

[IKWeb]

Personally at the moment I have a subscription for 10 bucks per month to CrashPlan pro and have around 4TB of data stored with them. (and a docker installed which manages it all)

 

That said, if I ever did come to do a restore man it would take like 2 days to get all that data back. 

 

I personally would love to see some kinda LTO support as I have a 24 tape autoloader in my cab, connected to my UNRAID server, that at the moment I cant use. 

It does shcok me (I am new to UNRAID 5 months in) the amount of people I read on these forums who use the UNRAID server as a backup device... Makes me want to bang my head on the wall... 

 

So yeah - I would like to see LTO support in UNRAID with some kinda web app / docker that aloows the selection of files to offload to the autoloader. 

Edited January 4, 2020 by IKWeb

[cameronconner]

I'm looking at setting up a 2nd Unraid box using the cheapest drives I can at a remote location for backups. 44TB ain't going to crash plan any time soon, even on 100meg upload.

[cameronconner]

I'm not versed with tape solutions, but couldn't you just set up a VM and pass thru the tape drive to an OS that can interact?

 

EDIT: Apologies, II think you meant the LTO comment as a 'warm storage' application, not cold backup, my bad.

Edited January 4, 2020 by cameronconner

[FreeMan]

2 minutes ago, IKWeb said:

It does shcok me (I am new to UNRAID 5 months in) the amount of people I read on these forums who use the UNRAID server as a backup device... Makes me want to bang my head on the wall... 

It's a perfectly fine NAS server - using a 2nd one as a backup for disk failure, or a 2nd one off-site as a fire/theft/tornado insurance seems perfectly reasonable. What do you see as the problem?

[IKWeb]

1 minute ago, FreeMan said:

It's a perfectly fine NAS server - using a 2nd one as a backup for disk failure, or a 2nd one off-site as a fire/theft/tornado insurance seems perfectly reasonable. What do you see as the problem?

What I see if people keeping files they have only one copy of on a NAS - Not just UNRAID and thinking its safe... Thats not the case. 

[FreeMan]

Just now, IKWeb said:

only one copy

Agree 100% with that statement! You'll see many, many veterans of the site warning people that parity protection is not a replacement for backups.

 

I thought you were condemning Unraid itself as a backup. 

[IKWeb]

3 minutes ago, FreeMan said:

Agree 100% with that statement! You'll see many, many veterans of the site warning people that parity protection is not a replacement for backups.

 

I thought you were condemning Unraid itself as a backup. 

Noooo - Not at all - UNRAID as a NAS (from what I have seen so far) is great. As my post up there ^^^ says I would want to see LTO support so I can backup my UNRAID box should it fail - and I have no doubt at some point it will. 

Yes I use CrashPlan - but for VM's and large files (30GB plus) CrashPlan isnt really a good option - where as my LTO8 autoloader would cover it with no issues. 

Edited January 4, 2020 by IKWeb

[Senord]

On 1/4/2020 at 10:08 PM, cameronconner said:

I've noticed the same thing with that plugin. I wasn't sure if I was doing something wrong, or it was.

 

You need to call a

killall rsync

Maybe set up another user script to just be that?

 

Personally, here's my script. I'm not sure it's the best way to go, but it's working for now. I figure if the two parity disk rebuild fails me, I can clear the array and copy and merge everything together if needed... Maybe someone has a better way than our rsync ideas?

 

#!/bin/bash
#BKP DISK to BKP DISK.

rsync -avhP /mnt/disk3/ /mnt/disks/BKP_DISK3/bkp_3/ 2>&1 | tee /mnt/user/Storage/UNRAIDbkpSCRIPTS/log_disk3.txt

d=$(date +%Y-%m-%d)

echo $d >> /mnt/user/Storage/UNRAIDbkpSCRIPTS/lastrun-DISK3.txt
cp /mnt/user/Storage/UNRAIDbkpSCRIPTS/lastrun-DISK3.txt /mnt/disks/BKP_DISK3/lastrun-DISK3.txt

 

Thanks for the reply mate, i've been working so it took till now to notice!

 

I absolutely will make another script with that, seems like the most straightforward option. i like straightforward. 

 

Having said that, always interested to hear new ideas and software, i definitely enjoyed the simple gui of duplicati. 

 

Let me know if you find anything!

[cameronconner]

I haven't tried Duplicati, but I was previously using a Mac with USB Drives, so I used to use a Mac app ChronoSync, which is a very nice GUI wrapper for rsync, basically. Copying everything over to UNRAID, I used rsync and got the syntax figured out, so decided "if it ain't broke, don't fix it" kind of thing. rsync!