SeattleBandit Posted July 31, 2020 Share Posted July 31, 2020 so i am rsync'ing my entire array to a back up unraid box - i tried to do it initailly by mounting the unraid share in my primary unraid server and then just rsync-ing wiht in my primary server - but rsync hung and stopped working so i nixed it and switched to rsync'ing via ssh and it worked like a charm ... i mean it took like two plus days to do the back up but it worked. now i'm running rsync again on my master to push a fresh update via the smb share --- again it got half way through (ish) and crashed setting up a recurring job via ssh would be a pita because unraid doesn't retain ssh keys between boots and i'm lazy and i dont' want to do ssh-key-add or whatever it is has anyonen else had issues with rsync'ing lots of data across a smb share? i don't want to use a backup docker app because i don't want archived or encrypted local backup - i just want a slave Quote Link to comment
Energen Posted July 31, 2020 Share Posted July 31, 2020 15 minutes ago, SeattleBandit said: i don't want to use a backup docker app because i don't want archived or encrypted local backup - i just want a slave It doesn't need to be either of those things. You can use a docker like Syncthing to make a 1:1 clone and a lot easier and more reliable than it seems to be using rsync manually. I'm sure there are others, maybe like Duplicati, to do the same thing. Quote Link to comment
SeattleBandit Posted August 1, 2020 Author Share Posted August 1, 2020 6 hours ago, Energen said: It doesn't need to be either of those things. You can use a docker like Syncthing to make a 1:1 clone and a lot easier and more reliable than it seems to be using rsync manually. I'm sure there are others, maybe like Duplicati, to do the same thing. tried duplicati - it archives stuff and compresses it syncthing crapped the bed when i tried to set it up for my entire array any other suggestions? Quote Link to comment
testdasi Posted August 1, 2020 Share Posted August 1, 2020 12 hours ago, SeattleBandit said: ... i'm lazy ... There lies the problem. Laziness works fine with small number of files. It tends to crap the bed when things become too big to deal with. Try rsync a small section of the server each type (e.g. one share at a time) to identify the share that tends to cause issue and then break it into smaller chunks. If something as barebone as rsync hangs then there isn't quite any other solution out there that would work properly. Quote Link to comment
superloopy1 Posted August 1, 2020 Share Posted August 1, 2020 I'd wait for Hoopster to chip in here. He will offer a full blown rsync solution complete with SSH keys and all ...Sent from my CLT-L09 using Tapatalk Quote Link to comment
Energen Posted August 1, 2020 Share Posted August 1, 2020 7 hours ago, SeattleBandit said: syncthing crapped the bed when i tried to set it up for my entire array I used 2 syncthing dockers simultaneously as the "source" (unraid server) and as the "destination" (external usb client via unassigned devices) to back up entire shares and did not have any problems with that... syncthing is a little bit of a hassle to set up properly with the shared folders and all that but once it's going.. it goes. I currently use it to pull files from an internet server, similar to ftp... so that I'm always receiving new files. I would personally suggest that you look at syncthing again and try to get it set up because I think it's going to be the most reliable for you.. Note that it does take quite a while for syncthing to scan and build a database for an entire array -- perhaps that's where you were not giving it enough time.. when I first did this it easily took more than a day to scan everything up. It will not begin transferring data until the initial database is built. Quote Link to comment
testdasi Posted August 1, 2020 Share Posted August 1, 2020 6 minutes ago, Energen said: I would personally suggest that you look at syncthing again and try to get it set up because I think it's going to be the most reliable for you.. The guy proudly proclaimed he's lazy. There isn't a software that would solve laziness. Quote Link to comment
superloopy1 Posted August 1, 2020 Share Posted August 1, 2020 I tried synching and a real pita it was too. I even registered on the ST forum, asked a couple if questions and was told that it wasn't really designed to be used in the way I wanted use a regular backup to another machine. I have a backup unraid server which I wanted to power up, say weekly, and rsync changes with. Hoopster's your man ...Sent from my CLT-L09 using Tapatalk Quote Link to comment
SeattleBandit Posted August 1, 2020 Author Share Posted August 1, 2020 49 minutes ago, superloopy1 said: I tried synching and a real pita it was too. I even registered on the ST forum, asked a couple if questions and was told that it wasn't really designed to be used in the way I wanted use a regular backup to another machine. I have a backup unraid server which I wanted to power up, say weekly, and rsync changes with. Hoopster's your man ... Sent from my CLT-L09 using Tapatalk thats basically my use case - either daily syncs or weekly syncs - in the interim i'll just rsync the individual shares because it seems to lose its shit when backing up my plex meta stuff - i acutally kicked off a rsync for the entire array last night via ssh and let it run all night (forgot to add the 'time' command so i don't know how long it took) but it actually completed - maybe rsync just needed to relax and lets its emotions go to finish the sync. id love to script it but i haven't found a good solution for ssh keys yet - i've looked at the user scripts plugin (basically a front end for cron it seems like) but the few scripts i've put together to run on a schedule don't seem to work the way i expect them too. i'm eager to hear back from this mythical 'hoopster' Quote Link to comment
superloopy1 Posted August 1, 2020 Share Posted August 1, 2020 Drop him a pm, he's v.helpful.Sent from my CLT-L09 using Tapatalk Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.