Endy Posted December 6, 2017 Share Posted December 6, 2017 Thanks to some good deals around Thanksgiving, I was able to get an 8TB external hard drive to use for actual backups. Now I just need to figure out the best way of doing this. I know the easiest way would be to just copy everything over manually, but I would like something so that I can plug this drive in every week or so and and maybe just click a button or start a script and it then only copies over files that have changed and it might be a good idea if it could remove files that I may have deleted since the last time it was plugged in. What are my options for doing that? Quote Link to comment
Endy Posted December 6, 2017 Author Share Posted December 6, 2017 4 minutes ago, BRiT said: Unassigned Devices. I would say that's a given. It's what the best way to actually do the backing up that I am wondering about. Quote Link to comment
allanp81 Posted December 6, 2017 Share Posted December 6, 2017 User Scripts plugin and write yourself a simple rsync script. 1 Quote Link to comment
tdallen Posted December 6, 2017 Share Posted December 6, 2017 Since my server is difficult to access in a closet in the basement, I have a Windows based robocopy script that I execute from a pc/laptop. Works well. Quote Link to comment
Endy Posted December 7, 2017 Author Share Posted December 7, 2017 Does rsync have the ability to delete files from the backup if they are no longer on the live copy? Quote Link to comment
JorgeB Posted December 7, 2017 Share Posted December 7, 2017 3 hours ago, Endy said: Does rsync have the ability to delete files from the backup if they are no longer on the live copy? Yes, add --delete to the command. 1 Quote Link to comment
Endy Posted December 7, 2017 Author Share Posted December 7, 2017 Ok, so if I want to backup my Media share and I want it to delete any files that I may have deleted from the source and I also want to exclude a few directories from the backup, does this look right? I'm also not sure if I need to add anything to make it only backup files that aren't changed. rsync -delete --exclude='/mnt/user/Media/Movies/Blu-ray Movies' --exclude='/mnt/user/Media/TV/Blu-ray TV' --exclude='/mnt/user/Media/TV/DVD TV' /mnt/user/Media /mnt/disks/Seagate_Backup_Plus_Drive/Turtle To make it a user script I need to create a folder in the scripts folder on the flash drive and save the above command to a file called script and is that it? To backup my other shares just add their rsync commands to that file? Quote Link to comment
JorgeB Posted December 7, 2017 Share Posted December 7, 2017 You want to add -av, progress is also good and there's a missing - on delete: rsync -av --progress --delete 1 Quote Link to comment
allanp81 Posted December 7, 2017 Share Posted December 7, 2017 you can also add the -n option to do a dry run so you can see what changes it will make. 1 Quote Link to comment
JorgeB Posted December 7, 2017 Share Posted December 7, 2017 1 minute ago, allanp81 said: you can also add the -n option to do a dry run so you can see what changes it will make. Thanks, I was going to suggest that also and forgot Quote Link to comment
Endy Posted December 7, 2017 Author Share Posted December 7, 2017 So that would be -avn? (Archive, verbose, dry run?) Quote Link to comment
Endy Posted December 7, 2017 Author Share Posted December 7, 2017 Thank you Doing a test run right now. Quote Link to comment
Endy Posted December 7, 2017 Author Share Posted December 7, 2017 On 12/6/2017 at 10:40 AM, BRiT said: Unassigned Devices. I may not have understood the full extent of your suggestion. I didn't realize that UD can run scripts. Looks like it may even be possible to automatically have it run just by plugging the backup drive in which would be cool. I'll have to look into that more. Quote Link to comment
Endy Posted December 8, 2017 Author Share Posted December 8, 2017 Well, the excludes didn't work. Quote Link to comment
Endy Posted December 8, 2017 Author Share Posted December 8, 2017 For exclude, I was using --exclude='/mnt/user/Media/Movies/Blu-ray Movies' After some searching, should I be using this instead? --exclude 'mnt/user/Media/Movies/Blu-ray Movies' Quote Link to comment
allanp81 Posted December 8, 2017 Share Posted December 8, 2017 https://linux.die.net/man/1/rsync Looks like you were right with the 1st command if I'm reading that page correctly. Quote Link to comment
Endy Posted December 8, 2017 Author Share Posted December 8, 2017 1 hour ago, allanp81 said: https://linux.die.net/man/1/rsync Looks like you were right with the 1st command if I'm reading that page correctly. That's what I was thinking, but that didn't work. When searching I found an example that was not using the =. The more likely problem may be in the path where using a leading / is the problem because that makes it relative to the root directory of the source? (In my case /mnt?) I'm not really clear on that. Quote Link to comment
JorgeB Posted December 8, 2017 Share Posted December 8, 2017 You probably need to escape the space, try: --exclude 'mnt/user/Media/Movies/Blu-ray\ Movies' Quote Link to comment
Endy Posted December 8, 2017 Author Share Posted December 8, 2017 2 minutes ago, johnnie.black said: You probably need to escape the space, try: --exclude 'mnt/user/Media/Movies/Blu-ray\ Movies' Possible. I was thinking being in quotes it would ignore the space, but maybe not. I may make some dummy folders and files to test this out so I don't have a huge log to look through. Quote Link to comment
JorgeB Posted December 8, 2017 Share Posted December 8, 2017 14 minutes ago, Endy said: Possible. I was thinking being in quotes it would ignore the space, but maybe not. I may make some dummy folders and files to test this out so I don't have a huge log to look through. You're probably right, IIRC when I used exclude was without path, try: --exclude 'Blu-ray Movies' Quote Link to comment
kgregg Posted December 8, 2017 Share Posted December 8, 2017 i got this done a different way this week. I connected USB hard drive to small, low power Win10 PC running Resilio. Installed Resilio docker on unRaid box and got Resilio to backup my music, videos, and pictures shares from unRaid. It was slow but it worked. Many thanks to those behind the Resilio docker Quote Link to comment
Endy Posted December 8, 2017 Author Share Posted December 8, 2017 1 hour ago, johnnie.black said: You're probably right, IIRC when I used exclude was without path, try: --exclude 'Blu-ray Movies' Created some test directories and files and --exclude='Blu-ray Movies' should work. I believe that doing it that way will mean that any directory with that same name will be excluded. So if for some reason I had another folder somewhere else that was named Blu-ray Movies, it would be excluded as well. But since I don't and this appears to work, I'm good with it. Hopefully tonight I can run it and see. 1 hour ago, kgregg said: i got this done a different way this week. I connected USB hard drive to small, low power Win10 PC running Resilio. Installed Resilio docker on unRaid box and got Resilio to backup my music, videos, and pictures shares from unRaid. It was slow but it worked. Many thanks to those behind the Resilio docker I don't think I've heard of Resilio before, but it sounds like it might have been an option. Quote Link to comment
Endy Posted December 9, 2017 Author Share Posted December 9, 2017 I'm in the middle of running the script now and it's gotten through my Media share, which was the only one using excludes and it looks like it did exactly what I wanted it to do. One thing I am noticing it's that it seemed pretty speedy at first, but it seems like it's copying pretty slow now and even the UD section on the Main tab in the unraid gui is pretty slow to load. Maybe because it had to copy a ton of small files now compared to the larger media files it was doing before? The external drive is also running rather warm. Between about 45 and 48 degrees. I'm wondering if I shouldn't have done a preclear first just to test the drive. Seagate software comes preinstalled on the drive and I didn't want to just delete (even though I will probably never use it). I also kept it as ntfs so that I could read the contents from a Windows pc. I'm wondering if there are any problems with doing it this way especially when it comes to permissions if I ever have to load the data from the backup back onto the array. Maybe I should have switched it to xfs. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.