Is it possible to get an rsync plugin or docker that makes it simple for us newbs so its source, destination lines and which copy options you want with a check box or a yes or no field. and an option for it to loop after its done?

 

 

 

Let me flip the question back: why do you need rsync?

There are simpler apps out there to keep 2 folders in sync without needing endless loops (i.e. only run upon changes) e.g. Syncthing, Resilio Sync, etc.

 

Sounds to me like you picked a tool beyond your skill level to solve a problem it may not be the the most suitable tool for.

2 hours ago, nicksphone said:

Is it possible to get an rsync plugin or docker that makes it simple for us newbs so its source, destination lines and which copy options you want with a check box or a yes or no field. and an option for it to loop after its done?

 

 

If you want/need to use rsync and you need it to run "continually" you can use the User Scripts plugin to schedule your rsync task every minute with a * * * * * custom cron setting.

 

I back up all my unRAID shares from one server to another unattended via an rsync script.   I only do it once a week, but you can set the frequency to anything cron allows and it can be as granular as every minute.

18 hours ago, testdasi said:

 

Let me flip the question back: why do you need rsync?

There are simpler apps out there to keep 2 folders in sync without needing endless loops (i.e. only run upon changes) e.g. Syncthing, Resilio Sync, etc.

 

Sounds to me like you picked a tool beyond your skill level to solve a problem it may not be the the most suitable tool for.

syncthing and resilo sync i end up with duplicated files like File1 and file1 is there a setting im missing?

That's not duplicated file. Linux is a case-sensitive system, unlike Windows. So File1 and file1 are 2 different files.

1 hour ago, testdasi said:

That's not duplicated file. Linux is a case-sensitive system, unlike Windows. So File1 and file1 are 2 different files.

which is fine in a perfect world but when a windows user changes the capital letters in a file name linux sees it as a diffrent file on sync and windows has a hissy fit when there are 2 files named the same as it sees 

5 hours ago, nicksphone said:

which is fine in a perfect world but when a windows user changes the capital letters in a file name linux sees it as a diffrent file on sync and windows has a hissy fit when there are 2 files named the same as it sees 

But rsync on Unraid, which is Linux based, will do the same.

23 hours ago, nicksphone said:

which is fine in a perfect world but when a windows user changes the capital letters in a file name linux sees it as a diffrent file on sync and windows has a hissy fit when there are 2 files named the same as it sees 

But the file on unRaid will be changed from say "test" to be "TEST".  There will not be 2 files on unRaid.  Your backup solutions will see that there is only a single file, but the history / old versions of the files will still exist.  

I use a switch in my rsync backup script called "--delete-before" and it will drop/delete any file or folder in the target location that was either changed or doesn't exist anymore prior to rsync making a new backup from source. So if a file was renamed file1.txt to File1.txt since the last backup, file1.txt on my backup target will be deleted and a new copy of File1.txt will take place.

Edited July 20, 20196 yr by cybrnook

5 hours ago, cybrnook said:

it will drop/delete any file or folder in the target location that was either changed or doesn't exist anymore prior to rsync making a new backup from source.

So how do you recover from accidental deletions or corruption?

Well I use the wonderful recycle bin plug-in for accidental deletes, and to be honest I have been using unraid for around 10 years and have never once accidently deleted anything.

 

And for corruption I have redundant backup servers that take backups at different times. Again, have yet to run into corruption due to anything like bit rot etc.

 

Key is I write my own scripts, so I understand what they are doing and create my own assumed and acceptable risk factors trading things like versioned backups for smaller backup footprints.

Edited July 21, 20196 yr by cybrnook

9 hours ago, cybrnook said:

Key is I write my own scripts, so I understand what they are doing and create my own assumed and acceptable risk factors trading things like versioned backups for smaller backup footprints.

That's great for you, not so good for the OP in this thread.

Question was asked if rsync could handle changes in file names instead of duplicating, and I answered. Yes it can. Seems relevant to me.

Edited July 21, 20196 yr by cybrnook

On 7/20/2019 at 9:28 PM, jonathanm said:

So how do you recover from accidental deletions or corruption?

One could argue that syncing is not backup and not meant to protect against deletions or corruption.

@cybrnook

I know this thread is way old, Pre-Pandemic one might say, but I'm now dealing with something similar and your approach seems like something that would be beneficial to me. Can you give me an idea of a script you would use for my situation?

Basically, I'm a photographer; I have three 2TB external drives where I store all of the photos from each photoshoot and event that I photograph. The way I have organized their directory structure is by Year, then by project, then raw files along with folder(s) broken out by the concepts of edited, Full-Res, Low-Res, Watermarked Low-Res, etc. 
On each drive I have a 2022 folder, then inside that folder on one drive might be folders 2022-03-19 and 2022-07-21, then the other drive might have 2022-03-21 and 2022-05-11, and the third drive has several other 2022 project folders.

My goal is for my destination folder on the NAS to have all 2022 folders from each drive copied into a single 2022 destination folder. I have sporadically been moving one to two folders at a time from each drive into the single 2022 destination folder. What I feel is my problem now is, if I use RSYNC to move all sub folders under each external drive's 2022 folder but some of those folders are already on the NAS, how do I do it so it moves only new files and folders into the combined folder and skips over any files that are already on the nas and the latest version?

I hope this makes sense. Thanks for your willingness to read through this.
 

1 hour ago, DSPhoto said:

@cybrnook

I know this thread is way old, Pre-Pandemic one might say, but I'm now dealing with something similar and your approach seems like something that would be beneficial to me. Can you give me an idea of a script you would use for my situation?

Basically, I'm a photographer; I have three 2TB external drives where I store all of the photos from each photoshoot and event that I photograph. The way I have organized their directory structure is by Year, then by project, then raw files along with folder(s) broken out by the concepts of edited, Full-Res, Low-Res, Watermarked Low-Res, etc. 
On each drive I have a 2022 folder, then inside that folder on one drive might be folders 2022-03-19 and 2022-07-21, then the other drive might have 2022-03-21 and 2022-05-11, and the third drive has several other 2022 project folders.

My goal is for my destination folder on the NAS to have all 2022 folders from each drive copied into a single 2022 destination folder. I have sporadically been moving one to two folders at a time from each drive into the single 2022 destination folder. What I feel is my problem now is, if I use RSYNC to move all sub folders under each external drive's 2022 folder but some of those folders are already on the NAS, how do I do it so it moves only new files and folders into the combined folder and skips over any files that are already on the nas and the latest version?

I hope this makes sense. Thanks for your willingness to read through this.
 

I think "--ignore-existing" would be the switch you want in your rssync command.

 

https://linux.die.net/man/1/rsync

 

"--ignore-existing skip updating files that exist on receiver"

On 7/18/2019 at 1:02 PM, Hoopster said:

If you want/need to use rsync and you need it to run "continually" you can use the User Scripts plugin to schedule your rsync task every minute with a * * * * * custom cron setting.

 

I back up all my unRAID shares from one server to another unattended via an rsync script.   I only do it once a week, but you can set the frequency to anything cron allows and it can be as granular as every minute.

Came across this thread and this is eventually exactly what I would want to implement. Going to continue research to understand how to deploy this scheduled task.

 

Thank you.