Sync directories between two UnRaid servers

[deionmann]

So how would you append it to the file listed above -

 

sync -a --progress --log-file=/mnt/cache/apps/RSYNCLOG.log /mnt/disks/godzilla_pictures/ /mnt/user/godzilla/pictures

 

Here's how I've appended it

 

rsync -avh --log-file=/mnt/user/godzilla/pictures/pictures-backup.log --stats /mnt/disks/godzilla_pictures/ /mnt/user/godzilla/pictures

 

then my notifications call looks like this

/usr/local/emhttp/webGui/scripts/notify -s "`hostname` Rsync Backup complete" -i "normal" -d "Backup of Pictures share complete `date` `tail /mnt/user/godzilla/pictures/pictures-backup.log`"

[gundamguy]

I want to say great job on this.

 

I know I jumped in to help you get your rsync call set up right, but you've done a great job documenting how you put all the pieces together, and helped me get my backup to have notifications.

[deionmann]

 

Sent from my SM-N920V using Tapatalk

 

 

[chip]

deionmann

 

thanks for all the input.

 

Hopefully a simple question but what is the last part and how does it work?

 

/usr/local/emhttp/webGui/scripts/notify -s "`hostname` Rsync Backup complete" -i "normal" -d "Backup of Pictures share complete `date` `tail /mnt/user/godzilla/pictures/pictures-backup.log`"

 

I got your first command to work on a test yesterday....pointing to my needs and it created the log file...again thanks!

sync -a --progress --log-file=/mnt/cache/apps/RSYNCLOG.log /mnt/disks/godzilla_pictures/ /mnt/user/godzilla/pictures

[bonienl]

deionmann

 

thanks for all the input.

 

Hopefully a simple question but what is the last part and how does it work?

 

/usr/local/emhttp/webGui/scripts/notify -s "`hostname` Rsync Backup complete" -i "normal" -d "Backup of Pictures share complete `date` `tail /mnt/user/godzilla/pictures/pictures-backup.log`"

 

I got your first command to work on a test yesterday....pointing to my needs and it created the log file...again thanks!

sync -a --progress --log-file=/mnt/cache/apps/RSYNCLOG.log /mnt/disks/godzilla_pictures/ /mnt/user/godzilla/pictures

 

-d == description part of the notification. I recommend that any file content attached to the notification is put in the message part (-m). So the example above becomes:

 

/usr/local/emhttp/webGui/scripts/notify -s "`hostname` Rsync Backup complete" -i "normal" -d "Backup of Pictures share complete `date`" -m "`tail /mnt/user/godzilla/pictures/pictures-backup.log`"

[chip]

So that all that info gets written to /mnt/user/godzilla/pictures/pictures-backup.log?

 

or am I missing something in regards to /usr/local/emhttp/webGui/scripts/notify ?

[bonienl]

So that all that info gets written to /mnt/user/godzilla/pictures/pictures-backup.log?

 

or am I missing something in regards to /usr/local/emhttp/webGui/scripts/notify ?

 

No, it is the other way around. The content of the given .log file is attached to the notification.

 

Ps. You can make a reference to any file / content as is needed.

 

[chip]

Is there a way to send an email after the sync is complete?

 

Than I know it is done and can look at the log file later.

 

I saw something in regards to ...

 

mail -s "Rsync completed " [email protected]

[deionmann]

I'm pretty sure all you have to do is setup the notifications settings in the gui to send emails and you'll get it via email from that notification script

 

Sent from my SM-N920V using Tapatalk

 

[chip]

IS the following possible.

 

Setup an rsync job to do this.

 

Share is read only

Turn share to read/write

Move files

After files are moved turn back to read only\

 

 

[gundamguy]

IS the following possible.

 

Setup an rsync job to do this.

 

Share is read only

Turn share to read/write

Move files

After files are moved turn back to read only\

 

I'm sure that is possible. I'm not sure what you tell SAMBA to do that, but I'm sure it's a thing you could do.

 

That said, if you use the ssh feature of Rsync instead of mounting the SMB share with unassigned devices (treating it like a local write), you wouldn't have to jump though those hoops.

 

The catch is that you've got to set up ssh keys instead of ssh passwords or else your script will prompt you to input a password when it goes to connect via ssh.

[Russ Uno]

<CTRL> + <X> to exit and Save output. The cron Job is now saved. I have noticed that the dynamix cron files are not loaded just because you create a file BUT you can force this. Just go into the Settings>Scheduler AND make a change (so the Apply Button gets un-greyed out) and change it back (Apply Button remains un-greyed out) and Hit Apply (essentially not changing ANYTHING) then the cron file gets loaded.

 

Can't you also use?

update_cron

I know I've used it when changing a cron file.

 

[chip]

IS the following possible.

 

Setup an rsync job to do this.

 

Share is read only

Turn share to read/write

Move files

After files are moved turn back to read only\

 

Guess I will have to do some digging on this.

 

I'm sure that is possible. I'm not sure what you tell SAMBA to do that, but I'm sure it's a thing you could do.

 

That said, if you use the ssh feature of Rsync instead of mounting the SMB share with unassigned devices (treating it like a local write), you wouldn't have to jump though those hoops.

 

The catch is that you've got to set up ssh keys instead of ssh passwords or else your script will prompt you to input a password when it goes to connect via ssh.

[gundamguy]

It's pretty simple again the problem is you have to setup SSH Key Login first.

 

But then your RSYNC call follows this format.

 

Access via remote shell:
  Pull: rsync [OPTIONS...] USER@HOST:/Source /Destination
  Push: rsync [OPTIONS...] /Source USER@HOST:/Destination

 

If you don't have SSH Keys set up, this will prompt you to log into the machine like normal, this of course is an issue if you are planning to set up a script executed by CRON, since that script requires manual input. If you have SSH Key Login set up with pre-shared keys it won't prompt you since you have the key.

 

[chip]

OK so to build off of this say moving data from one disk to another on the same server....

 

Right now all my disk shares/User shares are set to secure which means they are read only correct?

 

Yet my cron job below can write to the disk?  why?

 

# Monthly test backup

38 17 * * * rsync -avh --progress --log-file=/mnt/cache/RSYNCLOG.log --stats /mnt/user/files/NetflixMCE_BETA /mnt/disk4/RAID_Test |& logger

 

[gundamguy]

OK so to build off of this say moving data from one disk to another on the same server....

 

Right now all my disk shares/User shares are set to secure which means they are read only correct?

 

Yet my cron job below can write to the disk?  why?

 

# Monthly test backup

38 17 * * * rsync -avh --progress --log-file=/mnt/cache/RSYNCLOG.log --stats /mnt/user/files/NetflixMCE_BETA /mnt/disk4/RAID_Test |& logger

 

It's not going though SMB at all, so SAMBA's (SMB server) permission schemes aren't being applied.

 

There is setting a file to be read only in Linux using chmod, and then there is setting SMB to deny write permission on a share... You are doing the latter when you change SMB settings. It's not actually changing the permissions on every file in that share.

 

Also be very careful when mixing /mnt/user/xxxx with /mnt/diskx/xxxx if there is a collision (IE attempt to write to the same place) it'll cause data loss.

[chip]

any good suggestions than for doing that?

[trurl]

any good suggestions than for doing that?

What exactly do you want to do?

[IamSpartacus]

Lots of great info in this thread.

 

One question, how would one schedule a cron job to run perpetually?  This it to keep main and backup server in sync in as close to real time as possible.

[damelon]

Yes, I'm ressurecting an old thread...

 

So I am doing this to backup my unraid to a second NAS.      But after about 12 hours or so rsync terminates and my /mnt/user directory becomes inaccessable.   Even trying to go to it results in a "directory user transport endpoint is not connected" error.    Rebooting unraid fixes this issue, so I'm assuming it is an out of memory or swap file error or something like that.  I've seen a few people post about this issue in here but haven't found a definitive solution.   Anyone run into this? How did you solve it?

[trurl]

8 minutes ago, damelon said:

Yes, I'm ressurecting an old thread...

 

So I am doing this to backup my unraid to a second NAS.      But after about 12 hours or so rsync terminates and my /mnt/user directory becomes inaccessable.   Even trying to go to it results in a "directory user transport endpoint is not connected" error.    Rebooting unraid fixes this issue, so I'm assuming it is an out of memory or swap file error or something like that.  I've seen a few people post about this issue in here but haven't found a definitive solution.   Anyone run into this? How did you solve it?

Tools - Diagnostics, post complete zip, preferably after issue but before reboot.

[damelon]

Will do the next time it happens.