Hi folks, hoping for your kind assistance with something as I try to set up some redundancy on my LAN.

 

I have a main server which is on nearly always. I then have a Microsever Gen 7 which I want to use as a local back up. Wakes up at midnight, copies accross key server shares, then turns off again. But I'm having issues getting the back-up to take place properly.

 

Ideally this is what should happen:

1. Main server sends a Magic Packet to wake up my back-up server at 23:50. (this works)

• Starts the array.
• Mounts the SMB shares that I want backing up.

2. At 00:00 (using Cron) my back-up server, which has had enough time to get up and running, runs a user script with Rsync. The running of the script happens.

3.  User Scripts executes the following:

1. #!/bin/bash
   
   #documents
   rsync -avhP "/mnt/remotes/MAINSERVER_documents" "/mnt/user/documents"
   
   #backups
   rsync -avhP "/mnt/remotes/MAINSERVER_Backups" "/mnt/user/backup"
   
   #photography
   rsync -avhP "/mnt/remotes/MAINSERVER_photography" "/mnt/user/photography"

    

4. The Dynamix S3 server pugin then puts my server to sleep after 15 min of inactivity. I have disabled this whilst the initital large transfers happen.

Issue:

• My local back up is endlessly copying files. The SMB share called 'backups' is ~350GB but has copied ~500GB into 'backup' on my backup server. In my head, it is using rsync (see commands above) so should only be checking for changed files - I cannot see why it is duplicating.  This did not happen when I ran a test case on my SMB 'documents' folder - which remains at 16GB for both.

 

Question:

• What have I set up wrong? Are my rsync flags wrong? Should I not be mounting the server SMB shares?
• If an rsync file transfer fails - should it not just pick back up where it left off the next night the script runs?
• How can I make my user script more verbose and tell me where it is in the transfer process to help me with debugging this?
• What am I not understanding?

Edited October 19, 20241 yr by dreadnought
to be polite to the helpful people

Flags seem fine, maybe you've moved some files from a folder to another?

Thanks JorgeB, a great point.  I did double check that - as it would trigger the same issue as I'm seeing here - but I have not moved files around inside the folder structure. These shares are static in that way.

Do a dry run and see how much would it copy.

2 hours ago, dreadnought said:

#documents
rsync -avhP "/mnt/remotes/MAINSERVER_documents" "/mnt/user/documents"

 

Read these articles about trailing slashes in the rsync  command:

 

https://news.ycombinator.com/item?id=25006665

 

       and

 

https://unix.stackexchange.com/questions/402555/why-add-a-trailing-slash-after-an-rsync-destination

2 minutes ago, Frank1940 said:

Read these articles about trailing slashes in the rsync  command:

Yep, I missed that, recommend always having the end slash in both paths.

Trailing slashes. I have done it differently between doing a dry run the terminal, and then typing it in a user script for sheduling. As a result it has dupicated the share in the different file paths on the server - hence x2 of the share.

 

Many thanks for your help every one.