Compare two shares and list differences to create offload for backup


Recommended Posts

Hi Everyone,

 

I've searched and searched but can't find a how to for my precise issue.

 

I have 2 unraid servers for television shows but one of them is offsite with very very slow internet.  When I copy to one I also copy off to an external hd which gets plugged into the remote machine to update it.  I do this every few months as its a 4 hour drive away.  I got tired of having to duplicate everything on the external and had the idea of comparing the two shares and then only offloading the missing files to the external as a one shot deal rather than a little every day.  I need the results in some form that I can use excel to build a script to offload the missing files to the external.  what ideas can you guys think of?  I can load and run anything on both machines but I can't sync over the internet due to speed and bandwidth caps.  Any help will be most appreciated.

 

- Jonny

Link to comment

I’m away from my unRaid system until tomorrow evening, so this is just a suggestion until I can double check it ...

 

log into unraid2 system.

change directory to your TV share, eg

  cd /mnt/user/TV

run the following command to produce a list of files into the file list2

  find . -type f -print >list2

 

log into unraid1 system and do the same commands but put the output into the file list1

  cd /mnt/user/TV

  find . -type f -print > list1

 

you will need to somehow copy the file list2 from the unraid2 system onto unraid1 and put it in the same directory as the list1 file

 

then run the command

  comm -2 -3 list1 list2 >list

 

the file list should now contain those files that are on your unraid1 system but not on your unraid2 system.   Hopefully you can do something useful with this list.

Link to comment

Ah.   The dangers of scripting when you are not at a machine where you can check it out.

 

Add the sort command to the command lines doing the find, eg

 

    find . -type f -print | sort >list1

 

This will ensure the contents of the files being compared by the comm command are sorted.

 

Link to comment
  • 1 year later...

You can also use the rsync command to only copy the missing files.

rsync -avzh --progress /mnt/user/ShareSource /mnt/disks/ShareDestination

 

You can also add the --delete-before tag at the end if you want to remove old files from the destination share. Careful with this those. If you set up automatic sync, it can bite you by deleting data that may have been lost or corrupted.

 

(Came here searching for something and thought my addition might help)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.