App/Plugin for backing up/Sync external hard drive


luowilliam

Recommended Posts

I have searched around google as well as on the forum for other people's solution on this but couldn't find anything good.

Basically what i'm trying to do is that I have a few USB external hard drives I take with me for work. I would like them to be backed up or synced to be more precise (the external drive being the master copy). Right now I'm doing this by every once in a while plug them into my home computer and Acronis it and store it on the unRaid NAS. There are 2 flaws in this system:

1. I have to manually use another computer to Acronis it, taking up resource on that computer.

2. The backup is in the *tib file format, in order to view the content, I need to have a windows computer with Acronis installed to view.

I'm hoping to find a solution that would be natively installed on unraid server, where I can just plug in my USB drive to the unraid server, it would copy all the files that are different then what's in the backup already in plain files (not *tib). On the mac, there's this app called carbon copy cloner, but I couldn't find anything similar for unraid.

Any help is appreciated. 

Thanks!

William

Link to comment

I have a similar use case but I use BackBlaze and have it sync a USB drive when it is connected.

 

But your case, I believe, is doable:

 

In Unassigned Devices, you should see the Script icon next to each device (rather all the way to the right). From the help text:
 

UD includes scripts that are used for mounting and unmounting devices and SMB/NFS mounts. These scripts can be used from the command line or in scripts as necessary.

Unassigned devices script:
'/usr/local/sbin/rc.unassigned mount auto' - all devices and SMB/NFS mounts set to auto mount will be mounted.
'/usr/local/sbin/rc.unassigned umount auto' - all devices and SMB/NFS mounts set to auto mount will be unmounted.
'/usr/local/sbin/rc.unassigned umount all' - all devices and SMB/NFS mounts are unmounted in preparation for shutting down the array.
'/usr/local/sbin/rc.unassigned mount /dev/sdX' - mount device sdX where 'X' is the device designator.
'/usr/local/sbin/rc.unassigned umount /dev/sdX' - unmount device sdX where 'X' is the device designator. You can use this command in a UD script to unmount the device when the script has completed.

I haven't tested this too much, so this is more a outline, but the default script (left most button when editing script) outputs a simple script template which I have done some quick additions to:

#!/bin/bash
PATH=/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin
## Available variables: 
# AVAIL      : available space
# USED       : used space
# SIZE       : partition size
# SERIAL     : disk serial number
# ACTION     : if mounting, ADD; if unmounting, REMOVE
# MOUNTPOINT : where the partition is mounted
# FSTYPE     : partition filesystem
# LABEL      : partition label
# DEVICE     : partition device, e.g /dev/sda1
# OWNER      : "udev" if executed by UDEV, otherwise "user"
# PROG_NAME  : program name of this script
# LOGFILE    : log file for this script
## vars
mountPath=$MOUNTPOINT
destPath=/mnt/user/meta
destFolder=Toshiba.backup

## function to email
  
case $ACTION in
  'ADD' )
    # touch /mnt/user/meta/touched # for testing
    if rsync -Carv $mountPath/ $destPath/$destFolder # remove v for quietness
    then
      echo "Copied! Ready to remove."
      #/usr/local/sbin/rc.unassigned umount $mountPath # doesn't seem to work
      #umount $mountPath # ready to remove
      # call function to email
    else
      echo "Error! I need help!"
      # call function to email
    fi
  ;;

  'REMOVE' )
    # do your stuff here
    echo "Removed"
  ;;
esac

The only thing left to add would be an email notifing you that it is complete and ready to remove.

 

Thats roughly what I would do. I'd build and test it a bit more but I'm at work can't test this too much. (Under the disk partition you'll see a lighning bolt when you have a script there for running from the GUI. That was a nice find, I was expecting to have to manually remove and add it.)

 

Hope this helps you a bit.

Link to comment

Wow that was really detailed response. I’m new to linux command line and scripting. It looks like rsync is the executable used to do the copying? It has ability to recognize new and changed files and only copy those?

thanks again I will definitely have to try this out.

William

Link to comment
  • 2 weeks later...

Years ago (decades actually!) I stopped using any Windows backup solution that involved proprietary compression formats. I use Robocopy .cmd files to do all my backup work between desktops, external drives and unRAID disks. It uses very little resources and has many switches for very granular control of parameters. It was introduced as a free tool from MS in the Windows Resource Kit for NT and has been a "hidden" tool in every version of Windows since Vista. Granted, it doesn't run on the unRAID server, but it is highly reliable. Google it or just enter robocopy /? in a Windows command prompt window.

Link to comment
On ‎7‎/‎27‎/‎2018 at 12:14 PM, shaunsund said:

I kept making changes, here's the script: GitLab Snippet

 

UPDATE: Just learned that the /tmp/$SERIAL.log and the $LOGFILE aren't the same. /tmp/$SERIAL.log is the log the Log Icon looks at. (via UD maintatiner)

 

So if I'm following this thread and script correctly, the end result would be:

 

-Plug in external drive

-UD mounts the device and executes the script

-Script runs rsync to sync contents of the source share on the array to the destination share on the external

-Script generates e-mail alert is generated when rsync is done saying complete or failed

-UD unmounts the external drive (unless unmounts is disabled for testing?)

-The external drive is ready to be disconnected

 

And this would all be automated and logged? I'd really like this kind of functionality if what I'm thinking it does is correct.

 

**EDIT: Actually, this looks like it's running rsync to copy from an external drive to somewhere else. So to have this do what I want, could I change the $destPath and $destFolder variables to $sourcePath and $sourceFolder and change lines 89 and 90 to say:

echo "rsync -Carv $sourcePath/sourceFolder/ $mountPath/" >> $LOGFILE
if rsync -Carv "$sourcePath/sourceFolder/" "$mountPath/"

 

Edited by baconborn
  • Upvote 1
Link to comment

I've tested it plenty of times and that is how it works.

 

Please note, the $LOGFILE is a location for your custom messages/output while the /tmp/$SERIAL.log seems to be the raw output for the script. I think that Log icon on the UD page should read /tmp/$SERIAL.log, but doesn't seem to.

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.