I am trying to use rsync to make a local backup of my data.  I would like to make incremental backups where the first folder is a full backup and then the folder will get moved as days go on keeping a hard link to the first folder.  I found an article on this here: http://www.admin-magazine.com/Articles/Using-rsync-for-Backups/(offset)/2   The problem i am having is that when I seem to do the mv command it copies all the files rather than making a hard link thus I lose all my hard disk space.  Any advice on what I am doing wrong.  

 

Here is my script:

#!/bin/bash 

Source4=/mnt/user/Recordings/Test

Destination1=/mnt/disks/BackupB1/Daily

#####################
### Destination 1 ###
#####################

### Source 1 ###
rm -rf $Destination1/TVShows/Daily5
mv $Destination1/TVShows/Daily4 $Destination1/TVShows/Daily5
mv $Destination1/TVShows/Daily3 $Destination1/TVShows/Daily4
mv $Destination1/TVShows/Daily2 $Destination1/TVShows/Daily3
mv $Destination1/TVShows/Daily1 $Destination1/TVShows/Daily2
cp -al $Destination1/TVShows/Daily0 $Destination1/TVShows/Daily1
rsync -a --delete $Source4/ $Destination1/TVShows/Daily0/ |& logger

I have also tried it with this as my last two lines but no luck:

 

mv $Destination1/TVShows/Daily0 $Destination1/TVShows/Daily1
rsync -avh --delete --link-dest=$Destination1/TVShows/Daily1/ $Source4/ $Destination1/TVShows/Daily0/ |& logger

 

Edited March 22, 20188 yr by mgranger

9 hours ago, mgranger said:

I seem to do the mv command it copies all the files rather than making a hard link

mv = "move".  It doesn't create any links.  Hard drive space will briefly go down due to the move, but after its finished it'll be back where it started.

 

You're looking to use either ln or symlink

10 hours ago, mgranger said:

cp -al $Destination1/TVShows/Daily0 $Destination1/TVShows/Daily1

 

I was thinking this line would move everything from Daily0 to Daily1 and make them links and then from then on everything would get passed with mv as a link but obviously I am wrong because it is not doing this.  

You're definitely getting there.

I had this working before and while I don't use rsync for backups now the most valuable article I read to get this working was http://www.mikerubel.org/computers/rsync_snapshots/

I'd strongly suggest looking through that article as it really explains the concepts and walks through how to make this work correctly.

Ok so I think i figured out getting rsync to work with a local source disk and a unassigned device for the destination.  I am now trying to rsync a local source disk to a unassigned smb share but I am really struggling.  It seems to be working sort of but i have some major concerns.  I basically use the same code as above except modified it a little to sync to the share folder.  So it looks like the following

 

#!/bin/bash 

Source2=/mnt/user/Media

Destination2=/mnt/disks/FinalizerMediaBackup/Media

#####################
### Destination 2 ###
#####################

### Source 2 ###
rm -rf "$Destination2"/Daily5
mv "$Destination2"/Daily4 "$Destination2"/Daily5
mv "$Destination2"/Daily3 "$Destination2"/Daily4
mv "$Destination2"/Daily2 "$Destination2"/Daily3
mv "$Destination2"/Daily1 "$Destination2"/Daily2
rsync -avh --delete --link-dest="$Destination2"/Daily1/ "$Source2"/ "$Destination2"/Daily0/ |& logger

When I run this the folders are populated on the share and it starts to run however a couple minutes in I get an error message that pops up and the process stops.

 

The message is

 

root: rsync error: error in file IO (code 11) at receiver.c(853) (receiver=3.1.3)

I have ran it a couple of times and it seems to stop on the same file however I looked at the permissions of the original file and it seems to be the same as the others.  If I try running the script again it acts like it is running but then weird things start happening and it will start removing some of the files and eventually alarm out on files that were originally there but has since removed.  This doesn't really make sense to me yet.

 

The weird thing is I have copied the original source to a local unassigned device without any errors but when I try to do it over the network it errors out.  My locations are correct because I see the files/folders start to populate.  Is there something I am missing.

 

The destination is a Windows PC which is sharing the drive and the drive is a NTFS drive which is mounted in unraid using the unassigned devices SMB shares.

Not sure if this helps but I try to follow along with rsync and watch as it adds files.  It seems that one file gets missed and then from then the rsync keeps working but never adds anything else and will eventually produce and error after some time saying that this file was an error.