Simple RSYNC Between two unraid towers

[petebnas]

I've been running unraid on two towers for a while...basically tower #2 just backs up tower #1 every few months.

 

These are completely mirrored hardware boxes...same number/type of disks/drives/shares/etc.  What i'm having an issue with is getting rsync to complete, and not complain about not having enough disk space to send the incremental updates over to the backup box...

 

Tower #2 has a rsyncd.conf file of:

uid=root

gid=root

use chroot=no

max connectinos=4

pid file=/var/run/rsyncd.pid

timeout=600

log file=/vor/log/rsyncd.log

 

[mnt]

  path=/mnt

  comment=/mnt files

  read only=FALSE

 

On tower 2, I manually enter

rsyncd --daemon --config=/boot/config.rsyncd.conf

 

On tower #1, I manually enter

cd /mnt

rsync -avrtH user/ hal::mnt/user

 

It goes into the backup cycle, then errors out, complaining that it's out of disk space.  Meanwhile On tower #2 (the destination), I have all of the shares set to split level 1, and each share is showing 1.7tb of free space... The individual disks are pretty full though...but I'm not sure why it's not flowing over to the other drives, since i have each share set to a 'most free' config as well..

 

Any tips are appreciated..

Pete

[BetaQuasi]

Are these 'new' folders?  Unfortunately rsync writes the folders first, then the content.. so if your destination is a user share, you can end up with a whole stack of folders (which are considered small in size) on a drive you don't want them on.  rsync then writes the content and you run into the issue you describe.

 

I hit the same thing when shifting everything from my ReadyNAS NV+ x2 to my new unRAID box.  In my case, they were one off transfers, so I'd just move a stack of folders (full and empty) to a different physical disk with plenty of free space and restart the rsync.

 

I'd be interested in a permanent solution though - I'm no rsync guru, and have done a fair bit of googling, but wasn't able to work it out.  While I don't need this functionality right now, I'll probably be putting a backup box online in the near future and will be looking to do the same thing you are.

[S80_UK]

Are these 'new' folders?  Unfortunately rsync writes the folders first, then the content.. so if your destination is a user share, you can end up with a whole stack of folders (which are considered small in size) on a drive you don't want them on.  rsync then writes the content and you run into the issue you describe.

 

I also use rsync between my two servers.  The backup is switched off most of the time and I run the backup as a manually invoked process using a very crude script.  I don't mirror the shares, however.  Instead I have created shares on the backup server called disk1_backup, disk2_backup, etc.  These can be mapped to a drive or drives on the backup server as needed.  The backup from the main unRAID server is then run on a disk-by-disk basis.  So I back up /mnt/disk1 on the main server to /mnt/user/disk1_backup on the backup server.  I did it this way to give me some flexibility with the disk drive population in the backup server, but I think this also prevents the problem described above.

 

 

[petebnas]

These are all existing shares...the same shares were setup on both systems originally, and then I ran that rsync command it just up the destination system nicely, and that was it.  I'd run some subsequent rsync's to get changed data, and that would go through fine... 

 

Somewhere down the line, and I think it's when the originating system got additional drives added to it, I started to run into issues.  The originating system had an extra drive that the dest did not, but it wasn't 100% full, so things went ok, but when the originating started maxing out, to the point of where I needed to expand the dest. system with more drives, that's when things went off.  All my disk models are the same, same size, type, number, etc.  Same controller, share levels, motherboard, unraid versions, even the flash drives are duplicates.

 

I guess I'm not quite understanding something about unraid...or catching why this wouldn't work... or why I'd have to do separate rsyncs between drive letters.  Isn't disk1, disk2, disk3, etc..these are just shares that unraid creates, right?  Why would that rsync behave differently than my different user shares (as along as I didn't do something funny with split levels...

 

Pete