Linux, Windows following different rules when copying


Recommended Posts

I've created a backup UnRAID server.  I has two data disks (parity will come later, after copy done).

 

I've noticed a difference in behaviour about where files go depending if I'm copying the files from Windows or Linux (specifically, another UnRAID server).  All computers are on the local network.  The destination server has a Media share set to Split first two levels (within the share I have Movies and TV Shows directories, then under that a folder holding each movie or TV series).  Method is Fill Up, 10G minimum space, but the same behaviour is seen regardless if this is High Water.  Included disks All, Excluded None, use cache disk No, Security Public, Enhanced OS X No.

 

What I'm seeing is that when copying via rsync from the other UnRAID machine, only disk 1 is used, and the copy continues until there is no space left, ignoring both the allocation method and minimum free space settings.  When copying the same files from Windows, the copy spills over to use disk 2 when the minimum space is hit and the copy continues as expected.  Even if allocation method is high-water, rsync will totally fill disk 1 and error out about no space left, when disk 2 is sitting there unused.  I copy some files from Windows to the same share and now I see files going to disk 2.

 

rsync is writing to /mnt/user/Media, the same share I'm writing to when I using WIndows (via SMB to Media share).  I notice there is also a /mnt/user0/Media - maybe I should be using that directory instead of /mnt/user/Media?

 

Anyone else notice this behaviour?

Edited by jjjman321
remove os x references for clarity
Link to comment

I think the issue is that by default rsync creates all folders at the start before copying the files (not sure if there is a parameter option to change this behaviour).  The Split Level setting is then forcing files to the first disk.   When copying from Windows the directories are created more dynamically as the copy proceeds thus allowing Split Level to function as expected.

Link to comment

Yes, I believe you are right. There was a directory for every source media file created on disk 1.  When the command errored out, I deleted all the empty directories.  Since my UnRAID minimum space was already triggered, it then moved on to disk 2 - but rsync then created a directory for EVERY media file on disk 2 - even movies that were already on disk 1!

 

I'll need to see if I can turn this behaviour off with a switch.  I head several people here use rsync to copy between servers.  I'd love to hear how others have gotten around this issue.

 

Thanks

Link to comment

I use rsync to backup one unRAID server to another. They both have a cache pool and the user shares are set Use Cache=Yes. When rsync creates all those empty folders it does so on the backup server's cache and the mover subsequently moves them to the expected place on the array (in accordance with the rules). My backups are generally small so they usually don't overflow the cache. If overflow does occur the folders are created on the array as required.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.