MOVER is full and won't move the files?


Recommended Posts

Hello,

 

I bought a 2 pack of the max edition unRAID back in the 5.x era, and I have been upgrading it ever since.  I had the 6 BETA for the longest time and finally upgraded to the full 6.

 

I have had the same system for years now, but upgraded the hardware about ever 4th year. My last upgrade was to an ASRock Z87 extreme11 a/c board and i5 CPU, which has been great (the board comes with 22 SATA ports and an internal USB port, so it looks nice.)

 

I have 15 devices and 32TB, running dual parity.  Over all the years and upgrades, I have never lost a file!

 

Now, my problem is the mover seems to be stuck.  I have included my log files, I have a 15k 73GB SAS drives, everything else is SATA.  Everything is green, and from what I can tell rsync runs but never moves the files off of the cache drive?  I SSH'd into the box and ran top, I see rync running.  Of course I have rebooted the box several times.

 

When I ran ps-ef grep rsync, it looks like it runs these commands:

 

root     10420 10301 23 14:42 ?        00:01:07 /usr/bin/rsync -i -dlDIWR --inplace -pAXogt --numeric-ids ./share1/AcronisImages/1366BLK/My_partitions_full_b1_s1_v1.tib /mnt/cache
root     10421 10420  0 14:42 ?        00:00:00 /usr/bin/rsync -i -dlDIWR --inplace -pAXogt --numeric-ids ./share1/AcronisImages/1366BLK/My_partitions_full_b1_s1_v1.tib /mnt/cache
root     10422 10421  4 14:42 ?        00:00:12 /usr/bin/rsync -i -dlDIWR --inplace -pAXogt --numeric-ids ./share1/AcronisImages/1366BLK/My_partitions_full_b1_s1_v1.tib /mnt/cache

 

IS there a way I can look at what rsync is choking on?

 

The only other problem I have noticed is when I tried to copy a 2.5TB file to my array, I would get the disk full error.  I tried changing the share settings, that did not work.  Not sure if that is related?  The only other thing I can think of is I recently added a Mellanox ConnectX-2 10GBe card, but I disabled that during my troubleshooting.  I was thinking of powering down and pulling the card?  Could a NIC really cause this issue?

 

Thanks for the help, the community looks better than ever!  I have several people at my working looking to switch toi unRAID from the Commercial NAS'es.

 

David

 

 

 

unraid-diagnostics-20170415-1429.zip

Edited by [email protected]
Link to comment

No, not caused by a NIC. Check your share settings.

 

You have a bunch of shares with "use cache" set to "prefer", which means that the files should be moved FROM the array disks TO the cache disk when mover runs (if there is a cache disk). For most shares you probably want "yes" instead. Enable help on the page to read more about the options.

Link to comment
20 hours ago, jonathanm said:

Take a look at the help item for minimum free space in the share properties, specifically on the share you are trying to copy to.

 

I read this and set it to 2.5TB, and I have about 3 drives that could handle a file of that size.  Still did not work.

 

So, I am using WinSCP to copy the file to a big enough drive, and it is taking its time, but appears to be working.

 

I don't have many files over 2TB, but I do have a few.  This has stirred a question about unRAID in general.  Let's say you do have large files, everything over 1TB, with some files in the 3TB range (think video editor).  I guess the way unRAID stores files, you would need to buy drives larger than the biggest file you hope to create?  While I might have 25TB free, I cannot currently write a 10TB file to any of my shares b/c none of my drives are over 4TB?

 

In 99% of the use cases, this would not be a problem very often (except big backup files).  However, if this is a limitation, it would be nice to know.  If I use a backup program to stream backups to the array, I wonder what happens when Acronis starts to write a file, to 1TB drive, and then the backup goes to 2TB?  I could test this, but I wonder if others have had this issue?  I guess the backup would fail?  A workaround would be to use Acronis Backup Splitting, where it makes multiple 4GB files for large backup jobs, so if a drive fills up, it can spill over to the next drive?

 

I might need to start another thread.  I love unRAID, so I am not changing.  My server is about to have its 10th birthday.

 

David

 

 

Edited by [email protected]
Link to comment

Yeah, each file needs to fit on one disk, and unRAID won't move files around for you to make a new one fit.

 

Regarding your failed copies, you need to be a bit more specific. Does the copying fail immediately or after some time? How do you transfer the file(s)? Do you get an error message? Where? What does it say? New diagnostics after your set minimum free space and failed transfer?

Link to comment

I would have thought that files in the TB range are far outside the typical usage of unRAID.   Even an uncompressed BluRay is normally only about 40GB.

 

if you are wanting to manipulate files in this size range then you definitely need to be aware of the fact that unRAID has to be able to store the whole file on a single drive (I.e. It will not be split across drives).   At least with 10TB drives becoming mainstream this can actually be managed.   However the write performance limitations on unRAID definitely makes files of this size unwieldy.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.