Cache drive question

[mwoods98]

Hi all. I'm a bit new here and have a question. I have unraid setup v .6.6.6 and everything seems to be working just fine. I have some docker containers up and working and after playing around a bit I'm at the point where I want to start moving "real data" (ie: shows and movies) and start the retirement of an old machine. 

 

I began moving over (with a FileZilla docker) shows and just decided to move files that begin with A-G just to make sure things are working properly. After checking back in I noticed the cache disk giving a warning that it was 95% + full. So, I stopped the remaining transfer went to mover and ran it. It took a while but the cache disk got back down to under 10%. 

 

My question is: When transferring a large amount of files is this normal? Is there a setting that should be set for my cache drive? (I can post images or logs I just need to know what will be helpful)

 

Thanks for the help!

 

[dboonthego]

The size of your cache disk should be larger than the combined size of any files transferred in a 24hr period.  When a share is configured to use a cache disk, all files copied to the share will be stored to the cache disk until the mover runs.  This is when those files are moved off the cache disk and to the array. 

 

In your case, you could edit the share settings for the share you're copying data to and set the "Use cache disk" option to No.  Copy your files.  Then re-enable when done.  Or I think if "Use cache disk" is set to Yes and the disk fills up, subsequent new files are stored to the array instead of the cache disk.  Then when the mover is invoked, the cache files are moved to the array; completing the transfer.

Edited January 5, 2019 by dboonthego

[trurl]

49 minutes ago, dboonthego said:

I think if "Use cache disk" is set to Yes and the disk fills up, subsequent new files are stored to the array instead of the cache disk. 

Actually, you can't really allow cache to fill completely or you will get an error. For overflow to the array to work, you must have Minimum Free for Cache in Global Share Settings larger than the largest file you will write to any cached user share.

 

Unraid has no way of knowing how large a file will become when it begins to write the file. If a disk has less than Minimum Free remaining when Unraid chooses a disk to begin writing the file, it will not choose that disk.

 

Examples:

 

1. Cache currently has 6GB free, and Minimum Free setting is 4GB. You begin writing a 5GB file to a cached user share, Unraid will choose the cache disk because cache has more than minimum free. After the 5GB file is written cache will only have 1GB free. Any subsequent files will go to the array since cache no longer has more than minimum.
2. Cache currently has 6GB free, and Minimum Free setting is 4GB. You begin writing a 7GB file to a cached user share. Unraid will still choose cache for the file because cache has more than minimum free. But the file won't fit, and you will get an out-of-space error.

 

Minimum Free setting for each User Share works in a similar manner. If a disk has less than minimum free, Unraid will choose a different disk. If a disk has more than minimum free, Unraid can choose the disk even if the file won't fit.

[trurl]

But for the case of an initial data load, I recommend not caching anything. If your initial load is larger than cache it is just going to get in the way and slow things down.

 

Some people even do the initial load without a parity disk since writes without parity are faster. Then they build parity after the initial load is done.

 

There is also Turbo Write mode which can speed parity writes up some:

 

 

[mwoods98]

Thanks for the tips. I'm going to turn it off until I get the data loaded. Then come back and play around with the settings.

 

Thanks!!

[dboonthego]

16 minutes ago, trurl said:

Actually, you can't really allow cache to fill completely or you will get an error.

Thanks for the clarification.  I wasn't 100% sure about it.