[help]Auto disable write to cache on share when it's full

[KombatJam]

I hope this is the correct place to put this..

 

So I am looking for a way / script that can automatically disable a share's ability to use the cache during a write operation. Basically, I am migrating from a 5 disk array to Unraid, but when I put the cache disk and it fills up (250GB ssd vs 9TB of data), it will fill the cache drive, then throw errors during write, either not enough space.

 

I was wondering if there is a script that I can run that would disable that share's access to the cache when it hits, lets say 85%, so it would write to disk normally again, start the mover process, and once it's at ~10%, enable the cache again. I tend to do this manually when I can, but I still have 3-4 more days of transferring, so if I can use a script for this, it would be a big time saver.

 

Cheers

[trurl]

This is not the way to solve this problem. A user share set to cache-yes will automatically overflow to the array if you have Minimum Free for cache set correctly in Global Share Settings. You should set Minimum Free for cache there, and Minimum Free in each User Share's settings, to be larger than the largest single file you expect to write.

 

If unRAID sees that a disk has less than minimum free space, it will choose another disk when it begins to write a file. If it sees that a disk has more than minimum free space, then it can choose the disk to begin to write a file, and if it runs out of space when writing the file then you get errors. It can't know how large a file will become when it decides which disk to write it to. So you must make it choose another disk by setting Minimum Free larger than the largest file you will write.

[trurl]

20 minutes ago, KombatJam said:

I hope this is the correct place to put this..

 

No but I moved it.

[KombatJam]

15 minutes ago, trurl said:

This is not the way to solve this problem. A user share set to cache-yes will automatically overflow to the array if you have Minimum Free for cache set correctly in Global Share Settings. You should set Minimum Free for cache there, and Minimum Free in each User Share's settings, to be larger than the largest single file you expect to write.

 

If unRAID sees that a disk has less than minimum free space, it will choose another disk when it begins to write a file. If it sees that a disk has more than minimum free space, then it can choose the disk to begin to write a file, and if it runs out of space when writing the file then you get errors. It can't know how large a file will become when it decides which disk to write it to. So you must make it choose another disk by setting Minimum Free larger than the largest file you will write.

 

Oh I see, that makes total sense, I basically, set it and forget it. I'll sit down and configure that properly tonight, thanks for the guidance, it should help me fix this issue.

 

 

[John_M]

Do that by all means, because it's important to get it right. But, in addition, since you're in the process of copying a large amount of data to your array you might want to consider temporarily bypassing the cache (by setting Use cache disk to No for that share) and writing directly to the array, but with "turbo write" enabled (in Settings -> Disk Settings change Tunable (md_write_method) to reconstruct write). That will spin up all the disks but speed up writing to the array significantly. It should save you a lot of time. Once you've copied your initial 9 TB of files you can re-enable the cache and disable turbo write if you want to.

[KombatJam]

So the data transfer is all done, throw in a few gigabit connections, 4 USB 3 externals, and about 8TB in no time. Rebuilt the array by removing the parity drive (temp until data was done), enable turbo write, transferred all the stuff (from 30-40MB/s to 150-200MB/s (x2) is a big plus).

 

Once done, I changed the cache minimum size properly this time (second attempt as I added one too many zeros), enabled the parity check, and it's currently at 80% done in it's backup. I'd be transferring data until saturday afternoon if I didn't find this out!

 

Thanks again for all your help, it really saved me a headache, as well as get things right from the early stages vs down the road.