CACHE drive - if fills up on ONE file, then what?

[RobertP]

I have just installed a 1TB ssd cache drive.

I use my unRAID system to mostly store backups, which can get large for some of my drives.

If I have my user share set up to use cache, and the backup program starts out using the cache drive when it first begins the backup, what happens if the file ends up being too big to fit on the cache drive?  Does it get split between cache and the share (and eventually all moved to the share), or does it get moved to the share "mid-stream" during the backup, or does unRAID just throw up its hands and say "no-can-do" and thus the backup program aborts the backup?

I realize this may partially depend on how my backup program responds, but I'm wondering how unRAID acts in that situation?  Similar situation could happen if recording a huge video file on a cache that is already mostly full - what happens to that recording file at the "cache full" moment?  Obviously at the start of the backup (or video recording), the file size is unknown so it would start writing the file to the cache, but what happens when the cache fills up during a mid-file creation?  (This could happen if a SINGLE backup file is too big for the cache, or it could happen if doing multiple backups (and thus multiple files) and the cache fills up during the creation of the third or fourth file.)

Thanks,

Bob

[testdasi]

If a single file fills up the cache drive, it will throw an out of space error to your backup app, which usually will error out instead of splitting stuff out. (I'm assuming you want the backup file to be moved to the array by the mover).

In your use case, I would just write directly to the array (cache = No) with turbo write turned on.

 

[trurl]

20 minutes ago, RobertP said:

Similar situation could happen if recording a huge video file on a cache that is already mostly full - what happens to that recording file at the "cache full" moment?  Obviously at the start of the backup (or video recording), the file size is unknown so it would start writing the file to the cache, but what happens when the cache fills up during a mid-file creation?

Set Minimum Free for cache in Global Share Settings to larger than the largest file you expect to write to cache.

 

If a user share is set to cache-yes or cache-prefer, then if cache has less than Minimum Free, it will choose the array (overflow) to begin writing the file. Once it has chosen a disk to begin writing a file it won't change and will just give an error if it runs out of space.

 

There is a similar Minimum Free settings for each user share. If an array disk has less than minimum free unraid will choose another.

[RobertP]

Thank you.  The only problem - at the start of a backup (or video record), there is no way to be SURE what size the file will be.  One can make an educated guess, but can't be sure.  If I set it to my guesstimated biggest size, some of my FULL backups are more than 1TB so the cache would NEVER get used for any backups.  I guess, since I usually just do DIFFERENTIAL backups, I can set the cache "minimum free" size to a "typical large DIFF file size" - and if I do a full backup, then turn off cache for that share for that backup run (hoping I remember to do that!  lol ).
Thanks for the info on how unRAID will react.  That's kinda what I expected would happen, just wanted to verify.

[JonathanM]

32 minutes ago, RobertP said:

The only problem - at the start of a backup (or video record), there is no way to be SURE what size the file will be.

That is the root of the issue for Unraid's disk use as well. Since any single file must entirely fit on a single volume, even though the folders can span multiple volumes, you must guess correctly when setting the minimum free space or you will fill a volume and generate an error.

 

Unraid is not well suited for HUGE files. It works much better with backup software that has the capability to generate multiple discrete files for a single backup, which most backup software can be configured to do. Directly recording unbounded video files to disk is not friendly at all, for multiple reasons. Better to use an Unassigned Device as the target, and then manage the results from there.

 

If you truly need 100's of TB of single volume space, Unraid can be wrestled into compliance, but you will lose much of what makes Unraid work so well otherwise.

 

There are rumblings and rumours that things may change on that front, possibly Soon™. Maybe later this year.