Cache drive

[aaronwt]

What happens when the cache drive gets full? Does it start writing directly to the array? I currently have a 1.5TB drive for my cache and want to change it to 500GB, but there might be some times when it gets full since I will be copying over several terabytes of data. I want to use the 1.5TB in the array.

 

Sent from my HTC Incredible using Tapatalk

[Joe L.]

you must set the

Min free

setting. The cache drive will be bypassed if that min amount is not free.

[SSD]

So basically - yes, it will copy to another disk in the user share if the cache disk does not have the minimum free space you configure.

 

But if you are trying to copy a file larger than the cache minimum disk free, and unRAID starts coping to the cache, and it runs out of space on the cache, it will NOT then switch to copying to an array disk.  If it starts to copy to cache and runs out of space, you'll get the normal "out of space" error, even if there is plenty of space on the array disks.

[aaronwt]

Thanks

 

Sent from my HTC Incredible using Tapatalk

[chuck23322]

So basically - yes, it will copy to another disk in the user share if the cache disk does not have the minimum free space you configure.

 

But if you are trying to copy a file larger than the cache minimum disk free, and unRAID starts coping to the cache, and it runs out of space on the cache, it will NOT then switch to copying to an array disk.  If it starts to copy to cache and runs out of space, you'll get the normal "out of space" error, even if there is plenty of space on the array disks.

 

Hate to bump a month old thread,  but I'm pondering something similar.

 

I'm planing on moving a SSD over to my UnRaid server to be a cache drive.   Normally, I don't do a lot of big copying to my unraid -- but occasionally I can copy an entire ISO of a blu-ray -- could be nearly 50gb.

 

Now, with my 64gb SSD,  I see how I would set it to be "minimum disk free" of 50gb.   So I never start copying unless there is at least 50gb free on the cache drive.

 

Are these assumptions correct:  {The numbers are just illustrative, I know it's +/- a little for the numbers}

 

1) Out of a 64gb SSD, with a 50gb "min free" setting -- basically I can send up to 64gb minus 50gb = about 14gb of data to it {assuming it hasn't off-loaded from the cache drive} before the 'next' file will bypass the cache and go straight to a real drive.

 

2) This infers that because I want to protect myself / be able to copy a 50gb iso -- that the most my cache drive can ever "hold" is up to 14gb (plus whatever size is the last file under 13.9gb of total data written to the cache drive) of completed, waiting to copy data.   Upon the next file being sent, it goes to a real disk.   So, on a "bad" day,  I have 13.9gb written to the cache, it hasn't moved yet, I can copy one more file, it'll go on to the cache, and send up to 50gb if I'm writing that ISO I described.   But, at 14gb, it'd copy the 'next' file onto a real drive.

 

3) If I happen to copy TWO big ISOs,  or anything bigger than 50gb + 14gb -- before the cache can off-load, I'll be getting a 'disk full' error.

 

4) The cache doesn't off-load until the file being written to it completes?  And the file handle closes Right?

 

5) If I copy a file, and the file handle closes.  But something else "opens" that file (say I play back the ISO right after copying it) -- does the cache -> real drive process stop again?  Or is that file unavailable while it's in transit from cache -> real drive?

 

 

Do I have the situation(s) right?

[lionelhutz]

The cache moves files on a schedule, usually at night around 3am.

 

You could  have a 50gig free space setting and then copy a 50gig file to the cache disk. But if you copied 2 15 gig files then the first one goes to the cache and the second one to the array. The next file copied after there is less than the min free space goes directly to the array. You'll basically get to copy 1 big ISO to the cache and then the next one goes to the array.

 

I'd say that size of cache is too small unless you only intend to ever copy 1 ISO at a time. Otherwise, you are better off with a cheap $40 500gig or 1t drive as a cache and having the files copy a little slower compared to the SSD but still quicker overall since they all go to the cache.

 

Peter

[chuck23322]

Otherwise, you are better off with a cheap $40 500gig or 1t drive as a cache and having the files copy a little slower compared to the SSD but still quicker overall since they all go to the cache.

 

Peter

 

Thanks Peter.  I got it on your comments on the size/space issues.  But you lost me {I'm missing something} on the 'but still quicker overall since they all go to the cache'

 

If I put in (or re-purpose) a 500gb drive as a cache (I have several) -- I'm missing then, how copying to that 500gb would be any faster than just not having a cache drive at all.

 

Or are you assuming that I buy a fancier higher-end 500gb drive specifically for the cache?

[PeterB]

But you lost me {I'm missing something} on the 'but still quicker overall since they all go to the cache'

 

Writing to the cache drive is faster than writing directly to the array (but your files are not protected until mover copies them to the array).

 

It's all very well having a very fast cache device but if it only has capacity for a small part of the data that you write in one day (or whatever your mover schedule is), then you gain very little because the first file might go to cache, but all the rest is written directly to the array.

 

Having a much larger, but slower, cache device may give you more benefit.

[chuck23322]

But you lost me {I'm missing something} on the 'but still quicker overall since they all go to the cache'

 

Writing to the cache drive is faster than writing directly to the array (but your files are not protected until mover copies them to the array).

 

It's all very well having a very fast cache device but if it only has capacity for a small part of the data that you write in one day (or whatever your mover schedule is), then you gain very little because the first file might go to cache, but all the rest is written directly to the array.

 

Having a much larger, but slower, cache device may give you more benefit.

 

Got it!  Thanks.

 

---

 

 

Ought to have read more

 

http://lime-technology.com/forum/index.php?topic=1731.msg11937#msg11937

 

"Since there is a lag between the time objects are created on the cache disk, and when they are moved to the array, it may be desireable to disable the cache disk for certain shares.  For this purpose, there is a setting for each user share to disable use of the cache disk for that share."

 

That works for me.  I don't need to really cache the ISO writes (the big ones) -- and that is it's own share(s)

 

At that point, I'm happy getting the full 'fast' SSD 64gb cache for routine use, and just disable the cache drive on certain shares.  A perfect compromise to me.

 

And for those finding this thread, a great starting point is here:

 

http://lime-technology.com/forum/index.php?topic=5754.msg54158#msg54158

[lionelhutz]

Thanks Peter.   I got it on your comments on the size/space issues.   But you lost me {I'm missing something} on the 'but still quicker overall since they all go to the cache'

 

First writes to the SSD cache will be a little quicker compared to a platter cache drive but then the rest of the writes go the array at a much slower speed compared to a platter cache drive.

 

The SSD cache really won't gain you too much compared to a platter cache unless you are willing to tweak both the sending computer and the unRAID machine to optimize the transfer speed. A larger platter will gain you about 2X the speed when writing all files including the big ISO files.

 

 

[aaronwt]

So basically - yes, it will copy to another disk in the user share if the cache disk does not have the minimum free space you configure.

 

But if you are trying to copy a file larger than the cache minimum disk free, and unRAID starts coping to the cache, and it runs out of space on the cache, it will NOT then switch to copying to an array disk.  If it starts to copy to cache and runs out of space, you'll get the normal "out of space" error, even if there is plenty of space on the array disks.

 

Hate to bump a month old thread,  but I'm pondering something similar.

 

I'm planing on moving a SSD over to my UnRaid server to be a cache drive.   Normally, I don't do a lot of big copying to my unraid -- but occasionally I can copy an entire ISO of a blu-ray -- could be nearly 50gb.

 

Now, with my 64gb SSD,  I see how I would set it to be "minimum disk free" of 50gb.   So I never start copying unless there is at least 50gb free on the cache drive.

 

Are these assumptions correct:  {The numbers are just illustrative, I know it's +/- a little for the numbers}

 

1) Out of a 64gb SSD, with a 50gb "min free" setting -- basically I can send up to 64gb minus 50gb = about 14gb of data to it {assuming it hasn't off-loaded from the cache drive} before the 'next' file will bypass the cache and go straight to a real drive.

 

2) This infers that because I want to protect myself / be able to copy a 50gb iso -- that the most my cache drive can ever "hold" is up to 14gb (plus whatever size is the last file under 13.9gb of total data written to the cache drive) of completed, waiting to copy data.   Upon the next file being sent, it goes to a real disk.   So, on a "bad" day,  I have 13.9gb written to the cache, it hasn't moved yet, I can copy one more file, it'll go on to the cache, and send up to 50gb if I'm writing that ISO I described.   But, at 14gb, it'd copy the 'next' file onto a real drive.

 

3) If I happen to copy TWO big ISOs,  or anything bigger than 50gb + 14gb -- before the cache can off-load, I'll be getting a 'disk full' error.

 

4) The cache doesn't off-load until the file being written to it completes?  And the file handle closes Right?

 

5) If I copy a file, and the file handle closes.  But something else "opens" that file (say I play back the ISO right after copying it) -- does the cache -> real drive process stop again?  Or is that file unavailable while it's in transit from cache -> real drive?

 

 

Do I have the situation(s) right?

 

Not much point in wasting an SSD for the cache drive. With a green hard drive it already comes very close to saturating a gigabit connection when being written to. An SSd won't do much especially when dealing with large files like BD ISOs.

 

I use a Seagate 2TB 5900rpm drive for my cache drive. I already get between 800 and 880mb/s transfer rates to the cache drive over my network. of course transferring from the cache to the Array will be much much slower.

 

I use a 2TB cache drive since it is the same size as my parity drive. If my Parity drive has an issue I can just make the cache drive my parity instead until I get a replacement.

 

Plus I also use my unRAID for all my Win7 machine backups and HD DVD ISOs(and the occassional BD ISO but I use my WHS mainly for that). I have my cache drive set to transfer to the array every eight hours so sometimes I will have alot of data to transfer if I've had several machines backup during that eight hour window.