To Cache drive or not to Cache drive?

[itimpi]

What happens if you fill up the cache drive before it gets a chance to move the files over? Will unraid just bypass the full cache drive and move the files into the array?

Yes.

 

You can configure the limit for the cache drive free space so that if it falls below this value then unRAID starts bypassing the cache drive.  You should set this to be more than the largest file you intend to copy.  The decision as to where to write the file is made at the point the file is first opened for writing, and if this limit is too low you may find a file starts being written to the cache and then fails because space runs out.

[WinHac]

Where do you adjust that setting at?

[dgaschk]

Settings->Share Settings

[WinHac]

Default is " 2000000 " How do I set it for 17GB?

[dgaschk]

17000000. The default units are kB. I think "17 GB" may work as well.

[mikeybunting]

Could someone tell me about how long it should take to format a 2TB WD drive being setup as my "cache" drive for the 1st time?

 

My server is unresponsive and it has been running about 30 minutes...it went unresponsive after the formatting started on the cache drive that I just added.

 

Many thanks!

 

Mike

[itimpi]

Could someone tell me about how long it should take to format a 2TB WD drive being setup as my "cache" drive for the 1st time?

 

My server is unresponsive and it has been running about 30 minutes...it went unresponsive after the formatting started on the cache drive that I just added.

That sounds far too long.  I would expect it to be a minute or less.

[JimPhreak]

How exactly does the cache drive work?  If I write files to my shares I assume it gets written to my cache drive first.  Does it then simultaneously start writing to my array or does it happen at certain times of the day (and if so how does this get configured)?

[itimpi]

How exactly does the cache drive work?  If I write files to my shares I assume it gets written to my cache drive first.  Does it then simultaneously start writing to my array or does it happen at certain times of the day (and if so how does this get configured)?

The move from cache to array is configured under the Settings tab Share option to start at a specified time (default is 3:40 AM).  The mover process can also be manually started immediately from there.

[JimPhreak]

How exactly does the cache drive work?  If I write files to my shares I assume it gets written to my cache drive first.  Does it then simultaneously start writing to my array or does it happen at certain times of the day (and if so how does this get configured)?

The move from cache to array is configured under the Settings tab Share option to start at a specified time (default is 3:40 AM).  The mover process can also be manually started immediately from there.

 

Thank you!

[Zonediver]

Hi folks,

I have a smale problem with my cache disk and need some help:

I changed my chache disk today from a 2.5" wd black to a 240GB SSD.

Now, the write speed is very slow - about 19MB.

With the wd black, i had 65MB.

My question now, are there some parameters to adjust for the chache drive or

Are there some other hints to improve the write speed?

 

Edit: I did some tests with hdparm and it shows me a rate of 285MB.

 

Thanks for your help

Zoni

[pdxrealtor1]

What happens if you fill up the cache drive before it gets a chance to move the files over? Will unraid just bypass the full cache drive and move the files into the array?

Yes.

 

You can configure the limit for the cache drive free space so that if it falls below this value then unRAID starts bypassing the cache drive.  You should set this to be more than the largest file you intend to copy.  The decision as to where to write the file is made at the point the file is first opened for writing, and if this limit is too low you may find a file starts being written to the cache and then fails because space runs out.

 

If unRAID will automatically bypass the cache drive and start copying to the array, what's the point of configuring the limit for the cache drive free space? Does it have to do with the way in which unRAID bypasses the cache drive or??

 

I have a cache drive and I"m trying to understand the ins and outs of getting it setup.

[itimpi]

If unRAID will automatically bypass the cache drive and start copying to the array, what's the point of configuring the limit for the cache drive free space? Does it have to do with the way in which unRAID bypasses the cache drive or??

 

I have a cache drive and I"m trying to understand the ins and outs of getting it setup.

The problem is that once unRAID picks a drive and starts writing a file to the drive it will not change which drive is to be used if while writing the file it runs out of space on the drive being written to - you will simply an error reported.  That is why the value for min-free-space should be equal to or larger than the largest file you are going to want to write.

[DaleWilliams]

itimpi has it. The 'mover' that takes files from cache and puts them on the array runs on a clock timer. It doesn't start moving as soon as a file shows up. Typically, the mover starts at 3:40 AM. By that time, you could have overrun cache drive. Neither Windows nor OS X handle the problem correctly if you fill up the cache drive, and a file is incompletely written. You have to clean up from the unRAID interface.  It can be messy.

 

If you have a minimum free space limit on the cache drive, then you never get in this situation.

If, for example, you never are likely to save anything larger than a DVD to unRAID's cache, just set the minspace to 5 Gig...

If your cache is nearly full, and you try to put another DVD onto the array,

then if there's not 5 Gig available on the cache, then it'll just write directly to the array. Slower, but safer.

[pdxrealtor1]

Thanks both ^^^ for the reply. It makes perfect sense now.

 

What would be a safe limit for bluray files? 60 GB? I have a 1TB cache drive so there's quite a bit of room to spare taking into consideration my burning habits. On a real busy day I might go through 12 bluray rips.

[Falc410]

Just a short question: Is the first post in this thread still correct today?

- Do I need a Cache drive for certain plugins?

- Does it really speed up write speeds? I only tested unRaid on old hardware without a cache drive and the write speeds were the same as if I would have written to a normal harddisk (WD Green had 60-70 MB/s). So I doubt a cache drive would be "much" faster than this.

 

The original post is quite old, as are the write speed examples written there. So I believe with a reasonable fast CPU the parity can be written on-the-fly without impacting the write process.

[dgaschk]

Writes to the the array are limited to about 35-40MB/s due to the rotational delay of the disks involved with updating parity. Writes to a cache drive are only limited by gig-e speed. There is an option in the latest versions that causes parity to be updated by reading from all disks. This should allow writes to proceed at the same speed as a parity check (70-110MB/s). However, all array disks must be spinning during any write.

[DaleWilliams]

- Do I need a Cache drive for certain plugins?

Probably not. I'm not heard of any that require a cache drive.

 

There are plugins which are nearly constantly writing to storage. (Plex, torrents, netnews, etc.)

As a consequence, One of your drives will always be spun up to accept the write.

If that's an array drive, then the Parity drive will ALSO be constantly spinning.

 

By putting the 'frequent writers' on a cache drive, and isolating them from the mover, only the cache drive is constantly running. Saves energy and wear and tear on your array's protected drives.

[Falc410]

I'm about to build two Unraid Servers, one for me and one for my dad. Both servers will serve videos and photos for XBMC. So there are rarely written to, mostly used to read from. I don't plan on any plugin in particular (no xbmc library update plugin since the videos are home records).

 

Would you still recommend a cache drive? And if I use one, let's assume I have a Movies Share on Unraid Server. Will the file be automatically copied to the Cache drive and be available right away at the Movie Share location even tough it has not yet been to moved to the physical disk? Or do I have to wait until the mover script moves it?

[DaleWilliams]

Once installed, from the user's point of view a 'cache' drive is invisible.

Files on the cache are automatically moved to the array. Once the copy TO the cache drive is complete, they are IMMEDIATELY available in the regular share. (although they are not yet protected by parity)

When they are still on the cache drive, to the user they appear to already be in the user share folder.

You edit, delete, change, read, copy, etc., just as if they are in the regular folder. unRAID does all that in the background.

 

Cache drives trade WRITE SPEED for parity protection. Until data is moved from cache to the array it is unprotected by parity.

 

Cache speeds up the WRITE operations...but 'READ' is from the normal share and runs at full speed. Some users also put certain plugins and apps on their cache drive, simply because they don't need them protected by the parity drive, or because they want to restrict the number of physical drives that are constantly drawing power by spinning.

 

From what you describe, you'll notice a speed up in setting up your two arrays at the beginning, but with just occasional writes there's not a lot of added value.  One thing you might consider at initial startup is setting up your arrays without assigning parity...copy all your photos and other data onto the array, and THEN assign parity. That way, the copies occur at full speed, and the parity can run overnight while you're asleep.  Later, incremental adds and changes sound like they'll be pretty minor.

[Falc410]

Thanks for the explanation. I guess I can really afford to not have a Cache Drive in that Scenario then.

[pdxrealtor1]

I just put a cache drive in and my limit is now the BD drive. I hear they have a faster one I wouldn't mind trying.

 

I'm now pegged at 21-22 MB/s on my rips. Worth it IMO.

 

I got a 1 TB 7200.

[figurefive]

Does anyone know why write speeds to my SSD cache drive would be so bad or what I could look at to try to figure it out? I'm very new to unraid and linux so I'm not really sure how to troubleshoot this.

 

When I'm writing to the cache drive, my speed starts out fine but after about 5 seconds they start becoming really slow and erratic. On the other hand when I write directly to one of the hard disks in the array, I get a pretty constant 100mb/s. I'm not currently using a parity drive because I'm waiting for it to arrive, but it seems to me that should make my speeds faster if anything.

 

Writing to the cache drive:

 

Writing directly to hdd in array:

[DaleWilliams]

Does anyone know why write speeds to my SSD cache drive would be so bad or what I could look at to try to figure it out? I'm very new to unraid and linux so I'm not really sure how to troubleshoot this.

I believe you, but your graphs don't seem right!

Most of us would *LOVE* to have 111 meg write speed to a regular hard drive.

And if we were only getting 28 meg from an SSD, we'd throw it out!

So you aren't calculating parity at all, correct?

There is a thread in the forums about SSD drives and how to make sure their internal format is properly aligned. It was a problem with older SSD...but I think the newer ones don't have that problem.  You might search the forums for it.  (something like: "SSD internal block align" or SSD format)

 

 

 

 

If you're still concerned, add some details about your configuration (number of drives, motherboard, RAM, etc.) so we know what you're working with.

Also, details about the SSD (make,  model, size, etc.)

And a Syslog from unRAID would be very helpful.

 

 

[RobJ]

A little off-topic, but have to say that it's great to see more very helpful people providing this level of support.  Your post Dale is exactly what I was hoping someone would come up with, as I came up relatively blank, yet knowing there were some good ideas and suggestions to be made, but couldn't think of them.