when you mix and match hard drive caches is there performance differences with unraid?

[comet424]

so my hard drives are WD gold or ultrastar with a 512mb cache.. but they harder to come by now

if i go with a seagate  they are only 256mb caches tehy dont have offer 512mb 

 

do you notice performance  issues or does unraid compensate etc?

 

and all my drives  are 7200 rpm

so i looking at Seagate 7200 rpm drives  since WD/Ultrastar  are harder to come by  in the Gold/Ultrastar brand

 

[Squid]

Unraid doesn't stripe data across disks, so you're always going to be limited by the read performance of any given drive.  That being said, for most people's use cases on the array with large media files, cache size is irrelevant.

[comet424]

oh ok  so the cache is only for striping data so then it doesnt matter.. here i thought 512mb  was always better for unraid thats how i was thinking that it would copy files faster or plex would run just that extra faster.. 

 

so cache  then is only good for raid then ...

 

as i was going with  Gold and Ultrastar  over  RED Nas drives as i wanted reliablity and performance.. 

but really no performance then between caches  only if i did a Raid 0  or other raid then

 

[Squid]

No.  Caching on a spinner is used so that commonly used small files will stay in the cache and not be read again from the platters.  It's why the drive manufacturers all state up to 6gbs transfer rate, and yet the sustained rate is much, much less.

 

With an OS installed on the drive, where they are tons and tons of small files read over and over again, it makes a measurable difference.  With media, not so much.

[comet424]

ah ok  and now my computer  store  doesnt show you can buy SATA drives now they replacing SATA with SAS drives  so now i gotta upgrade mystuff.. are we leaving SATA for SAS now..

 

so small commonly used files   that means files less then 256 or 512mb  then right..  so  like a i dunno   some windows file it changes  thats small..  

so even me copying pictures  or documents moving files around  on the array..   the cache does nothing for it either..   becaue they are bigger then 512mb  or if you were to access music  it have to be the same music song  right  other then that it has to read off the platter.. and when does it go from the cache back to the platter...

 

so then it really doesnt matter just as long as the Drives are 7200rpms  and i normally get the drive with 5 yr warrenty...     but i guess to get best performance across array too..  stick with 7200 or stick with 5400  dont mix and match as speeds would vary correct

 

learn something new everyday

[arturovf]

3 hours ago, Squid said:

No.  Caching on a spinner is used so that commonly used small files will stay in the cache and not be read again from the platters.  It's why the drive manufacturers all state up to 6gbs transfer rate, and yet the sustained rate is much, much less.

 

With an OS installed on the drive, where they are tons and tons of small files read over and over again, it makes a measurable difference.  With media, not so much.

TIL thank you

[codefaux]

OP can disregard this as off-topic because I wasn't paying attention but it's a lot of important information so I'll leave it here for historical purposes, and might reference it in the future. My bad, sorry.

 

 

 

 

In a normal cache system, important data -- not files or folders, just blocks -- is kept available someplace performant. If you need to write lots, it's something that writes fast. If you need to read lots, it's something that reads fast. Not ALWAYS both the same devices, but usually. Linux uses your RAM as cache, and frees it only as needed -- that's why you can install 72gb of RAM and only have 8gb free on a server running Linux -- you'll see that in most cases, the VAST majority of it is stored in cache, because it's fast and that space wasn't being used.

 

Filesystem cache is different -- generally that's a faster disk that stores files needed most often, or that need to be selected from swiftly, or similar. Swap space is also technically a form of cache, it's where Linux's kernel moves RAM that can't really be freed but isn't needed. That's off topic.

 

 

unRAID's definition of a Cache drive is a little different than the "cache drive" most people know of. To break it down, we need a deeper dive into unRAID.

 

At its core, unRAID is a bunch of partitions mapped onto virtual devices run against a Parity drive or two, with a Cache drive for Important Performance Reasons I'll get into below.

 

Each drive is formatted, typically XFS. Those drives are all mounted in folders under /mnt/disk/disk__ -- DON'T TOUCH though, seriously, it's managed by things that deeply expect to not be disturbed.

 

From there, another process layers all of those individual drives together to form your array. It manages spreading files between disks, reading the file you asked for from the right disk, keeping track of timestamps and metadata -- seriously, a busy process.

 

On top of that, those virtual devices I mentioned are assembled together with your Parity drive -- you're using a Parity drive, right? Data security is important, stop squirming and invest, it's actually the most important thing about unRAID to me. Anyway -- together with your Parity drive, they keep your data safe. A write to the actual device triggers an update to your parity device through one of two means, which keeps your data SAFE.

 

The problem is, between the busy process and the safe data, it can be a little....slow, sometimes. Especially with big arrays. More disks equals more places to shuffle for the filesystem manager, and more data to shuffle and math for Parity. This takes time, which slows down writes especially. It also slows down frequent/expansive reads, like directory scans and other edge cases.

 

Enter, unRAID's implementation of a Cache drive. It's not smart, it's not managed the same as a block or filesystem cache, but it works well and here's how.

 

When you have Cache enabled and you write a hundred files into that Share folder that's spread across nine drives with a bunch of other files in it, unRAID's filesystem manager performs admirably, but it can get bogged down -- justifiably. The Cache drive sits in front, instead. Any writes to a Share with Cache enabled, writes first to Cache. Cache fills up, but it's a single drive (in most cases, but pooled drives don't suffer the same spanning/parity problem, and IMPORTANTLY are not protected by Parity and we'll cover why that's a big deal in a second this is already a long note)  so it doesn't choke as readily. Eventually this fills up -- and if your Cache drive isn't large enough, it could fill up while you're using it and hurt performance worse than if it weren't there. Filling up is the Mover's job, and hurting performance is an important side effect to consider.

 

Mover does what it says on the tin, if the tin had some fine print. Mover takes Cache and moves it onto your array, where the Filesystem Manager distributes it according to your rules. If it happens while you're using the array heavily, it'll slow you down. Schedule it to take that into acount. Find that at Settings, Scheduler, Mover Settings.

 

Whew, still with me? I'm getting drymouth writing this.

 

SO. Cache saves performance most significantly for writes to heavily split folders. That's it, right?

 

Not quite. unRAID takes advantage of it and stores more there, which is where the lack of Parity becomes important, and where extra performance gains come in. It's important to have a Cache drive.

 

 

Ignore the first three folders, those are caches of my shares. You also see appdata/ domains/ and system/ -- those may or may not be important, depending on your setup.

 

appdata/ is where most Docker containers' volumes are stored by default. I won't get into it but for the uninitiated, Docker runs services in a really flexible and easy-to-deply way. Learn more. Volumes are the part that stores unique data, aka system-specific, databases, config files, etc.

domains/ is where VMs live

system/ is where the docker.img file is stored. unRAID uses docker.img to house the containers, layout, runtime information, and other things.

 

appdata/ is only important if the information in your Docker containers is A) Not on the Array, and B) Important or unrecoverable. It IS important that it's fast though.

domains/ is important if you have VMs with anything important on them.

system/ is only important if re-installing your Docker containers from templates is going to be a headache, or if you incorrectly store data inside a docker container without a volume mounted but there are better ways to save it and that's NOT THIS POST.

 

 

 

 

Anyway. Your READ performance of large files will be limited to the data throughput of the drive its files are on. Your read performance of small files will be limited to the seek performance of the drives they're scattered across AND the ability of those drives to rapidly return the information AND the Filesystem Manager's ability to crunch -- aka RAM and single-threaded process limitations.

 

 

If you have lots of small files or need a really fast Cache for Docker performance, VMs using disk images, or bulk writes, use a fast spinner Cache. 

If you need ultrafast VMs and/or Docker containers, consider passthrough disks/controllers or NVME/SSD cache.

 

Writing to a cached share will write to Cache until its minimum free space is reached (Settings, Global Share Settings, Cache Settings -- not the most intuitive) at which point the manager will write to the array.

 

Shares can be set Cache ONLY, Cache Preferred, Cache enabled (Yes) or disabled (no) -- Shares, Share Settings, click your share, "Use Cache" -- for performance reasons, IE if you write too big to cache frequently, if you don't care about write speed to certain arrays, if you'd rather not burn out your SSD, etc etc etc..

 

Whatever the reason, now you understand how unRAID UNIQUELY defines and uses Cache, why it's important TO understand, and what's there in case something goes wrong.

 

 

 

 

Holy crap. Did I miss anything? I'm trying to make this comprehensive so I can refer others here if needed.

Edited April 29, 2021 by codefaux
Not paying attention

[SimonF]

43 minutes ago, codefaux said:

cache

This thread is about physical cache in the drives not the unraid cache.

[codefaux]

5 minutes ago, SimonF said:

This thread is about physical cache in the drives not the unraid cache.

So it is. Not sure how I missed that. I'll see myself out, lol

[comet424]

@codefaux

 

sorry the delay some reason i didnt get the notifications till now i logged into unraid.. and its going to be a long reply bear with me

 

 

so thats alot of info and i have to re read it i disleixic  and need to re read things multiple times sometimes to understand

 

now i had questions

to answer the one i remember off hand

ya i use a Parity drive i tried replacing my drives one at a time  from 2TB and 2.5TB drives to 12.. my Parity is a 14TB drive and sitll have 2 2TB drives left so i have like 52TB array nothing big and ya i couldnt get 12 at the time  so  when i learned and understood parity i put the 14TB drive as the parity.. and loose that 2 TB 

 

as i back up my sisters files  and have noticed  guess its be for a different topic .. my sister lives couple hours away from me so i usually get her to fill a 2TB drive of her crap stuff from 3.5 diskettes  or her desktop  etc.. and meet once a week or month half way and just swap drives..  when i slapt that drive into unraid first i have to run that safe permissions things  as i always get admin issues trying to move or sometimes copy..  but if the drive is full  i can not move all the files just select all and move in krusader..  pain in butt  i have plenty of space on the array  the disk my sister gives cant move..   but if i do in windows and go from 2 Explorer windows i can do it  no issue but it seems slower and i like krusader  as i can just do it there.. unless i thinking now just use a VM  but whatever.. not sure why but i have to do  small batchs of selections  to grab and move instead or copy..   krusdader tells me i have not enough space but i got plenty..  but not sure.. and not sure if a Cache drive  would help that situation

 

so back to my questions

you touched up on a couple things i though from reading in past and avoided cache drive

 

1.  i have tried the cache drive  i had tried a 1TB or 500GB  drive  i followed the space invader videos   i think it was his in past..  and i was copying files from my computers back up windows  and external drives of 2 TBs 1 TBs etc.. just dumped it on the array.. and after a while id get and error i couldnt copy anymore  guess this was 2019..  i forget what version of unraid i was using.. but i found out.. i dunno what settings cache i had.. but i guess once cache drive was full..  Windows cancelled out copying told me not enough disk space etc.  would no longer let me copy files  and at that time i had like 10TB free for example..   and i could no longer copy to the array told me like disk full or something very frustrating..  dunno if its improved ..  had to click the mover button to move the files off..  was frustrating... so that was why 1 reason i didnt go with a cache ssd got fed up  telling me full. i thought maybe while its copying to the cache.. it would also be offloading it to the array  same time as if there is a pause in copying files it gives it a time to offload ing it so like 2 min  after your last copy  its now  moving to the array.. and then i could move more files  but then i learned you gotta schedule at like 12am or whatever  right..

and thats also why i thought going with 512mb cache  WD 12TB Drives was better because 256mb cache smaller and i figured i get better performance over Seagate etc.. but i learned i guess that doesnt matter

 

2..  another reason i didnt choose cache.  as it takes me along time to afford a hard drive and i read you can burn out these SSDs where as hard drive you dont really plus now you got that parity..  and since i simple low budget earning home owner.. loosing a SSD  at the price they are i couldnt afford to use it as a Cache where it drains the life out of the drive  only so many read/writes... as i already burned out  Pfsense USB  i had  i found out it was logging  and in within 3 weeks of the install  it burned the usb out.. another reason i was hesitent for unraid  i cant install it to the hard drive so it wont burn out the usb..  which has happened already once to me...

 

3...

so with appdata/dockers/domains/  you mentioned no protection on cache  and id have to re read everything over again.. but ah i forgotten lol

and about performance

 

but i find my Dockers are slow  and its on the Cache.. in the browser if it sits a long time  not using gui... its slow when i choose it  plugins always fast.. but docker slow like its spinning up  but once using it you can flip back and forth to Docker from another tab  and its fast..  

and this is using a NVME Gen 3 Xpg 1TB ssd as i though going from  from a WD SATA 520mb SSD   to this 3500/3000mb/s NVME would speed things up..

but it didnt help still slow

and i thought it would speed up PLEX read the database when you load it up  on the Xbox .. but i found no speed performance from when the Appdata was on the array or on this 3500mbs ssd nvme so it frustrates me..  

 

so i have like 52tb nothing big and i back up my parents my stuff and my sisters stuff

12tbs 2tbs data drives and a 14TB parity drive

i use 1 nvme 3500/3000mbs cache drive well really not cache i dont allow copying it to it i didnt want it filled up and i didnt want it wearing out i needed to last 

so i have 

appdata

vms    renamed from domains

docker

isos

system 

 

and they all have the not protected from array.. so i have that backup appdata... and i snot sure  as i read also  if you have 2 caches   if 1 goes the 2nd one craps out there is not parity for cache. so i read no point in 2 drives..   but then i read you can stripe i dunno..  all of it gets confusing as i only learned of unraid in 2018  as i was migrating from Windows Home Server 2003  and i couldnt get it in windows anymore..

 

but i also seen how spaceinvader i think when i watched his videos

he had a ssd/nvme 

1 for vms

1 for aoodata/dockers/isos/systems

1 for cache

 

is that the only way to get better performance? and does it matter what brands you get.. as i wanted to get the best performance a home user can get.. as i not a business so i cant purchase multi ssds hard drives 

 

so i still somewhat confused but i understand things better i more of a visual learner then reader so  i need to re read things

 

i also tried the transsmission question i had about downloading to the array to cache or so.. where was told to use cache drive i didnt have a SSD  and i seen a recent spaceinvader where you set it to prefer  or something i have troubles always remember so i have to also re watch things multiple times to understand.. due to some brain issues from cancer i cant remember everything like most or dont learn as fast but i try...

but i tried  adding a 2tb regular spinner as a cache drive for downloading   so it can keep the array shut off as i dunno  is it better to have the array running all the time or no  shut it off  as people say  on off wears out the drives faster i dunno..

but i did it different from space invaders 

 

and would using a 1 single spinner drive be faster then  copying to the array?  as you mentioned with the parity and the way copying works does slow things down least thats how i read it but i have to re read everything again

my drives are all for 12TB 512mbs  cache 14TB is 512mbs cache  but the 2 2TB drives are 256 or 128mb cache  so i never sure if that really matters but was told it doesnt from earlier post  as im not using it in raid  but single... 

 

reason i really like unraid over free nas  i dont need to buy 3 or so 12TB drives at once  just one every 6 months to a year  cant afford..  so im used to slow speeds  i dont know what a stripped speeds are etc.. i did try years ago 2 stripped data on Windows XP years ago before unraid was invented  and harware raid on my Asus board failed  i got a different motherboard  plugged it and i couldnt get my files   apparently i needed exact board and raid controller or something..  ever since then  i stayed away from Raid. and lived with the performance i got...  and liked how Windows Home Server 2003 worked 1 share and it spanned it over 4 hard drives..

 

sorry for the long post.. i have more questions but right now i wrote a novel.. my dislexia/learning disability has also run on sentences im sure and parts you may no understand.. to me i can understand it but others may not..   as i still wanna improve my performance if i can.. as i also try to make sure no errors in system log  or no issues in the fix common issues

 

 

[codefaux]

I thnk it may be best to break this into one smaller question at a time, in single-themed threads, with a bit less backstory and a bit more question. It might also be easier for you to keep track of relevant information coming back from helpful posters, if it's split into separate threads. If you'd prefer not to have separate threads, we can certainly continue here still.

 

I'd love to answer your questions in more detail, but with such a large single item to reply to, it can be rather difficult. I have difficulty keeping information straight sometimes, myself -- as evidenced by a rambling explosion of information to an entirely different question, lol.

 

I sincerely don't mean to be patronizing by saying this, so try not to be offended, but: If you'd like to try again, with one topic per message, I'll be much more able to help.

[JorgeB]

21 hours ago, comet424 said:

i forget what version of unraid i was using.. but i found out.. i dunno what settings cache i had.. but i guess once cache drive was full..  Windows cancelled out copying told me not enough disk space etc.

This will happen if the share's minimum free space setting is not correctly set, it should be set to twice the max file size you expect to copy to that share.

[comet424]

@codefaux  i wouldnt even know where to start or break it down  i just had so many questions/issues and things i seen towards caching and speeding up my transfers, to the multiple caches and the issues i all had.. 

i know i wrote a novel..

 

should been called the issues i having and how do you break it all down to fix it to proper.. i have had unraid now a 2018 2019  and i sitll new  i stopped using  linux 20 yrs ago i got fed up with Redhat in college  trying to mount a damn CDrom  every frigin time in the gui desktop i had enough and stuck with windows lol

 

@JorgeB  oh ok ill take a look.. i just been using default settings.. i not one to muck with settings leave the default...   i dont understand alot of the settings at times.. and im a visualer learner over a words reader...  and there is no real  Dummies videos  for all the settings  to understand..  like the yellow Dummies books..  although the spaceinvader videos  have thought me alot..  sometimes i wish there was live chat live like msn messanger discord facebook  where u can live chat and ask questions as your doing it.. 

but hey   still learning

 

so  my minum free space setting is set to  "0KB"

thats for all the shares... but when i copy to external hard drive its an unassigned drive   it doesnt have a share... or when i copy from it

 

so if its set to 0KB  the 1 share

and Array has 4TB free

and 2TB or sometimes 1TB  of stuff if i Moving from the external hard drive to the array it should work

 

or do you mean    that i should change that 0Kb to  the size of the External drive  so 2TB  i should set the free space too  or set it to 10TB  so there is no issues

 

[Squid]

 

3 hours ago, comet424 said:

sometimes i wish there was live chat live like msn messanger discord facebook  where u can live chat and ask questions as your doing it.. 

 

[JorgeB]

5 hours ago, comet424 said:

so  my minum free space setting is set to  "0KB"

If the shares are set to 0Kb any copy to the array that uses cache may run out of space when the cache if full, since cache floor default is only set at 2GB.

 

On 5/4/2021 at 12:15 PM, JorgeB said:

it should be set to twice the max file size you expect to copy to that share.

 

[comet424]

@JorgeB  i dont use a cache at the moment  for copying

all my shares look like in the pic..  i use the downloads folder to dump my sisters backups

but when in Krusader   i cant move entire 2TB of data off her drive...  lot of times i cant even move 100GB i gotta break it up into 50GB 

so in krusaderr  it kicks me back saying i dont have enough diskspace  either its on the array or on the external hd .. but  i mjust moving off the external hard drive  so disk space shouldnt matter..   but in Windows if i load up the 2 shares  i can  easily move the entire hard drive over..  but i like using krusader  minus  the damn rights  thing. i gotta keep resetting the rights in Tools  as it wont let me move files off the download dump location i have basiclly a temp folder.. and i sort her stuff.. but every time.. but i like the krusdar  works  out in background and its quick..  i was thinking just running a Windows VM  and do 2 shares and move it that way  avoid permissions issues etc.. as i thought these permission issues were for someone using XFS BTRFS external drive but i set it to NTFS  so avoid permissions and be able to shove into a windows laptop or her unraid and move files..

 

and that 2nd cache pool  thats what i experimenting with  from this topic  i didnt have a CAche SSD  so i tried  4TB to dump anything i download  etc to keep the array from spinning and keeping that parity drive healthy   so i practicing that way..

 

and the actual cache you see wehre i said also i find Docker  has a slow speed loading up but once its usually working the page loads up quick.. but plugins  page is always fast 

 

so thats why i was also asking in 1 of my questions  above to speed up unraid

do i need

1 SSD  - Appdata

1 SSD - VMS

1 SSD - Docker

1 SSD - CACHE

1 SSD (maybe for plex to speed it up?)

 

instead of having all those minus the CACHE on 1 SSD..

its a XP 3500/3000mb/s nvme 

i figured also if i have plex on its own SSD  you wouldnt have to see the loading up times in XBox sometimes takes long time so i dunno

 

i dont have ample money either so i just trying to learn cache  learn how to improve speeds  and tired sooo many people on when you search google  they hate unraid slow etc go with freenas they say  when i try to google improve speeds etc.. 

i like it just the way it is just trying to figure how to maximise the speed 

 

[comet424]

@Squid

Ok thank you squid ill add that ..  i appreciate it 🙂

[JorgeB]

7 minutes ago, comet424 said:

i dont use a cache at the moment  for copying

I was responding to this:

 

On 5/3/2021 at 2:27 PM, comet424 said:

1.  i have tried the cache drive  i had tried a 1TB or 500GB  drive  i followed the space invader videos   i think it was his in past..  and i was copying files from my computers back up windows  and external drives of 2 TBs 1 TBs etc.. just dumped it on the array.. and after a while id get and error i couldnt copy anymore  guess this was 2019..  i forget what version of unraid i was using.. but i found out.. i dunno what settings cache i had.. but i guess once cache drive was full..  Windows cancelled out copying told me not enough disk space etc. 

 

But if you don't have cache you still need to selected the correct setting, or will still get not enough space errors once disks start filling up, depending on allocation method.

 

10 minutes ago, comet424 said:

but when in Krusader   i cant move entire 2TB of data off her drive...  lot of times i cant even move 100GB i gotta break it up into 50GB 

That sounds like a Krusader issue, but I never used it.

[comet424]

@JorgeB  ah ok  so the 0KB is wrong  would i set the minum free space  to 100TB   this way you never have a problem and if you were to select the entire array and move it it wont have a problem?

 

and is there another program just as good like krusader..  im used to Krusdaer  as i used to use   in 90s early 200s   PC -Anywhere  so it was the same setup

 

[JorgeB]

54 minutes ago, comet424 said:

would i set the minum free space  to 100TB 

 

7 hours ago, JorgeB said:

it should be set to twice the max file size you expect to copy to that share.

 

[comet424]

well i guess from that reply  setting minum  to 100TB then then i have no issue period of copying nothing  or set it to 200TB  and then i wont have issue from years to come as i wont ever reach it

 

[JorgeB]

If you set it to that you'll get a not enough space error for any transfer, that's the minimum free space for disk in that share, when that threshold is met it starts writing to another disk, and there are no 200TB disks.

[comet424]

well then i confused..   so then i had it correct   0kb     i shouldnt have any issues 

way i read it and way you replied is

 

you set the minum  to the highest    by doubling it

so if i set to 200TB  i can move  2-4TB  off hte array in Krusader  with 0 issues  of  no diskspace  left on  the external drive   when i not moving to the external drive

 

so then i just leave it at 0kb

 

i simpliest     if  0KB =  unilimited file size    then that be easier

then  or  is it -1  i just want the exact setting for     Unlimited   file size and 

 

explain it for someone that is a noob      i learn  -1   equals   unlimited   

so do i set   0kb    to  -1kb  

 

since you didnt say yes or no my downloads share was configured right or not 

 

or if it is correct  then its 100% a krusader/unraid problem

as  i guessing here 

 

 

[itimpi]

I think you misunderstand that setting completely.     It is the amount of free space that allows UnRaid to select the disk for a new file.     There is a longer write up here in the online documentation accessible via the Manual link at the bottom of the UnRaid GUI

[kizer]

Ok lets rewind here a bit. 

 

Lets say your ripping DVDs and they are typically in the 4-8GB file size. 

So I would set your Minimum free space for your Cache Drive in the cache Settings: to around 16GB ....... NOT your Share Settings that is different < --- I personally have a 1TB SSD so I'm not really worried about giving up that much space 

You have a share that is called Movies 

 

Now in your Share you have a few choices regarding Shares and the use of your Cache which also utilizes the settings mentioned above.

- No Does not use Cache {Go straight to array for safety}

- Yes - Rights to Cache and then to Array if there is no more available space using setting mentioned above. {Speed}

- Only only to Cache which I use for my Dockers {Dockers, VMS}

- Prefer indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the array and onto the Cache disk/pool. {Maybe things that are accessed often and you want instant Access all the time, but not protected by parity}

 

So now files that are copied to your machine will first hit your Cache which could be an SSD or Spinner your choice. They sit on the Cache and then are moved to your array on a scheduled time of your choosing. It should be at max speed since the SSD is on and ready or your Spinner since its up and running. 

 

Your share settings if your confused is how you limit how much is written to your Share before it stops putting files on the disc and uses others.