One really nice feature of UnRAID is the ability to keep unneeded drives spun down.  HOWEVER, with a large share that spans a lot of drives (e.g. "DVDs"),  directory accesses cause most of those drives to spin up.  This seems completely unnecessary with the size of modern USB flash drives.  Why not have an option to cache the user share directories on the flash drive -- and get the directory info from there.  You could require a spare GB on the flash drive -- not a problem with most new units ... and you can keep a LOT of directory info in a GB      ... probably the complete directory structure for a full 16-drive array  [Although if you simply kept the user shares info cached that would solve this issue].

 

To keep it simple, I'd only allow it if there's a full spare GB on the flash unit ... enough that there's essentially no chance of running out of space in the cache.  A 4GB or larger unit would easily allow this.

 

This is absolutely my #1 choice for an improvement -- it would allow UnRAID to only spinup the individual drive needed for any specified file access.

 

 

The other key features I'd like to see are:

 

(a) a SMART display button for each disk, to display the SMART data (Yes, I know there are Linux utilities that do this ... but it'd be SO nice if this was simply part of the user interface, so no Linux knowledge was necessary.

 

(b)  a thermal spindown value (either global or per disk) as has been suggested in several posts on this forum.  This would be a nice feature to help protect against badly overheated disks.

 

©  a "don't auto allocate" choice for individual disks that are included in user shares.  I REALLY like the user shares ... lets me have a "\\Media\DVDs\..." share that provides a uniform access method for all of my DVDs no matter which disks they're on.    But I'd like to keep certain categories together ... episodic series;  movies of a certain type (James Bond, John Wayne, etc.) --> so when I add those I like to do so manually (to a specified directory that groups these together).  I'd like to have a disk (or two) that is essentially "reserved" for these ... but I want it to be part of the user share (DVDs).  But I'd like that disk to NOT be used when simply copying files to the user share.

 

Caching dirs can be "dumb" or "smart."

 

Dumb, is just read them in and keep them.  When there is a change, read them all in again.  While this sounds silly, it works perfectly for read-only media.

 

Smart means you manage individual entries and when writes are done, know which is dirty and re-reread just that one.  That is hard.

 

Couple this with © in your post, and I could see a "read only" option for some drives in a user share.  The read-only ones could easily support dumb directory cache, and never need to be spun up unless reading from a file on them.

 

As for (a) and (b), there are some excellent addins for unRAID that provide these features and more, such as unMENU, smarthistory, and unRAID-Web.  They are getting better all the time, and are approaching the "one-click install" paradigm.

 

For folks who "never want to 'see' Linux" there can be a way to install these addins from a web page.  Basically you click on a link in your browser, and it will download a program/script that does all the work for you in the background, including logging into your unRAID server, downloading the necessary files, and installing them.  I have this almost working!

I have ~900 movies in ISO format spanning 5 drives.  I expose them on my network as a read-only share.  the 'dumb' version of a directory cache would really be great. I feel garycase's pain in that when the drives are spun down it takes forever for them to spin up before the share shows the files.  And, it does seem like a waste to have to spin them all up just to read a single file off of one of the drives.  If it's really true that read-only shares could easily support a dumb directory cache, I too would be interested in this.

Agree an active (e.g. "smart") cache is more difficult to implement -- but UnRAID is already well-aware of all writes, as it's managing the parity updates, so it "knows" which disks have changed and could relatively easily maintain an up-to-date cache without unnecessary drive spin-ups => just re-read the directory for any drive that's just been written to (which is obviously already spun-up).  But even a "dumb" cache would be FAR preferable to what happens now.  Most of us who use UnRAID as a media server do relatively few writes once we get our collections loaded.

 

Note that a "read-only" disk from a user share perspective is fine -- as long as writes work if the disk is directly accessed [e.g. if disk3 was "read-only" r.e. the user shares, then writes to \\Media\DVDs would not use disk3; but writes to \\Media\disk3\DVDs would work just fine.

 

I bought UnRAID primarily for its ability to easily mix/match drives and the ability to use a single user share that spans those drives.  Although I'm very PC-literate (had PC's since 1975), I've procrastinated big-time with regards to learning Linux ... so am definitely in the "never want to see Linux" crowd -- that's why I bought UnRAID      But I don't mind copying a few files as needed to the flash ... I just don't want to reduce in any way the stability of UnRAID.  Do the add-ons you've developed work through the web interface once installed?

 

 

I would love to see this ass well. As others have said, it really stinks to have all of your drives spin up just to watch one movie. But hey Unraid has come a long way for me...I use to have all of my movies stored on the HTPC and if I lost a drive I lost my movies....so I will deal with it for now but hopefully in the future....

Just a small note that the ls -R trick still working great here. No spin ups for complete browsing 14 drives

For some reason I cannot get the LS-r trick to work at all for me. I can see at bootup that it runs, I spin down all drives, call a movie directly without browsing (\\tower\Movies\movie.mkv) and it spins up almost all of the drives. Weird thing is I know the movie is on Disk5, I see it spinning up drives 1 by 1 including disk5 and going all the way to disk 9 then back to 5 to play it.

 

Wish I could get it working!!!!

Just a small note that the ls -R trick still working great here. No spin ups for complete browsing 14 drives

 

Hello,

what is the "ls -R" trick ?

you need to do it non stop for it to work.

 

theres not much point in me posting details on the trick here there a whole thread dedicated to stuff like it. If you cant find it post again and i will rake it out

Mine is running every 10sec. should it be more?

no that should be fine unless you reading/writing data non stop

Hello,

what is the "ls -R" trick ?

 

Follow this linik to the Best of the Forums, "How To" section.

 

A little more than 1/2 way down you'll find a link titled "How to cache directory entries (avoid disk spinups when searching for files) ".

 

The details of this technique are provided there.

Ok... I ready the "How to" post... and am a bit confused as to what exactly I should do to "cache" directories.  Simply do the 'ls' trick, or also do the "sysctl vm.vfs_cache_pressure=0" setting (which i'm not sure what it even does).

 

 

do both. the cache pressure thing is a tuneable to make the ls trick work better. (over simplified but essentially true)

Thanks... and is there any real improvement if a Swap drive/file is setup?  I have 2GB of RAM.

 

Thanks... and is there any real improvement if a Swap drive/file is setup?  I have 2GB of RAM.

 

Sorry, but no.  unless of course you are using up all of memory and crashing... Then it might delay the crash till swap fills too.

 

Joe L.

There could be. If the swap was on the cache drive you could probasbly in theory stop a video, go to bed do some stuff and continue to watch the video from ram and swap.

 

I often see this happening just with more ram.

 

OK so its not show stopping improvement but it aint bad especially since cache drive will probably be up alot more than the other data drives.

What I'm hoping, is that if I add swap space that the 'ls' trick will work better (won't be "pushed out" of the cache as quickly)... ??

try and see. :0

 

rememebr you can use swap files you dont need a partition

I too read the whole post.  I have installed JoeL’s script to run (via cron) every minute and directory browsing of my user shares has certainly improved.  A couple of things:

 

1) You have to make sure that you don’t have duplicate files spanning your shares.  Either look for them and fix them or modify Joe’s script so that it exits if dup’s are found or else you’ll have log file explosion.

 

2) I too wish there were more explanation about the “sysctl vm.vfs_cache_pressure=0” setting.  I don’t quite understand this.  I see in this thread that it is “tunable” (I’m assuming this means that I can set it to a value other than 0, but don’t understand why I should.) But I’m not even sure if I need to be setting this parameter at all, or why.

 

2) I too wish there were more explanation about the “sysctl vm.vfs_cache_pressure=0” setting.  I don’t quite understand this.  I see in this thread that it is “tunable” (I’m assuming this means that I can set it to a value other than 0, but don’t understand why I should.) But I’m not even sure if I need to be setting this parameter at all, or why.

 

I too would like to know more, I've been googling for a while now trying to learn more about it but can't seem to find coherent documentation.  Would anyone who understands the command mind sharing what they know about it?

 

Cheers,

Matt

 

this is what I have found:

 

- at vfs_cache_pressure=0 we don't shrink dcache and icache at all.

 

- at vfs_cache_pressure=100 there is no change in behaviour.

 

- at vfs_cache_pressure > 100 we reclaim dentries and inodes harder.

 

But have no idea what exactly is happening...

In simple terms it means "how hard should we try not to dump the directory cache when were running short of ram"

 

A value of zero should = never but it doesnt in real life. This is the setting you want if you are wanting to cache to save drive spin up.

 

I've added a raw entry for this directory caching topic to the Improving unRAID Performance wiki page (here).  It is more of a dump of the info, with links to all of the related discussions, not yet in an easy tutorial style, so be kind if you check it out before it gets polished.  And if you feel like doing the polishing/rewriting, PLEASE feel free, to edit in whatever way you feel is necessary.

 

To summarize it, see this post for Tom's introduction and explanation of "the 'vfs_cache_pressure' tunable".  For a summary of what needs to go in the go script, see this post, and if you continue reading that thread, you will see improved versions of the ls-r script.

To test the new beta properly i turned of ls -R on my unRAID box....

 

holy cow the difference in real world usability is unbelievable. I had forgotten how much better unRAID feels when using ls -R. It doesnt make it any faster but boy oh boy does it help with browsing and the perception of availability... and most of all disks spin up all over the damn place without it.

 

Really think this should be made official with all the various caveats