Has anyone else noticed that when you go get a directory listing it takes a second or two for the drives to spin up? Does this slight delay bother anyone besides me?

 

I was wondering if it would be possible to have the directory tree in memory, to avoid spinning up all the drives (if you're using User Shares that have files across more than one drive).  Not sure how much of an additional memory hog such a cache would be (would depend on how many drives and files you have).  In any event, it could be a configurable (Cache Directory Contents: Yes/No) type setting.

 

The benefit would be instant directory listings, and no unnecessary spin-up of disks (and consequent wait for spin-down) of disks.

 

Thanks,

Ralph.

Yes, it bugs me too.  But given unRAID's fairly light memory requirements, I'm afraid adding the directory structure of each drive into memory would either:  a) force other needed features (not sure what) out of memory or b) significantly increase the minimum memory required.

 

It might be a nice option though (i.e., "Check the box if you'd like your directory structures loaded into memory - - warning - - an additional XXXMB of RAM is usually recommended").

You might try adding the following command to the end of your "go" script

sleep 30; find /mnt -print >/dev/null 2>/dev/null

 

It will force all the directories to be scanned and be in the buffer cache.

They should be readable even after the disks spin down.

I don't know of any way to lock specific blocks representing the directories in the RAM cache.

 

Joe L.

You might try adding the following command to the end of your "go" script

sleep 30; find /mnt -print >/dev/null 2>/dev/null

 

It will force all the directories to be scanned and be in the buffer cache.

They should be readable even after the disks spin down.

I don't know of any way to lock specific blocks representing the directories in the RAM cache.

 

Joe L.

 

I just tried it:

 

* Got a directory listing while the drives were spun down

* It took 3-4 seconds and forced a spin up

* Spun up all drives

* Got a directory listing

* It took less than one second

* Tried your command directly (not via the go file)

* Spun the disks down (set the clock forward)

* Got a directory listing in less than one second

* Drives still spun down

 

Nice.

 

BTW, it took over a minute to run that command - not sure exactly since I went out and did a bit of yardwork.

 

 

Bill

You might try adding the following command to the end of your "go" script

sleep 30; find /mnt -print >/dev/null 2>/dev/null

 

It will force all the directories to be scanned and be in the buffer cache.

They should be readable even after the disks spin down.

I don't know of any way to lock specific blocks representing the directories in the RAM cache.

 

Joe L.

 

I just tried it:

 

* Got a directory listing while the drives were spun down

* It took 3-4 seconds and forced a spin up

* Spun up all drives

* Got a directory listing

* It took less than one second

* Tried your command directly (not via the go file)

* Spun the disks down (set the clock forward)

* Got a directory listing in less than one second

* Drives still spun down

 

Nice.

 

BTW, it took over a minute to run that command - not sure exactly since I went out and did a bit of yardwork.

 

 

Bill

My command was actually two commands... separated by a semi-colon.

The first slept for 30 seconds.  (If at the end of the "go" script I wanted unRaid to have finished its startup tasks and set up the shares before I issued the second command.  You really did not need to use it if the two commands were issued on the command line.

 

The second "find" command traversed the directory hierarchy, starting at "/mnt", printing all the names of the files and folders.  We sent the output to /dev/null (this basically throws the output away) as we really were not interested in seeing the names of the folders and files.  The length of time it takes is dependent on how long it takes to spin up each disk in turn as it traverses the file systems.

 

A side effect of the "find" command is that all of the blocks representing the directory structure are read into the i/o buffer cache used by the disk system.  Now, if you start to play  a movie, the blocks representing the directory listing will probably be overwritten once space is needed, but at that point the disk is spinning, so a directory list would be pretty fast anyway.

 

Another side effect of the "find" command is that it will spin up all the drives as it reads their files and folders.  It is therefore probably only helpful when you first start up your array.  (If you ran it on a regular basis, your disks would always be spinning)

 

I would expect the  command to run in 40 - 50 seconds, depending on how many drives need to be spun up.  30 seconds for the initial sleep, and 3 - 5 seconds each to spin up each drive in turn.

 

Glad it worked. 

 

Joe L.

I probably don't give off the techie-vibe like I used to, but I am an old unix hack (Solaris) - I didn't bother with the sleep command.

 

 

Bill

 

P.S. A quick "sleep" story.  I joined Sun Microsystems back in '89 when we were still using command line unix for everything.  I had an early flight on the Wednesday before Thanksgiving but didn't want my boss to know I was leaving that early.  So "sleep 15000" (i.e. pause for a bit over four hours then execute the next command in that line) with a quick mail to my department as the second command on the line gave the impression that I was at work a lot later than I was.  Because I was in a largely non-technical group (Finance), nobody had a clue we could do stuff like that.  Between fun uses of sleep and melting people's screens remotely, those were the good old days.

Cute story...  My first UNIX login was in 1980... with a UID of 0.  The unknowing sysadmin cloned the one line in the password file for root, changed the name from root to his initials, made a second cloned line with my initials, set our passwords and we then had two "user accounts" with super-user privileges.  We did not know we had super-user  privileges for  many months.

It was CB-UNIX 1.0 (a variant of of PWB UNIX) out of Columbus Bell-Labs running  on a DEC 11/70.

 

The "shell" was the Mashey-Shell...it had labels and "goto".  it pre-dated the Bourne shell, which predated the Korn shell.

First Linux was an initial release of Slackware... somewhere around 1994. Linux was version 0.99.15

 

My favorite mail prank was something like this...

 

su root

LOGNAME='the boss'

TZ=EST29GMT

export LOGNAME TZ

echo "IBM representatives will be visiting our office tomorrow, please wear a blue shirt in their honor" | mail -s "Blue Shirt Day" ...

I did this when almost everybody in our programming group happened to wear a blue shirt to work one day.  All except the one poor newbie in the cubicle next to me that is.  He remarked that morning "Why does everybody have on blue today?"  I remarked, "Didn't you get the memo... today is "Blue Shirt Day"

 

When he later checked his e-mail, the memo was there... seemingly sent the day before. Well... 29 hours previously... 

 

Guess I'm showing my age...

 

Joe  L.

Once again, very clever Joe.  Probably the directories will stay there "forever" because linux uses a separate "dentry" cache and the vfs is configured to not use all available memory - can't remember offhand how much it's not allowed to touch, but probably enough that most of the file system hierarchy is retained.

I'd like to cast my vote for a dedicated directory cache, to keep my drives spun down.  I use (and highly recommend) SageTV which checks all media folders every 5 minutes for additions and changes.  This normally works great with unRAID, since the directories ARE cached now, until a large enough file is copied and wipes the cache, at which time all of the other drives have to spin up and stay spun up until the large file operations have finished.  They are all video drives, most are exclusively video storage, and 2 have a relatively small backup portion.  With full directory caching, most of my drives would almost never spin up, some would stay down for weeks at a time.

 

I'm wondering if a feature could be added that would provide a configurable option to specify the maximum depth in the directory tree to be cached, similar to the new split level security option.  A value of zero would imply no special directory caching, the current and default behavior.  It would be understood that those of us who wish to use more directory cache would need to provide more RAM.

 

Until there is time to implement more fully, any dedicated directory caching would be gratefully received.

 

I'm going to try Joe's script, thanks Joe, but I'd also like this as a feature.  Some media players (TVix of NFS) seem to "time out" before the drives have managed to spin up and be read.

 

Thanks Joe for the script mod, I haven't stopped smiling about the performance boost one, and this is also great!

 

Mark.

I'm going to try Joe's script, thanks Joe, but I'd also like this as a feature.  Some media players (TVix of NFS) seem to "time out" before the drives have managed to spin up and be read.

 

Thanks Joe for the script mod, I haven't stopped smiling about the performance boost one, and this is also great!

 

Mark.

My own Al-tech MG-35 media players time out before the drives get to spin up.  Worse yet, they remove the folder from further browsing and require a re-scan of the LAN in order to play any subsequent file.  (I have to go back to its main menu, then select the network, then have it scan for machine with network shares, then select the share, then the movie)

 

Unfortunately for me, the same type of time-out occurs even if the directory listing is in the cache.  The directory listing of the movies works, but then the time-out occurs when attempting to read the ISO image as the disk spins up. 

 

I've been trying to figure out a script that would spin up my movie drives when the media players are powered up.  I originally was trying to use the "smbstatus" command to know they are logged in, but even that is too late as my drives do not spin up quickly.

 

I'm going to try a loop with a "ping" of my network media players (fortunately, they are on fixed IP addresses) and then,

once they are detected on-line, a spin up would occur.

 

If I'm successful, I'll post yet another script.  Who knows, somebody out there might benefit and improve upon it.

 

Joe L.

I was successful in creating a script to automatically spin up my drives when my network based media player is powered up.

 

It can be found in this thread:

http://lime-technology.com/forum/index.php?topic=1035.0

 

Now, it is not an issue  to have the directory in cache, as I no longer have to wait for spin-up, and no longer time-out when initially accessing any unRaid server based media.

 

The script makes very creative use of the "nc" (netcat) command.  It is used to simulate a web-browser accessing the unnRaid management web-page.

 

Joe L.