Jump to content
Joe L.

cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up

1086 posts in this topic Last Reply

Recommended Posts

Hi,

 

I love using cache dirs, but I'm trying to track down an issue with my music steamer running on my unRaid server (Subsonic). When I add a new folder, it doesn't show up unless I reboot. I shouldn't need to do this. Its not the browser cache and since I rebuilt the search index for Subsonic and deleted the folderCache files it uses, I'm looking at other possibilities.

 

Stopping and starting the cache_dirs script doesn't help. Is there a way to force cache_dirs to rescan? Are files left over that could be deleted?

 

Many Thanks!

 

I've not experienced what your experiencing with Subsonic. Sometimes I have an issue and I have to restart whatever browser I was using, but I've never had to reboot.

Share this post


Link to post

I'll preface this by saying I haven't read the entire thread..

 

But for Joe..

 

I'm noticing that executing a cache_dirs -q will sometimes result in the process not fully getting killed or taking a VERY long time to kill all the spawned processes.

 

I'm using an rc.cache_dirs with your unraid_addon_control.sh which works fine for everything I use it for (notify, sab, sickbeard, lighttpd, etc.) but with the cache_dirs, it executes the command to -q but I would say 50% of the time, it won't fully quit, requiring me to ssh/telnet into the unraid box and force kill the "orphaned" processes.

 

Any idea why this is happening?

 

Share this post


Link to post

Here is my directory structure that I am caching:

 

Media

|_____Movies

|_____Shows

|_____Pictures

|_____Videos

 

Command I am invoking:

/boot/cache_dirs  -d  3  -m  3  -M  5  -p 10 -w -i "Media"

 

It is working when I browse starting from the Media share.

 

However, I also have shares, via smb-extra.conf, that points directly to the sub-folders.  It appears that when I start browsing from the sub-folder shares, the directory cache is by-passed.

 

Do I also have to add the  -i "Movies", -i "Shows", -i "Pictures", and - i "Videos" if I plan to map to the sub-folder shares?

Share this post


Link to post

Here is my directory structure that I am caching:

 

Media

|_____Movies

|_____Shows

|_____Pictures

|_____Videos

 

Command I am invoking:

/boot/cache_dirs  -d  3  -m  3  -M  5  -p 10 -w -i "Media"

 

It is working when I browse starting from the Media share.

 

However, I also have shares, via smb-extra.conf, that points directly to the sub-folders.  It appears that when I start browsing from the sub-folder shares, the directory cache is by-passed.

 

Do I also have to add the  -i "Movies", -i "Shows", -i "Pictures", and - i "Videos" if I plan to map to the sub-folder shares?

I would suggest not using any option except the -w

 

Invoke it as

cache_dirs -w  I think he issue is you limiting the depth cached to 3.  The other Min and Max times adjust themselves automatically.  Unless you have massive numbers of files in other top level shares, using the "-i" is also not needed.  I personally use -e "data" because I have an old backup of older files under my "data" share that has over a hundred thousand files.

 

Share this post


Link to post

I would suggest not using any option except the -w

 

Invoke it as

cache_dirs -w  I think he issue is you limiting the depth cached to 3.  The other Min and Max times adjust themselves automatically.  Unless you have massive numbers of files in other top level shares, using the "-i" is also not needed.  I personally use -e "data" because I have an old backup of older files under my "data" share that has over a hundred thousand files.

 

 

Thanks Joe.  I will give your suggestion a try.

Share this post


Link to post

Set this up last night, and it really seems to make a difference with my XBMC live.  Now I have a question.

 

When I need to powerdown, do I need to do it like this?

 

cache_dirs -q
powerdown

 

If I setup clean powerdown in the packages of unMENU, then I would just be able to do the powerdown, is that correct?

 

I tried to download the clean powerdown last night but it errored on me saying in the log that it could not be found.  Not sure if I need to update my unMENU or not to get it to work, I'm using unMENU 1.3.

 

Thanks for this, combined with using the dds fanart, my XBMC is running smooth and snappy!

Share this post


Link to post

The powerdown package in unmenu will handle it for you. I also often power down from the default unRAID gui and haven't had any issues.

 

Sent from my DROID2 using Tapatalk

Share this post


Link to post

cach_dirs is a must in my opinion for XBMC. IT scans so freaking fast when your building your library its insane.

Share this post


Link to post

Can I get some clarification on the -B option?

 

It says use it to prevent an unmounted drive to show up as unformatted (when it's actually formatted, I'm assuming). Is this more of a cosmetic issue?

 

Can I do /boot/cache_dirs -w -b?

Share this post


Link to post

Can I get some clarification on the -B option?

 

It says use it to prevent an unmounted drive to show up as unformatted (when it's actually formatted, I'm assuming). Is this more of a cosmetic issue?

 

Can I do /boot/cache_dirs -w -b?

older releases of inRAID would show an un-mounted drive as un-formatted.  The option is not as important as in the past.

 

 

Share this post


Link to post

Since I started using the cache_dir, I am seeing faster write times with DVDFab to the unRAID.  They are averaging about 1/3 faster than I have been normally getting. I am not complaining at all, but are improvements like this normal?

Share this post


Link to post

I wanted to post a quick thank you for developing this script. This has greatly improved access times to the large number of picture files I have stored on my array!

Share this post


Link to post

I also wanted to say thanks. My shares load much quicker than before, even when the disks were spinning.

 

Is this an appropriate script to stop the cache_dir:

 

#define USER_SCRIPT_LABEL Stop cache_dirs
#define USER_SCRIPT_DESCR Stop the cache_dir script
#
# stop cache_dir

cache_dirs -q

 

I copied the syntax of the stop python menu, simply changing the relevant commands. It's named 10-unmenu_user_script_stop_cachedirs and is in the unmenu dir.

Share this post


Link to post

I also wanted to say thanks. My shares load much quicker than before, even when the disks were spinning.

 

Is this an appropriate script to stop the cache_dir:

 

#define USER_SCRIPT_LABEL Stop cache_dirs
#define USER_SCRIPT_DESCR Stop the cache_dir script
#
# stop cache_dir

cache_dirs -q

 

I copied the syntax of the stop python menu, simply changing the relevant commands. It's named 10-unmenu_user_script_stop_cachedirs and is in the unmenu dir.

It might work, if cache_dirs was in a directory already in your search PATH.

 

It would be better if you put the full path to cache_dirs, like this as an example:

#define USER_SCRIPT_LABEL Stop cache_dirs

#define USER_SCRIPT_DESCR Stop the cache_dir script

#

# stop cache_dir

/boot/cache_dirs -q

 

Before you go crazy, cache_dirs already monitors the syslog and can detect when the array is being stopped.  When this occurs, it suspends itself.  It does not prevent the array from stopping.  If you do not reboot, but start the array again, it re-starts itself..

 

Joe L.

Share this post


Link to post

I put the cache_dirs package in /boot/packages/

 

Is this location ok?

 

Mine is in the root of the flash drive. When you say package, I only had a single file. Did you have more?

Share this post


Link to post

It might work, if cache_dirs was in a directory already in your search PATH.

 

It would be better if you put the full path to cache_dirs, like this as an example:

#define USER_SCRIPT_LABEL Stop cache_dirs

#define USER_SCRIPT_DESCR Stop the cache_dir script

#

# stop cache_dir

/boot/cache_dirs -q

 

Before you go crazy, cache_dirs already monitors the syslog and can detect when the array is being stopped.  When this occurs, it suspends itself.   It does not prevent the array from stopping.  If you do not reboot, but start the array again, it re-starts itself..

 

Joe L.

 

Thanks Joe.

Share this post


Link to post

I put the cache_dirs package in /boot/packages/

 

Is this location ok?

 

Mine is in the root of the flash drive. When you say package, I only had a single file. Did you have more?

 

Yes - just the one.

Share this post


Link to post

I put the cache_dirs package in /boot/packages/

 

Is this location ok?

 

Mine is in the root of the flash drive. When you say package, I only had a single file. Did you have more?

 

Yes - just the one.

 

I asked if you had more than one file, and you replied yes, just the one. :P

 

If it's one file, cache_dirs, copy it into root of your USB drive.

Share this post


Link to post

I personally put all custom files in /boot/scripts just to keep all addons seperate from anything stock like that came from unRAID.

Share this post


Link to post

I asked if you had more than one file, and you replied yes, just the one. :P

 

Whoops... correction... 'No - just the one!'

Share this post


Link to post

Quick question... I just added this to my unRAID install. It is working perfectly except for one detail. I have some files that are more than 5 folders deep. Which numbers in the go script do i need to edit? Both of the 5's or just one of them? My current structure is below, I just need to up it to 6 i believe.

 

Media>TV>Completed>Show>Season>Episodes

 

The 'episodes' is the actual video files within the Season X folder, not another folder. I thought that would technically be 5 levels deep but the drives were spinning up when i got this deep. I added the line listed in the wiki to my go script so it looks like....

 

/boot/cache_dirs  -d  5  -m  3  -M  5  -w

 

Thanks guys!

 

Share this post


Link to post

Hi Joe,

 

First, kudos on the script. You have more patience with shell scripting than me.

 

One idea for supporting more powerful exclusions is to use "rsync --list-only" instead of find to do the cache loading. That way you could just pass the include/exclude commands directly to rsync. They can be a little... finicky... but a few examples would probably address most of that.

Share this post


Link to post

Hi Joe,

 

First, kudos on the script. You have more patience with shell scripting than me.

 

One idea for supporting more powerful exclusions is to use "rsync --list-only" instead of find to do the cache loading. That way you could just pass the include/exclude commands directly to rsync. They can be a little... finicky... but a few examples would probably address most of that.

You can use rsync now with the "-c command" option:

-c command  = use command instead of "find"

  ("command" should be quoted if it has embedded spaces)

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.