cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up


Recommended Posts

Hi,

 

I love using cache dirs, but I'm trying to track down an issue with my music steamer running on my unRaid server (Subsonic). When I add a new folder, it doesn't show up unless I reboot. I shouldn't need to do this. Its not the browser cache and since I rebuilt the search index for Subsonic and deleted the folderCache files it uses, I'm looking at other possibilities.

 

Stopping and starting the cache_dirs script doesn't help. Is there a way to force cache_dirs to rescan? Are files left over that could be deleted?

 

Many Thanks!

 

I've not experienced what your experiencing with Subsonic. Sometimes I have an issue and I have to restart whatever browser I was using, but I've never had to reboot.

Link to comment

I'll preface this by saying I haven't read the entire thread..

 

But for Joe..

 

I'm noticing that executing a cache_dirs -q will sometimes result in the process not fully getting killed or taking a VERY long time to kill all the spawned processes.

 

I'm using an rc.cache_dirs with your unraid_addon_control.sh which works fine for everything I use it for (notify, sab, sickbeard, lighttpd, etc.) but with the cache_dirs, it executes the command to -q but I would say 50% of the time, it won't fully quit, requiring me to ssh/telnet into the unraid box and force kill the "orphaned" processes.

 

Any idea why this is happening?

 

Link to comment

Here is my directory structure that I am caching:

 

Media

|_____Movies

|_____Shows

|_____Pictures

|_____Videos

 

Command I am invoking:

/boot/cache_dirs  -d  3  -m  3  -M  5  -p 10 -w -i "Media"

 

It is working when I browse starting from the Media share.

 

However, I also have shares, via smb-extra.conf, that points directly to the sub-folders.  It appears that when I start browsing from the sub-folder shares, the directory cache is by-passed.

 

Do I also have to add the  -i "Movies", -i "Shows", -i "Pictures", and - i "Videos" if I plan to map to the sub-folder shares?

Link to comment

Here is my directory structure that I am caching:

 

Media

|_____Movies

|_____Shows

|_____Pictures

|_____Videos

 

Command I am invoking:

/boot/cache_dirs  -d  3  -m  3  -M  5  -p 10 -w -i "Media"

 

It is working when I browse starting from the Media share.

 

However, I also have shares, via smb-extra.conf, that points directly to the sub-folders.  It appears that when I start browsing from the sub-folder shares, the directory cache is by-passed.

 

Do I also have to add the  -i "Movies", -i "Shows", -i "Pictures", and - i "Videos" if I plan to map to the sub-folder shares?

I would suggest not using any option except the -w

 

Invoke it as

cache_dirs -w  I think he issue is you limiting the depth cached to 3.  The other Min and Max times adjust themselves automatically.  Unless you have massive numbers of files in other top level shares, using the "-i" is also not needed.  I personally use -e "data" because I have an old backup of older files under my "data" share that has over a hundred thousand files.

 

Link to comment

I would suggest not using any option except the -w

 

Invoke it as

cache_dirs -w  I think he issue is you limiting the depth cached to 3.  The other Min and Max times adjust themselves automatically.  Unless you have massive numbers of files in other top level shares, using the "-i" is also not needed.  I personally use -e "data" because I have an old backup of older files under my "data" share that has over a hundred thousand files.

 

 

Thanks Joe.  I will give your suggestion a try.

Link to comment
  • 4 weeks later...

Set this up last night, and it really seems to make a difference with my XBMC live.  Now I have a question.

 

When I need to powerdown, do I need to do it like this?

 

cache_dirs -q
powerdown

 

If I setup clean powerdown in the packages of unMENU, then I would just be able to do the powerdown, is that correct?

 

I tried to download the clean powerdown last night but it errored on me saying in the log that it could not be found.  Not sure if I need to update my unMENU or not to get it to work, I'm using unMENU 1.3.

 

Thanks for this, combined with using the dds fanart, my XBMC is running smooth and snappy!

Link to comment

Can I get some clarification on the -B option?

 

It says use it to prevent an unmounted drive to show up as unformatted (when it's actually formatted, I'm assuming). Is this more of a cosmetic issue?

 

Can I do /boot/cache_dirs -w -b?

older releases of inRAID would show an un-mounted drive as un-formatted.  The option is not as important as in the past.

 

 

Link to comment
  • 2 weeks later...
  • 2 weeks later...

I also wanted to say thanks. My shares load much quicker than before, even when the disks were spinning.

 

Is this an appropriate script to stop the cache_dir:

 

#define USER_SCRIPT_LABEL Stop cache_dirs
#define USER_SCRIPT_DESCR Stop the cache_dir script
#
# stop cache_dir

cache_dirs -q

 

I copied the syntax of the stop python menu, simply changing the relevant commands. It's named 10-unmenu_user_script_stop_cachedirs and is in the unmenu dir.

Link to comment

I also wanted to say thanks. My shares load much quicker than before, even when the disks were spinning.

 

Is this an appropriate script to stop the cache_dir:

 

#define USER_SCRIPT_LABEL Stop cache_dirs
#define USER_SCRIPT_DESCR Stop the cache_dir script
#
# stop cache_dir

cache_dirs -q

 

I copied the syntax of the stop python menu, simply changing the relevant commands. It's named 10-unmenu_user_script_stop_cachedirs and is in the unmenu dir.

It might work, if cache_dirs was in a directory already in your search PATH.

 

It would be better if you put the full path to cache_dirs, like this as an example:

#define USER_SCRIPT_LABEL Stop cache_dirs

#define USER_SCRIPT_DESCR Stop the cache_dir script

#

# stop cache_dir

/boot/cache_dirs -q

 

Before you go crazy, cache_dirs already monitors the syslog and can detect when the array is being stopped.  When this occurs, it suspends itself.  It does not prevent the array from stopping.  If you do not reboot, but start the array again, it re-starts itself..

 

Joe L.

Link to comment

It might work, if cache_dirs was in a directory already in your search PATH.

 

It would be better if you put the full path to cache_dirs, like this as an example:

#define USER_SCRIPT_LABEL Stop cache_dirs

#define USER_SCRIPT_DESCR Stop the cache_dir script

#

# stop cache_dir

/boot/cache_dirs -q

 

Before you go crazy, cache_dirs already monitors the syslog and can detect when the array is being stopped.  When this occurs, it suspends itself.   It does not prevent the array from stopping.  If you do not reboot, but start the array again, it re-starts itself..

 

Joe L.

 

Thanks Joe.

Link to comment

I put the cache_dirs package in /boot/packages/

 

Is this location ok?

 

Mine is in the root of the flash drive. When you say package, I only had a single file. Did you have more?

 

Yes - just the one.

 

I asked if you had more than one file, and you replied yes, just the one. :P

 

If it's one file, cache_dirs, copy it into root of your USB drive.

Link to comment
  • 3 weeks later...

Quick question... I just added this to my unRAID install. It is working perfectly except for one detail. I have some files that are more than 5 folders deep. Which numbers in the go script do i need to edit? Both of the 5's or just one of them? My current structure is below, I just need to up it to 6 i believe.

 

Media>TV>Completed>Show>Season>Episodes

 

The 'episodes' is the actual video files within the Season X folder, not another folder. I thought that would technically be 5 levels deep but the drives were spinning up when i got this deep. I added the line listed in the wiki to my go script so it looks like....

 

/boot/cache_dirs  -d  5  -m  3  -M  5  -w

 

Thanks guys!

 

Link to comment

Hi Joe,

 

First, kudos on the script. You have more patience with shell scripting than me.

 

One idea for supporting more powerful exclusions is to use "rsync --list-only" instead of find to do the cache loading. That way you could just pass the include/exclude commands directly to rsync. They can be a little... finicky... but a few examples would probably address most of that.

Link to comment

Hi Joe,

 

First, kudos on the script. You have more patience with shell scripting than me.

 

One idea for supporting more powerful exclusions is to use "rsync --list-only" instead of find to do the cache loading. That way you could just pass the include/exclude commands directly to rsync. They can be a little... finicky... but a few examples would probably address most of that.

You can use rsync now with the "-c command" option:

-c command  = use command instead of "find"

  ("command" should be quoted if it has embedded spaces)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.