cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up


1137 posts in this topic Last Reply

Recommended Posts

@Maticks

I might be wrong, but seeing at your screenshot tells me you don't have the most recent version of cache_dirs.

I can't see the version, it's on the right side, cut in your screenshot. There should be an option for "scan user shares".

Please read this thread backwards and install the most recent version.

And then, if your issue is persistent, eventually post your diagnostics along with your problem report.

Link to post
  • Replies 1.1k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

I've publish new version (sorry to bonienl who just merged, but I respond to no blame for bugs in my code-base )   I noticed from @wgstarts logs that it didn't sleep as I expected. When usin

It looks good interwebotech, so your problem is not the line i noticed had changed.    After I sent the message I noticed I completely had missed pulling all the changes from the release-rep

Ups, forgot attachment as Frank mentioned. Sorry. I havn't read other comments yet.   Download this file: https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg  

Posted Images

Might anyone point me to the correct there were i find a tutorial how to install the new version oft his plugin? mine showed that it was up to date, but hat the same issues like u all... (disks not spin down, app shows its not running)

 

While installing the version from page 37 i get:

 

plugin: installing: https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg
plugin: downloading https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg
plugin: downloading: https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg ... done
plugin: file doesn't exist or xml parse error
 

Since this thread is a complete mess it would be cool if the new developer could open a new thread for his fork (?) and give proper installation infos... :) Thanks

Edited by nuhll
Link to post
1 hour ago, nuhll said:

Might anyone point me to the correct there were i find a tutorial how to install the new version oft his plugin? mine showed that it was up to date, but hat the same issues like u all... (disks not spin down, app shows its not running)

 

While installing the version from page 37 i get:

 

plugin: installing: https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg
plugin: downloading https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg
plugin: downloading: https://github.com/arberg/dynamix/blob/master/unRAIDv6/dynamix.cache.dirs.plg ... done
plugin: file doesn't exist or xml parse error
 

Since this thread is a complete mess it would be cool if the new developer could open a new thread for his fork (?) and give proper installation infos... :) Thanks

Try using this link as @BRiT suggested. From your plugin tab, and "install plugin".

 

https://raw.githubusercontent.com/arberg/dynamix/master/unRAIDv6/dynamix.cache.dirs.plg

 

 

Link to post
1 hour ago, nuhll said:

Yeah, that is working, thanks.

 

Do i need to change anything? (I just added the directorys i wanna cache)

No problem!

From what i have read, you might want to include user shares. And set cache pressure to 1. Depending on if you are having problems or not.

 

Link to post
2 minutes ago, jowe said:

From what i have read, you might want to include user shares. And set cache pressure to 1. Depending on if you are having problems or not.

Right, try the default settings and see if it runs OK.

Scanning user shares did the trick for me. I didn't touch cache pressure.

Caching happens very fast.

You can spin down the drives manually and browse your shares to see if it works.

Link to post
6 hours ago, Fireball3 said:

@Maticks

I might be wrong, but seeing at your screenshot tells me you don't have the most recent version of cache_dirs.

I can't see the version, it's on the right side, cut in your screenshot. There should be an option for "scan user shares".

Please read this thread backwards and install the most recent version.

And then, if your issue is persistent, eventually post your diagnostics along with your problem report.

I am running cache_dirs version: 2.2.0j which is saying its the latest version.

 

Link to post

i've added -u to set user shares.

 

Nov 8 00:45:44 Vault cache_dirs: Starting cache_dirs:
Nov 8 00:45:44 Vault cache_dirs: Arguments=-i Cloud -i Movies -i Music -i TV -i isos -i system -p 0 -U 0 -l on -u -d 20
Nov 8 00:45:44 Vault cache_dirs: Cache Pressure=0
Nov 8 00:45:44 Vault cache_dirs: Max Scan Secs=10, Min Scan Secs=1 (min scan seconds ignored, always sleep max)
Nov 8 00:45:44 Vault cache_dirs: Scan Type=adaptive
Nov 8 00:45:44 Vault cache_dirs: Max Scan Depth=none
Nov 8 00:45:44 Vault cache_dirs: Use Command='find -noleaf'
Nov 8 00:45:44 Vault cache_dirs: Version=2.2.0j
Nov 8 00:45:44 Vault cache_dirs: ---------- Caching Directories ---------------
Nov 8 00:45:44 Vault cache_dirs: Cloud
Nov 8 00:45:44 Vault cache_dirs: Movies
Nov 8 00:45:44 Vault cache_dirs: Music
Nov 8 00:45:44 Vault cache_dirs: TV
Nov 8 00:45:44 Vault cache_dirs: isos
Nov 8 00:45:44 Vault cache_dirs: system
Nov 8 00:45:44 Vault cache_dirs: ----------------------------------------------
Nov 8 00:45:45 Vault cache_dirs: cache_dirs process ID 19516 started

Link to post
5 minutes ago, Maticks said:

I am running cache_dirs version: 2.2.0j which is saying its the latest version.

 

There is a fork in this plugin and that fork is the one being supported in this thread.  See here for the way to get on this fork!

 

Link to post
30 minutes ago, nuhll said:

Drives still wont go down. File activity doesnt show anything and im not browsing any shares.

Your spin-down delay is properly set?

Did you reboot after installing the plugin?

Edited by Fireball3
Link to post
58 minutes ago, nuhll said:

I dont know how long i use unraid, i never needed to restart after changing a plugin.

So is your mysterios uestion  a yes?!

Indeed, in my experience a reboot has solved many mysterious things.

 

51 minutes ago, Maticks said:

The plugin use to work really well maybe something in 6.x.x has upset it.

You have a point there!

51 minutes ago, Maticks said:

There is less activity when the plugin is off.

Less means, drives are spun down or not?

 

How is it with adaptive depth turned off?

How is it with cache pressure set to 1?

Edited by Fireball3
Link to post

With the plugin disabled the drives spin down as per below, but without the cachedir plugin a Plex scan will spin up all the disks that are part of that share.

The plugin did stop that from happening by holding the directories in memory during the scan.

Though disks are spun down without the cachedir its not an ideal situation. :)

 

In all fairness when i did my plugin upgrade i never rebooted, so i will go do that now.

But my settings for cachedir are below as well, do they look right.1203137414_plugindisabled..JPG.e88eee9c6dd9b866aa1601657b6373a7.JPGcache.thumb.JPG.130392a147025b0dd18e596187c04189.JPG

 

 

Link to post

the reboot was only once required after changing the "broken" script.

 

since then i didnt had to reboot on changes etc ... i think you can follow this in logs, i started this plugin manually after update to 664 this morning,

when u see all lines (not only the 1 line) you should be good, also here the disks spin down automatically after the timeout (here 15 mins)

 

they spinned up by activating the plugin and then ... as it should be, perfect ;)

 

image.png.eb69f8d23b84e0bf9fe410f0d2414cf4.png

Link to post

I’m seeing cache_dirs errors in my system log “excluded directory does not exist”. These directories do exist, they were selected from the drop down menu in settings. They are also all mounted via UD, so I’m wondering if that’s the reason for the error?

Link to post
15 hours ago, interwebtech said:

FYI

v2.2.4 started itself properly when the server was rebooted (os update). I still have the message on the console (last line before uR version & IP display):

 

cat: write error: Broken pipe

 

Same. Works for me but the cat error is still present. Could be considered cosmetic, all things considered, but something is clearly wrong still.

 

Thanks for the updated plugin btw, much appreciated.

Link to post

Updated to unRAID 6.6.4 and problem with spinning up disks is no longer present!

 

But it seems to use alot of CPU, around 10% with plugin disabled. And 30%+ when enabled.

Changed to adaptive, and it's sleeping for 10s insted of only 1 with fixed. And the cpu is much less spiky.

 

Thanks for your support @Alex R. Berg

Edited by jowe
CPU
Link to post

For me update plugin and restarting only worked half way. But since i upgraded to 6.6.5 its working perfeclty and disks spinning down again... but didnt saw any changes in log which indicates anything... mysterios.

 

If i enable loggin, where i can see activity?

Edited by nuhll
Link to post
On 11/7/2018 at 3:54 PM, nuhll said:

Do i need to reboot?

 

On 11/7/2018 at 3:58 PM, Fireball3 said:

Is it a computer?

ROFL :) Man just a week ago my wife was deeply frustrated because there was no sound on the HTPC, and it was important to be fixed right then and I was busy for 15 min in an online game. Too bad for her. A reboot fixed it for her when I came to her later, and then she frustrated said 'Oh I always forget that rebooting is an option'. There's Value Learning Experience. I think she just rose a level in the hacker-class :)

 

I just updated to 6.6.3 from 6.4.0 a week ago. I did a custom schedule of the mover, and found later that i didn't set it for every second month but every single day every second month. I doubt that's the issue others are seing, but that certainly caused my disks to spin, and cache-dirs will frequently lose its cache, when the system is under pressure I have found, though particularly when my md5-scan updates its data.

 

@Jowe how much memory do you have in your machine? I'm just curious, I probably cannot help with the problem.

 

@MaticksUpdate as @Fireball3 pointed out. I notice some disk-idle timestamp durations in your log, that should be fixed in new version, though I'm not sure it'll matter.

 

I'm not certain cache-pressure 0 will make machine crash if the memory is exhausted, I don't think I have ever experienced that, though it seems likely. If using cache-pressure 0, and if its still implemented in kernel to never release dirs, I would think it would be possible to spin down disks manually, and they should stay down unless other things are reading from the disk. 'sudo lsof | grep /mnt/' might help to figure out what is accessing disks, though it just gives a snapshot. So if cache_dir logs says scan duration 1s,1s,1s,1s,1s,64s,1s and the disks do not spin up after manually spinning them down, obviously the 64s was not due to lost cache, but must have been because CPU was under heavy load. However I find such debugging to be tiresome because its so difficult to narrow down what is accessing the disks. But still there might be valuable info from that test. My guess though is something is actually reading or writing the disks.

 

@wgstarks

I'm not sure I'll investigate much further in it, but maybe I'll throw a quick glance at your logs. What is UD mount? cache-dirs reports which dirs it scans. Is that list correct?

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.