cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up


Recommended Posts

@Alex R. Berg

 

ok, changed settings now once more, also increased some timings, may my disks are too slow ...

 

Stopping cache_dirs process 12039
cache_dirs version 2.2.2
Setting Memory ulimit to 50000
Setting cache_pressure=1
Arguments=-i Daten -i Media -i Temp -c 6 -X 300 -Y 120 -p 1 -u -l on -D 9999
Max Scan Secs=10, Min Scan Secs=1
Scan Type=fixed
Max Scan Depth=none
Use Command='find -noleaf'
---------- Caching Directories ---------------
Daten Media Temp
----------------------------------------------
Setting Included dirs: Daten,Media,Temp
Setting Excluded dirs:
min_disk_idle_before_restarting_scan_sec=60
scan_timeout_sec_idle=300
scan_timeout_sec_busy=120
scan_timeout_sec_stable=30
frequency_of_full_depth_scan_sec=604800
Including /mnt/user in scan
cache_dirs started
2018.10.31 08:22:35 Executed find in (0s) (disks 0s + user 0s) 00.31s, wavg=00.31s   depth 9999 slept 0s Disks idle before/after 311s/311s suc/fail cnt=0/0/0 mode=2 scan_tmo=300s maxCur=9999 maxWeek=9999 isMaxDepthComputed=1 CPU= 3%, filecount[9999]=18750
...

 

is this a working feedback, i cant interprete the log correctly

Disks idle before/after 311s/311s           <-- idle ok or should be active ?

suc/fail cnt=0/0/0                    <-- 0 success ? 0 failed ? 0 count ?

Link to comment



Regarding status stopped after reboot: Send me a unraid diagnostics, and also do a cache_dirs -L, and send me the logs it collects.

You can add this to the plg install
chmod +0755 /usr/local/emhttp/plugins/dynamix.cache.dirs/scripts/*



Or create an "install" directory within the dynamix.cache.dirs.txz package and place a doinst.sh file with that command in the install directory. That's the way all slackware packages handle permissions and other install functions.

E.g. This is from NerdPack doinst.sh

#!/bin/shln -sf /usr/local/emhttp/plugins/NerdPack/scripts/packagemanager /usr/sbin/packagemanagerchmod +0755 /usr/sbin/packagemanager

  • Upvote 1
Link to comment

thank you, i attached the diagnostic from latest release, update went fine, no more chmod needed ;)

2 minutes ago, Alex R. Berg said:

@alturismo I answered the idle just one above, maybe I posted as you were writing.

 

suc/fail: just ignore it. Its for adaptive. Its some counters saying how many of the last scans than completed completely or took to long so the script killed the scans (timeout params).

 

Your logs look good.

cache_dirs_diagnostics.zip

Link to comment
10 minutes ago, Alex R. Berg said:

Great.

 

I just pushed yet another plugin version 2018.11.01-1, which adds chmod 775 to more stuff in the scripts folder pr @dmacias suggestion, there are some other stuff there like logrotate and bin. 

update worked fine, and looking good in 1st views ;) could browse some shares without spinning ...

 

thanks for all the effort

  • Upvote 1
Link to comment
5 hours ago, alturismo said:

i only have 3 shares using the physical disks ... all with the setting to prefer cache (using scheduled mover).

“Prefer” cache would mean that any data on the physical disks would be moved to cache every time the mover runs. Perhaps this is exactly what you want for whatever your use case is. Just thought I’d mention this since some people have been confused by the terminology.

Link to comment
8 minutes ago, wgstarks said:

“Prefer” cache would mean that any data on the physical disks would be moved to cache every time the mover runs. Perhaps this is exactly what you want for whatever your use case is. Just thought I’d mention this since some people have been confused by the terminology.

urgs, no, of course i have set USE cache, so files stay in cache until mover runs and moves to disks in array, all good, my fault, USE cache, not PREFER ;) 

Link to comment

I'm having a problem with the plugin, the disks doesn't spin down. Have tried all latest versions, running 2.2.2 (2018.11.01-1) now. And getting this in the log file. Looks like its failing and restarting the process.

 

Have tried multiple settings, and are running a new install with just "enable" and one included directory right now.

 

Any ideas?

 

 

 

2018.10.31 19:09:59 Executed find in (0s) 00.16s, wavg=00.21s Idle____________  depth 9999 slept 10s Disks idle before/after 11s/11s suc/fail cnt=10/11/0 mode=3 scan_tmo=30s maxCur=9999 maxWeek=9999 isMaxDepthComputed=1 CPU= 9%, filecount[9999]=214446

 

2018.10.31 19:10:09 Executed find in (30s) 30.02s, wavg=00.21s NonIdleTooSlow__  depth 9999(timeout 30s:Error=1) slept 10s Disks idle before/after 9s/0s suc/fail cnt=11/0/1 mode=3 scan_tmo=30s maxCur=9999 maxWeek=9999 isMaxDepthComputed=1 CPU=10%, filecount[9999]=214446

 

2018.10.31 19:10:39 Executed find in (0s) 00.10s, wavg=00.20s Idle____________  depth 4 slept 1s Disks idle before/after 0s/0s suc/fail cnt=12/1/0 mode=3 scan_tmo=30s maxCur=5 maxWeek=9999 isMaxDepthComputed=1 CPU=66%, filecount[4]=117348

Link to comment

feedback here, yesterday during the day it seemed to be alot better then before with disk spinups, i could browse shares and almost never had any spinups.

 

now today morning i turned on my laptop, wich has the shares mounted.

 

just by turning the laptop on the disks started to run.

 

this i assume is a current bug in unraid cause its following me since a little while, i hoped i can catch this also with caching.

 

either way, its a huge improvement and thanks for this plugin ;)

Edited by alturismo
Link to comment

Getting this email:

 

subject: cron for user root /usr/bin/run-parts /etc/cron.daily 1> /dev/null

body:
error: stat of /var/log/cache_dirs.log failed: No such file or directory

error: stat of /var/log/cache_dirs_lost_cache.csv failed: No such file or directory

Running latest Berg version: cache_dirs version: 2.2.2 aka Bergware 2018.11.01-1

 

Link to comment
1 hour ago, alturismo said:

i dont even open the file Explorer ... ;-)

 

 

Then what do you mean by this?

3 hours ago, alturismo said:

and they spinned up when my laptop booted up (where the shares are mounted)

How is your laptop accessing the shares? Were the drives not spinning before you booted your laptop?

 

Link to comment
2 hours ago, trurl said:

 

 

Then what do you mean by this?

How is your laptop accessing the shares? Were the drives not spinning before you booted your laptop?

 

all drives sleeping, then laptop booting up, i log in.

 

then i "hear" my disks are spinning up ... got alot better with cache directories, but not completely solved as i would like to ;)

 

and my shares are all mounted like this, regular network shares in windows ...

 

image.png.f660ca012f0f9e4bf7293bcf35439057.png

 

and im NOT accessing them, i dont even open the file explorer at all, just logging into my win 10 laptop, but as i said, alot better now with cache_dirs ;)

 

and i cant remember its been always like this, im pretty sure it wasnt like this before, cant remember exactly when this behavior changed ...

but for me it definately changed, thats why i came to this plugin cause its disturbing me, and i use unraid for ~ 2 years now.

Link to comment
3 hours ago, alturismo said:

i dont even open the file Explorer ... ;-)

Windows will get some information about the folder when it mounts it which may or may not be cached.  Additionally, if the mapped drive is indexed, then a mount will trigger Windows to reindex the share which will also most likely cause all the drives to spin up 

  • Upvote 1
Link to comment
1 minute ago, Squid said:

Windows will get some information about the folder when it mounts it which may or may not be cached.  Additionally, if the mapped drive is indexed, then a mount will trigger Windows to reindex the share which will also most likely cause all the drives to spin up 

may its also since the win 10 1809 update ... i just can say its more then before, but why i still guess unraid, open a media file from cache also likes to spinup all drives here meanwhile ... wich also never was like this. client can be kodi on windows, kodi on shield tv, plex on sammy tv, plex from shield tv, plex from web ...

looks to me as unraid dunno directly where it really is and searches all disks ... i know this sounds strange ... ;)

 

but im happy it got better with cache_dirs, alot better ... specially the media playback issue

Link to comment
22 hours ago, interwebtech said:

Getting this email:

 

subject: cron for user root /usr/bin/run-parts /etc/cron.daily 1> /dev/null

body:
error: stat of /var/log/cache_dirs.log failed: No such file or directory

error: stat of /var/log/cache_dirs_lost_cache.csv failed: No such file or directory

Running latest Berg version: cache_dirs version: 2.2.2 aka Bergware 2018.11.01-1

 

Got this Email too, 2 days in a row since I installed Version 2018.11.01-1

Link to comment
23 hours ago, interwebtech said:

Getting this email:

 

subject: cron for user root /usr/bin/run-parts /etc/cron.daily 1> /dev/null

body:
error: stat of /var/log/cache_dirs.log failed: No such file or directory

error: stat of /var/log/cache_dirs_lost_cache.csv failed: No such file or directory

Running latest Berg version: cache_dirs version: 2.2.2 aka Bergware 2018.11.01-1

 

 

I suspect that adding 'missingok' option to /etc/logrotate.d/cache_dirs will prevent the error.  Also the permissions of this file could be changed to 644.

Edited by themaxxz
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.