cache_dirs - an attempt to keep directory entries in RAM to prevent disk spin-up


Recommended Posts

Thanks trurl, I completely missed that. It's a bit late where I'm at.

I've been using the script for quite a while but always assumed the cached directories are saved in some file.

I was confused as to why the script caused a lot of disk activity for some time after a reboot, it makes sense now.

  • Like 1
Link to comment
  • 4 months later...
  • 1 month later...
  • 4 weeks later...
  • 5 months later...
  • 4 weeks later...

Hello,

New to Unraid so please forgive me if I sound like a total newb.

 

I have a torrent client running on my Unraid server all the time seeding torrents (100% perfectly legitimate Linux distro ISOs, obviously). I have all my torrent stuff stored on one of the 3 disks (2 array, 1 parity)...so I don't expect that disk to spin down ever - and for some reason I have yet to ever see the other disks spin down of their own accord (though I could have something set wrong - I need to investigate).

 

ANYWAY...my issue would be: if my disks never spin down, does that mean cache_dirs is never going to run? I tried setting "

Disk Idle Timer (sec)" to 0.  As far as I can tell, it is not caching. I am not seeing an increase in use of memory.  Is there a way to actually tell what state the caching plugin is in? Whether it's waiting for something or whether it's actually doing something?

 

Thanks in advance.

Link to comment
  • 4 months later...

I'm new to Unraid, but this plugin has immediately become one I can never live without. Thank you!

 

My disks stay parked while browsing, searching, and even scanning with emby/*arr or freefilesync.

 

I tried countless solutions to achieve this in Windows over the years, with minimal success.

 

That said, the cpu spikes are wild. It's a low cpu utilization % per thread (<10%), but since it runs 1 thread per disk, it means my cpu basically never parks cores, and is always in some state of turbo-boost. My computer idles ~5 degrees (C) hotter with the plugin on, vs off (2x ancient E5-2600 v3).

 

I've tried turning off 'run scan of each disk in a seperate thread' but this results in my disks spinning up. 

 

I've also tried increasing the minimum interval between folder scans, but this also results in more spin-ups.

 

Does the 'hack' that this plugin exploits only keep folders in memory if there's a constant folder-scan happening?

 

Like I said, it's not that the cpu utilization % is so high, but it's just high and persistent enough to defeat all the power-saving features of my computer.

Edited by nmkaufman
Link to comment
  • 2 months later...

This is filling up my /var/log dir. How do I stop it from creating a runaway .csv?

 

root@mediasrv:/var/log# ls -l|grep cache_dirs
-rw-r--r-- 1 root   root 1.4M Feb 11 18:43 cache_dirs.log
-rw-r--r-- 1 root   root  42M Feb 11 18:43 cache_dirs.csv

root@mediasrv:/var/log# df -h .
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           128M   76M   53M  60% /var/log


 

Link to comment
36 minutes ago, jaylo123 said:

This is filling up my /var/log dir. How do I stop it from creating a runaway .csv?

Enabling Help on the Cache Dirs plugin provides the answer.  Disable logging if you do not want the .csv file to keep endlessly growing. As noted, it is not automatically rolled.  Or you can delete it and let it start over I suppose

 

image.thumb.png.4755d0265405ae192d5f01b5481ddbbd.png

  • Like 1
Link to comment
On 2/11/2024 at 7:25 PM, Hoopster said:

Enabling Help on the Cache Dirs plugin provides the answer.  Disable logging if you do not want the .csv file to keep endlessly growing. As noted, it is not automatically rolled.  Or you can delete it and let it start over I suppose

 

image.thumb.png.4755d0265405ae192d5f01b5481ddbbd.png

Didn't even think to check there. And I clearly missed that the log file isn't automatically rolled. I'll add to logrotate/tmpwatch/whatever Unraid uses (I float between different distros at work and can't remember what Unraid uses lol).

 

Thanks again!

Link to comment

Hi all,

 

Is this still relevant / helpful nowadays? I haven't tested much but I've got my disks to spindown after 2 hours of IO inactivity. I've got all turned off at the moment. If I navigate to plex and select a movie it is only spinning up the disk that has it. Would this script still be of value?

Link to comment
46 minutes ago, Sptz87 said:

Would this script still be of value?

if your disks stay down while not in use you should be fine.

 

this is more relevant when browsing your shares to keep the disks in spindown,

when you open a file it always will spinup as its physically readed ... ;)

Link to comment
On 12/4/2023 at 8:28 PM, nmkaufman said:

I'm new to Unraid, but this plugin has immediately become one I can never live without. Thank you!

 

My disks stay parked while browsing, searching, and even scanning with emby/*arr or freefilesync.

 

I tried countless solutions to achieve this in Windows over the years, with minimal success.

 

That said, the cpu spikes are wild. It's a low cpu utilization % per thread (<10%), but since it runs 1 thread per disk, it means my cpu basically never parks cores, and is always in some state of turbo-boost. My computer idles ~5 degrees (C) hotter with the plugin on, vs off (2x ancient E5-2600 v3).

 

I've tried turning off 'run scan of each disk in a seperate thread' but this results in my disks spinning up. 

 

I've also tried increasing the minimum interval between folder scans, but this also results in more spin-ups.

 

Does the 'hack' that this plugin exploits only keep folders in memory if there's a constant folder-scan happening?

 

Like I said, it's not that the cpu utilization % is so high, but it's just high and persistent enough to defeat all the power-saving features of my computer.

 

Hmm, are the spikes really that much higher that undermines hours of spun down disks? 

Link to comment
5 hours ago, alturismo said:

if your disks stay down while not in use you should be fine.

 

this is more relevant when browsing your shares to keep the disks in spindown,

when you open a file it always will spinup as its physically readed ... ;)

 

Gotcha! Thank you! Is this script basically included in dynamix's cache dirs?

Link to comment
  • 2 weeks later...

I may be misunderstanding what this does, would it fix an issue when I browse the movie folder share on a windows machine it takes 1-2 seconds to load all the movie folders......it's not the end of the world, I just wanted it to be more instant when I open the movie folder to browse. When I open it I can see some of the folders, like G-P movie names and then the rest load up.

 

I can see it counting, and the progress bar at the top is adding them

 

Once it loads all the movie folders it's fine, but right at first I can only see a portion of them when makes me feel like some drives aren't spinning up quick enough? I'm not sure.

 

I have 24 drives in a supermicro hot swap case

 

Thanks in advance

 

Edited by 1971camaroguy
Link to comment
  • 2 weeks later...

I am a little bit confused right now. Now and then I read here and there that die CPU usage goes up while scanning.

 

How did I even know if it is scanning or not? When I activate it (default settings with a few folders included) the CPU is going up every few seconds but not that much that I'd think that there is something cached right now.  It doesn't matter if the disks are all spun down or not it behaves the same, even when the disks are spun down for 12 hours. Deactivating the plugin is stopping this behavior.

 

Normally I've an Idle power consumption from about 43W. When the folder caching plugin is activated, I've about 78W with going up to 125-150W every 4-5 seconds (even with spun down disks and plugin activation half a day ago). Only way to stop this is by stopping the plugin.

 

I'd expect some readings on the disks or spinning up disks ore something like that. But with more than double the power consumption in idle it could spin up a disk from time to time and will produce wear, but less costs than with the plugin. I have a dynamic electricity rate (today 0,31€-0,40€/kWh). So it does matter a little bit.

 

Before I tried to used the plugin I could browse through folders with Dynamix File Manager terminal window (Unraid), SSH-Connection and Krusader and the disks remains spun down. Only browsing via MacOS (SMB) spun up the drives (I don't know why because DS_Store or other hiding files were not created).

 

Something is wrong here. How could I see if the plugin is running or how can I force It to run properly to get the CPU load down afterwards?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.