May 23, 20251 yr This is what my log is showing:```$ alias tailcachelogalias tailcachelog='tail -n 2000 -f /var/log/cache_dirs.log'$ tailcachelogcache_dirs version 2.2.7Setting Memory ulimit to 100000Setting cache_pressure=1Arguments=-p 1 -S -i appdata -i backup -i md5 -i media -i secure -i storage -i sync -i windowsbackup -e app -e appbackup -e cacheonly -e docker -e home -e muaddibhdd -e nanite -e pil -e svn -e svnbackup -e system -e ubuntu -e windowsbackup -c 4 -W 61 -Z 149 -U 100000 -l on -a -noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -name .sync -prune -o -printMax Scan Secs=10, Min Scan Secs=1Scan Type=adaptiveMin Scan Depth=4Max Scan Depth=noneUse Command='find -noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -name .sync -prune -o -print'---------- Caching Directories ---------------appdata backup md5 media secure storage sync----------------------------------------------Setting Included dirs: appdata,backup,md5,media,secure,storage,sync,windowsbackupSetting Excluded dirs: app,appbackup,cacheonly,docker,home,muaddibhdd,nanite,pil,svn,svnbackup,system,ubuntu,windowsbackupmin_disk_idle_before_restarting_scan_sec=61scan_timeout_sec_idle=150scan_timeout_sec_busy=30scan_timeout_sec_stable=149frequency_of_full_depth_scan_sec=604800ERROR: excluded directory 'cacheonly' does not exist.ERROR: excluded directory 'nanite' does not exist.cache_dirs started2025.05.23 19:24:23 Executed find in (72s) 72.40s, wavg=72.40s Idle____________ depth 4(timeout 30s) slept 0s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=2 scan_tmo=30s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=16%, filecount[4]=521982025.05.23 19:25:36 Executed find in (51s) 51.23s, wavg=58.29s Idle____________ depth 5 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=10%, filecount[5]=1306282025.05.23 19:26:28 Executed find in (22s) 22.75s, wavg=40.52s Idle____________ depth 6 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=12%, filecount[6]=2038442025.05.23 19:26:52 Executed find in (19s) 19.19s, wavg=31.99s Idle____________ depth 7 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU= 9%, filecount[7]=2970862025.05.23 19:27:12 Executed find in (21s) 21.86s, wavg=28.61s Idle____________ depth 8 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU= 6%, filecount[8]=3562812025.05.23 19:27:35 Executed find in (15s) 15.68s, wavg=24.92s Idle____________ depth 9 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU= 9%, filecount[9]=4108592025.05.23 19:27:52 Executed find in (4s) 04.29s, wavg=19.76s Idle____________ depth 10 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=16%, filecount[10]=4305182025.05.23 19:27:57 Executed find in (3s) 03.98s, wavg=16.25s Idle____________ depth 11 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=18%, filecount[11]=4486402025.05.23 19:28:02 Executed find in (5s) 05.83s, wavg=14.17s Idle____________ depth 12 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=14%, filecount[12]=4658102025.05.23 19:28:09 Executed find in (8s) 08.79s, wavg=13.19s Idle____________ depth 13 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=13%, filecount[13]=4855552025.05.23 19:28:19 Executed find in (8s) 08.55s, wavg=12.42s Idle____________ depth 14 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=13%, filecount[14]=5053552025.05.23 19:28:28 Executed find in (5s) 05.67s, wavg=11.38s Idle____________ depth 15 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=16%, filecount[15]=5180852025.05.23 19:28:35 Executed find in (2s) 02.66s, wavg=10.13s Idle____________ depth 16 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=26%, filecount[16]=5263422025.05.23 19:28:39 Executed find in (2s) 02.55s, wavg=09.12s Idle____________ depth 17 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=28%, filecount[17]=5330612025.05.23 19:28:42 Executed find in (2s) 02.28s, wavg=08.27s Idle____________ depth 18 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=33%, filecount[18]=5404542025.05.23 19:28:46 Executed find in (2s) 02.27s, wavg=07.56s Idle____________ depth 19 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=31%, filecount[19]=5448092025.05.23 19:28:49 Executed find in (2s) 02.27s, wavg=06.97s Idle____________ depth 20 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=29%, filecount[20]=5464362025.05.23 19:28:52 Executed find in (2s) 02.30s, wavg=06.48s Idle____________ depth 9999 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=29%, filecount[9999]=546900I'm running v2.2.7: see head -n 150 /usr/local/bin/cache_dirs```# Version 2.2.7 - Fixed -a option which was broken in 2.2# arberg####################################################################################version=2.2.7program_name=$(basename "$0")arg_count=$#run_log="/var/log/cache_dirs.log"csv_log="/var/log/cache_dirs.csv"lost_cache_log="/var/log/cache_dirs_lost_cache.csv"```Notice this at the top of the log file:```---------- Caching Directories ---------------appdata backup md5 media secure storage sync```My cfg looks like this: config\plugins\dynamix.cache.dirs\dynamix.cache.dirs.cfg ```options="-p 1 -S -i "appdata" -i "backup" -i "md5" -i "media" -i "secure" -i "storage" -i "sync" -i "windowsbackup" -e "app" -e "appbackup" -e "cacheonly" -e "docker" -e "home" -e "muaddibhdd" -e "nanite" -e "pil" -e "svn" -e "svnbackup" -e "system" -e "ubuntu" -e "windowsbackup" -c 4 -W 61 -Z 149 -U 100000 -l on -a '-noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -name .sync -prune -o -print'"service="1"suspend="-S"include="appdata,backup,md5,media,secure,storage,sync,windowsbackup"exclude="app,appbackup,cacheonly,docker,home,muaddibhdd,nanite,pil,svn,svnbackup,system,ubuntu,windowsbackup"adaptive="1"depth=""scanTimeoutIdle=""scanTimeoutBusy=""scanTimeoutStable="149"minimum=""maximum=""log="on"other="-a '-noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -name .sync -prune -o -print'"diskIdleTimer="61"cachePressure="1"ulimit="100000"mulithreaded=""scan_user_share="-u"mindepth="2"minDepth="4"shares=""```If you play around with it, you can probably solve your problem. Play around with the cfg and start and stop the script manually and watch the log file. Or rather that's probably what I would do. You can also find the command it runs with ps -ef | grep cacheAnd checkps -ef | grep findto see what it is calling find on. Though you have to catch. You can also ask deepseek for how to get process tree from the cache_dirs execution.
May 24, 20251 yr Well, thank you @Alex R. Berg for your contribution, but I have done it already, few days ago just copied your previously posted .cfg, replaced the shares with applicable ones, and replaced the exclusions, and still no result - number of files are not getting changed, and the find command is the same like with default one.
June 6, 20251 yr Managed to fix that issue by:replacing args="-noleaf" withargs="-noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name backups-prune -o -name .sync -prune -o -print"and to show the difference on filecounting, changed local thisDirCount=$(find $dir_to_scan -type f $depth_arg | wc -l)to local thisDirCount=$(find $dir_to_scan $args $depth_arg | wc -l)So in my opinion, those options are not working correctly without directly changing the script, as tried to add those parameters to option variable or args variable, but in my case those options were never included in the final find command.I know this is a temporary solution and kind of a hard-link one, @dlandon could you take a look at it? :)
July 13, 2025Jul 13 @silverkin good suggestion, I also tested locally for a couple of days, works great.@Alex R. Berg I have create a pull request here:GitHubAdd support for passing user-defined options to file coun...Ability to pass 'User defined options' to file counting Merge request for the fix suggested here: https://forums.unraid.net/topic/4351-cache_dirs-an-attempt-to-keep-directory-entries-in-ram...
July 14, 2025Jul 14 @silverkin @dlandon freeender Its rather time consuming to investigate for me and I don't have the details to detect what you are seeing and why. I really need to see the non-short version of the log file written. It contains lots of important information.You cannot just update args inside the script and add filters, it breaks other stuff such as determining which folders are under backup. Ah, maybe you are trying to just avoid backup_news via find filters? You cannot do that, an exclude is required. I didn't remember that before spending some hours reading the script, to review the change request...The '-type f' you found only affects initial counts, and it writes the find statement itself, because it has already built a list of relevant dirs. Removing the -type f, wont make it scan less, as the dirs to scan is in a for outside. These things will probably break if you hack the script, and enter overwrite the default args intead of using the proper settings. See build_dir_list function, which need to work for count_files to work. I think you broke build_dir_list by the manual edit of the file. Best Alex
July 14, 2025Jul 14 In my use case I have a share called /mnt/disk1/backup and under this share I have a dir called nas-docker. This dir is huge it has 30+ levels of depth and a lot of files. I noticed 2 issues:1) Use Adaptive depth: Adaptive.With default settings this would run 2 find commands (one that uses 'User defined options' and another one that counts files) back 2 back 30 times.Each run would be super slow.When I changed "Use Adaptive depth" to "Fixed" with Maximum level depth = 24, this reduced functions calls to just 2 finds per run.2) Then I exclude nas-docker from /mnt/disk1/backup by adding the following arg:-a '-name pictures -prune -o -name nas-docker -prune -o -print'as per instructions from the plugin descriptions:Its possible to tell cache_dirs to avoid traversing all directories with a certain name. The following example avoids '.Recycle.Bin' and 'temp' and 'log' folders: -a '-noleaf -name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -print' This would speed up the first find, however the second one would be running for a long time (It would travers all dirs because filtering arguments are not passed to the second find command)LogsThis is the log when the original script with default settings without any User defined options reaches depth level 7:cache_dirs version 2.2.9 Setting Memory ulimit to 50000 Setting cache_pressure=10 Arguments=-i backup -i cottonwood -i media -i trigkey -c 4 -l on Max Scan Secs=10, Min Scan Secs=1 Scan Type=adaptive Min Scan Depth=4 Max Scan Depth=none Use Command='find -noleaf' ---------- Caching Directories --------------- backup cottonwood media trigkey ---------------------------------------------- Setting Included dirs: backup,cottonwood,media,trigkey Setting Excluded dirs: min_disk_idle_before_restarting_scan_sec=60 scan_timeout_sec_idle=150 scan_timeout_sec_busy=30 scan_timeout_sec_stable=30 frequency_of_full_depth_scan_sec=604800 cache_dirs started 2025.07.14 12:00:54 Executed find in (0s) 00.24s, wavg=00.24s Idle____________ depth 4 slept 0s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=2 scan_tmo=30s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU= 8%, filecount[4]=27939 2025.07.14 12:00:55 Executed find in (0s) 00.61s, wavg=00.49s Idle____________ depth 5 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=150 s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU= 7%, filecount[5]=58493 2025.07.14 12:00:57 Executed find in (21s) 21.35s, wavg=10.92s Idle____________ depth 6 slept 1s Disks idle before/after scan 9998s/9998s Scan completed/timedOut counter cnt=0/1/0 mode=1 scan_tmo=15 0s maxCur=9999 maxWeek=9999 isMaxDepthComputed=0 CPU=16%, filecount[6]=167066Depth level 6 takes 21 seconds and level 7 was running for 3.5 minutes until I cancelled the test:
July 15, 2025Jul 15 I issued an update to the cache_dirs plugin some time ago to make some UI changes. Limetech has forked the plugin to their repository and will be maintaining it going forward.
July 21, 2025Jul 21 @dlandon Could you help us with link to the new limetech repo, if its public?@Freender Thank for the details and the log.The problem is that the file counting phase will not include the find-options with filters given with -a. But this file counting phase only runs on the initial scans at each depth. Of cause that is rather unfortunate as that means the files listed is incorrect, plus its not nice that it scans a huge amount of files on reboot. The adaptive runs the counts at each adaptive depth (initially), which is by default 20 levels (depth_max_incremental_depth=20) + infinite (9999). So shouldn't scan 30 but 21 times I think. But maybe you just rounded or guestimated which is fine.Indeed the non-adaptive does it in one go, and does not repeat it. The adaptive also does not repeat the count once initial 21 scans is complete.I have experimented with a fix. What worries me a bit is changing other peoples options by fixing them for the file count. But I think this worksI've created a fix here. I have temporarily placed it in a gist. We would have to reapply the changes to the real current version, rather than the one i had lying:https://gist.github.com/arberg/08adf6259f09c76274db7feaf38fc81eThe 1st revision is the original version I applied my changes to.I ran it with sudo cache_dirs -q; sudo rm /var/log/cache_dirs.log; sudo ./cache_dirs -p 1 -i appdata -i backup -i md5 -i secure -i storagehdd -i sync -i windowsbackup -e app -e appbackup -e docker -e home -e muaddibhdd -e pil -e svn -e svnbackup -e system -e ubuntu -e windowsbackup -c 4 -W 61 -Z 149 -U 100000 -l on -a '-name .Recycle.Bin -prune -o -name log -prune -o -name temp -prune -o -name .sync -prune -o -print'; psg cache_dirsmy alias:alias psg='ps -e x -o ppid,pid,pgid,user,group,uid,gid,%cpu,%mem,tty,vsz,rss,etime,cputime,rgroup,ni,fname,args | body grep'use 'psg cache_dirs', to obain the full command you use for cache_dirs before you stop the original running script, so you can copy the arguments you use.
July 23, 2025Jul 23 On 7/21/2025 at 7:07 AM, Alex R. Berg said:Could you help us with link to the new limetech repo, if its public?GitHubGitHub - unraid/cache_dirsContribute to unraid/cache_dirs development by creating an account on GitHub.
January 22Jan 22 Hello everyone,After a reboot, the entire cache is empty again. Is there a command that can be executed to cache all directories once? So that the hard disks are not gradually awakened again and again.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.