Thanks, dlandon.

 

I want to have a github version-tracking of non-zipped, so I can track changes properly. It would we valuable for the community if someone hosted such a one on Github. It would probably be better if someone else hosts the cache_dirs, maybe someone who responds as quickly as you do dlandon

 

I've changed my unRAID repository to contain the raw file, and added links to dlandon and dynamix githubs. https://github.com/arberg/unRAID

 

I've updated the cache_dirs a bit, but its still work in progress. I hope to make the script prevent spamming disks, which was my idea for the adaptiveness in the first place. So I might make the script back off a lot sooner and for a longer period when it detects it is spamming disk.

 

Alex

 

PS on your GitHub comment

>> Thank you Alex R. Berg for your updates to cache_dirs.

You are welcome

22 minutes ago, Alex R. Berg said:

Thanks, dlandon.

 

I want to have a github version-tracking of non-zipped, so I can track changes properly. It would we valuable for the community if someone hosted such a one on Github. It would probably be better if someone else hosts the cache_dirs, maybe someone who responds as quickly as you do dlandon

 

I've changed my unRAID repository to contain the raw file, and added links to dlandon and dynamix githubs. https://github.com/arberg/unRAID

 

I've updated the cache_dirs a bit, but its still work in progress. I hope to make the script prevent spamming disks, which was my idea for the adaptiveness in the first place. So I might make the script back off a lot sooner and for a longer period when it detects it is spamming disk.

 

Alex

 

PS on your GitHub comment

>> Thank you Alex R. Berg for your updates to cache_dirs.

You are welcome

I can add an unzipped script to my github and you can do pr's to make changes and I'll release it as the zipped version for the plugin.

48 minutes ago, Alex R. Berg said:

Thanks, dlandon.

 

I want to have a github version-tracking of non-zipped, so I can track changes properly. It would we valuable for the community if someone hosted such a one on Github. It would probably be better if someone else hosts the cache_dirs, maybe someone who responds as quickly as you do dlandon

 

I've changed my unRAID repository to contain the raw file, and added links to dlandon and dynamix githubs. https://github.com/arberg/unRAID

 

I've updated the cache_dirs a bit, but its still work in progress. I hope to make the script prevent spamming disks, which was my idea for the adaptiveness in the first place. So I might make the script back off a lot sooner and for a longer period when it detects it is spamming disk.

 

Alex

 

PS on your GitHub comment

>> Thank you Alex R. Berg for your updates to cache_dirs.

You are welcome

@Alex R. Berg you are all set on my github.  I can do some testing for you and I'll release the updated script when it's ready.

Good. I've committed my changes for now to my own repository here: https://github.com/arberg/cache_dirs

When I think its worthy of your time, i'll post a pull-request to your repository. Or wait a week and have a look, if I should suddenly get busy elsewhere and not return.

 

I have not read up on all the issues posted here on cache-dirs, but as far as I have picked up (and from my own experience) cache-dirs still in my adaptive version spams to much on the disks. I found some definite design-issues with my adaptive scan, where it would keep trying to hard to get to it previous best depth. I have committed my changes, and will keep looking at it.

 

I've considered updating the help for the cache-dir plugin. I've found it valuable to set max-scan depth even though I've now made the adaptive scanning. Its valuable to make sure not to have more files than needed under cache-dirs scanning eye, to reduce disk-spam, and I might write that in the documentation. 

 

Do we have any kind of guideline anywhere for how to write help? If I update FolderCachingSettings.page and make a pull-request, will BergWare do his magic?

https://github.com/bergware/dynamix/blob/master/source/cache-dirs/FolderCachingSettings.page

 

 

Best Alex

2 hours ago, Alex R. Berg said:

I've considered updating the help for the cache-dir plugin. I've found it valuable to set max-scan depth even though I've now made the adaptive scanning. Its valuable to make sure not to have more files than needed under cache-dirs scanning eye, to reduce disk-spam, and I might write that in the documentation. 

 

Do we have any kind of guideline anywhere for how to write help? If I update FolderCachingSettings.page and make a pull-request, will BergWare do his magic?

https://github.com/bergware/dynamix/blob/master/source/cache-dirs/FolderCachingSettings.page

 

 

Best Alex

Yes he will.  He's very responsive and would probably appreciate the pr.  Do a pull request and you'll see how the help is done.  Its pretty easy.  The plugin currently does not set up the script for adaptive scan.  I've asked bonienl to add that as an option.

Sure, go ahead make your pull request and I'll do further updates and publish the new version.

 

On 12.3.2018 at 1:22 PM, bonienl said:

I'll put this on my todo list. Had several (failed) attempts in the past, and need a good review of the approach to take.

 

Hi bonienl,

is there any intermediate fix for the problem with the S3 Script? Maybe a downgrade to the old algorithm to determinate if a disk is spun down or a way to make the hdparm check optional?

I just installed the autofan plugin, are there any instructions somewhere on how to get it working properly? It does pick up 3 fan controllers but if i then click detect it doesn't find anything.

 

Edited April 8, 20188 yr by Shamalamadindong

Is there any way to change the fan/temperature sensor labels? I tried changing all instances of 'MB Temp' to 'Chipset Temp' (for example) in all of the plugin files (including the ones in the .txz repository) but they all just revert back.

On 7.4.2018 at 5:40 PM, darkside40 said:

 

Hi bonienl,

is there any intermediate fix for the problem with the S3 Script? Maybe a downgrade to the old algorithm to determinate if a disk is spun down or a way to make the hdparm check optional?

I would be interested too

Cache_dirs update

 

I've update the cache_dirs plugin. Made many many small changes that I consider improvements. I've update the dynamix plugin folder, fixed a bug in the start script, and added options to the settings page and added help info.

 

I've forked dlandon and dynamix githubs, and pushed my changes. I have updated the dynamix-txz and cache_dirs.zip in my forks, so if you use this temporary plg attached here which points to my forks, you can test it. The new plugin requires the new cache_dirs script because I've added new options to the script.

 

The current attached cache_dirs plugin does not start when the server reboots on my server where it auto-start the array. I hope someone can fix that for me. 

 

If you send me logs from /var/log/cache_dirs.log and /var/log/cache_dirs_lost_cache.csv I can inspect them, though I'll probably quickly tire of looking at logs, but it would be helpful to see how cache_dir is behaving on other systems.

 

# Version 2.2.0g- When cache lost, don't wait for disks idle, instead decrease depth
#               - Now scans each disks and cache in separate processes. Thus removed option -u for scan user
#               - Fixed loss of adaptive after 1 week
#               - Weekly rescan only when disks been idle long
#               - Apaptive scan will not retry increasing level to previous seen stable level until disks have been idle for a long time (20s)
#               - -q now also kills subproccesses immediately (so cache_dirs does not prevent stopping the array)
#               - Added file-count and depth adjustment by filecount
# arberg 

 

If your disks are never idle for 20 seconds the adaptive scanning will no longer work for you. If the disks are idle once in a while, I believe it will work far better than before.

 

Best Alex

dynamix.cache.dirs.zip

Will this get updated automatic?

10 hours ago, nuhll said:

Will this get updated automatic?

 

Its not yet promoted to the automatic update stream. That will happen when its tested a bit more and approved. Till then you can try the changes by replacing the cache-dirs plugin with the version i linked. You can just copy the file over the existing in /boot/config/plugins/dynamix.cache.dirs.plg and reboot (or run something like installplg or remove and install from gui). 

 

Best Alex

On 4/7/2018 at 5:40 PM, darkside40 said:

 

Hi bonienl,

is there any intermediate fix for the problem with the S3 Script? Maybe a downgrade to the old algorithm to determinate if a disk is spun down or a way to make the hdparm check optional?

 

Sorry, I don't have time right now to look into this, and have to keep it on my todo list.

14 hours ago, bonienl said:

 

Sorry, I don't have time right now to look into this, and have to keep it on my todo list.

 

Fair enough mate, Is there a http command that i could send the server to tell it to manually sleep?

 

This way in the meantime i could have Home Assistant put it to sleep once my media players have been switched off for the night?

Cache_dirs update 2.2.0

 

I think I'm done with the 2.2.0 update. I've attached a version of the plugin that points to my forked github so you can test it until it goes live. Note that because of how the plugin system works

 

I've fixed the issues mentioned in my previous comment, about the plugin not starting on boot, it was my bad.

 

Users notes

• The new cache_dirs scans your disks when theive been idle for 60 seconds. This can now be adjusted in the options. If the disks are idle once in a while, it seems to work far better than before.

• Cache-pressure 0 works impressively for my 1.5 million files and 16 GB of memory, but it may crash the system, because linux docs says it will not release cache-memory even if the system is running out of memory.  

 

Programmers notes

I've moved the cache_dirs to the dynamix plugin github. Te plugin and the cache-dirs version are tied together, the current cache-dirs page does not work properly with older cache_dirs versions. The plugin contained the MD5 of the cache_dirs.zip, and if the cache_dirs.zip was updated but the plugin was not updated, it would cause 'unexpected' behaviour, such as redownload on each reboot.

Since I've moved cache_dirs to dynamix github I've added a note that its not under copyright of dynamix. You can move it back out if you want. The newest script is also committed in dlandons github (in my fork). 

 

Future ideas

• The cache_dirs log is very useful to tweak your personal system. If it was possible to copy the Syslog.htm page and point it to /var/log/cache_dirs.log and put a link to it in the Folder Caching page it would be very useful for users.
• It would be nice with restart buttons like on the PhAzE's Resilio sync plugin.

Release notes

# Version 2.2.0 - When cache lost, don't wait for disks idle, instead decrease depth
#               - Now scans each disks and cache in separate processes. Thus removed option -u for scan user
#               - Fixed loss of adaptive after 1 week
#               - Weekly rescan only when disks been idle long
#               - Apaptive scan will not retry increasing level to previous seen stable level until disks have been idle for a long time (20s)
#               - -q now also kills subproccesses immediately (so cache_dirs does not prevent stopping the array)
#               - Added file-count and depth adjustment by filecount 
# arberg 

 

Best Alex

 

dynamix.cache.dirs.zip

On 11.4.2018 at 8:02 AM, bonienl said:

 

Sorry, I don't have time right now to look into this, and have to keep it on my todo list.

Okay i just changed the script by myself using the HDD Disk active check method form 3.0.4.

Works perfect till now.

1 hour ago, darkside40 said:

Okay i just changed the script by myself using the HDD Disk active check method form 3.0.4.

Works perfect till now.

Do you mind sharing your changes?

3 hours ago, darkside40 said:

Okay i just changed the script by myself using the HDD Disk active check method form 3.0.4.

Works perfect till now.

 

Very good!!!

Do you publish your script?

Here you can find the modified script:

https://pastebin.com/knv869Aq

 

The modifications are done between line 233-236, just commenting out the new check and adding the old style activity check from 3.0.4.

 

I have also added an archive which includes the plg (because you need to change the md5sum of the txz after repackaging) and the txz with the script version mentioned above. Ignore the config, you will only need the plg and the txz.

For me it works, wouldnt take any guarantee  if it does for you.

 

It is just a really ugly hacky fix till bonienl has the time to make an official fix, because the 3.0.6 didnt work for me most of the time.

 

sleep.tar.gz

Edited April 24, 20188 yr by darkside40

11 minutes ago, darkside40 said:

Here you can find the modified script:

https://pastebin.com/knv869Aq

 

The modifications are done between line 233-236, just commenting out the new check and adding the old style activity check from 3.0.4.

 

I have also added an archive which includes the plg (because you need to change the md5sum of the txz after repackaging) and the txz with the script version mentioned above. Ignore the config, you will only need the plg and the txz.

For me it works, wouldnt take any guarantee  if it does for you.

 

It is just a really ugly hacky fix till bonienl has the time to make an official fix, because the 3.0.6 didnt work for me most of the time.

 

sleep.tar.gz

Thank you very much

Thanks for the modified script. I am also giving it a try. I disabled sleep for the moment. I installed the .txz file into the dynamic.s3.sleep folder on the flash drive and copied the .plg file overwriting the existing .plg file on the flash drive and restarted the plugin. Is that all that has to be done?

 

On 4/13/2018 at 4:09 AM, Alex R. Berg said:

Cache_dirs update 2.2.0

 

I think I'm done with the 2.2.0 update. I've attached a version of the plugin that points to my forked github so you can test it until it goes live. Note that because of how the plugin system works

 

I've fixed the issues mentioned in my previous comment, about the plugin not starting on boot, it was my bad.

 

Users notes

• The new cache_dirs scans your disks when theive been idle for 60 seconds. This can now be adjusted in the options. If the disks are idle once in a while, it seems to work far better than before.

• Cache-pressure 0 works impressively for my 1.5 million files and 16 GB of memory, but it may crash the system, because linux docs says it will not release cache-memory even if the system is running out of memory.  

 

Programmers notes

I've moved the cache_dirs to the dynamix plugin github. Te plugin and the cache-dirs version are tied together, the current cache-dirs page does not work properly with older cache_dirs versions. The plugin contained the MD5 of the cache_dirs.zip, and if the cache_dirs.zip was updated but the plugin was not updated, it would cause 'unexpected' behaviour, such as redownload on each reboot.

Since I've moved cache_dirs to dynamix github I've added a note that its not under copyright of dynamix. You can move it back out if you want. The newest script is also committed in dlandons github (in my fork). 

 

Future ideas

• The cache_dirs log is very useful to tweak your personal system. If it was possible to copy the Syslog.htm page and point it to /var/log/cache_dirs.log and put a link to it in the Folder Caching page it would be very useful for users.
• It would be nice with restart buttons like on the PhAzE's Resilio sync plugin.

Release notes

# Version 2.2.0 - When cache lost, don't wait for disks idle, instead decrease depth
#               - Now scans each disks and cache in separate processes. Thus removed option -u for scan user
#               - Fixed loss of adaptive after 1 week
#               - Weekly rescan only when disks been idle long
#               - Apaptive scan will not retry increasing level to previous seen stable level until disks have been idle for a long time (20s)
#               - -q now also kills subproccesses immediately (so cache_dirs does not prevent stopping the array)
#               - Added file-count and depth adjustment by filecount 
# arberg 

 

Best Alex

 

dynamix.cache.dirs.zip

Seems to be working nicely.  Hopefully @bonienl can make it official.  @bonienl let me know if you meed any help.

12 hours ago, bilben1 said:

Thanks for the modified script. I am also giving it a try. I disabled sleep for the moment. I installed the .txz file into the dynamic.s3.sleep folder on the flash drive and copied the .plg file overwriting the existing .plg file on the flash drive and restarted the plugin. Is that all that has to be done?

 

 

Should be. After a reboot you should see Script Version 3.0.6.1 on the S3 Sleep settings page.