So, "Disk2" refuses to spin down, it always seems to be in use (By that I mean if I force a disk shutdown, it'll stay down for ~ 10 minutes before starting back up). I started a inotifywait session using the following syntax:-

 

inotifywait -m -r /mnt/disk2

 

and after ~ 5 minutes every directory on the disk had a "OPEN,ISDIR" and "CLOSE_NOWRITE,CLOSE,ISDIR" request. Considering it's not just /TV Shows/ and /Movies/ (E.G. not plex), I have to assume it's the cachedir script. Can I ask something:-

 

Does the fact that cahedir scan the DIRs not reset the time to spin down back to 1 hours (my default) because it was accessed?

 

Only reason I don't remove cachedirs is because it works great on my other drives and plex scanning doesn't cause my disks to spin up, so, I'd prefer to keep it, I'm just trying to work out why disk2 is constantly up.

It does not reset the disk timer if the directory information is already in RAM, and no disk access is needed (in the linux disk buffer cache) since the disk is not accessed.

 

If something else is using your memory in an intense way, then it will be possible for all the memory to be used by it, and the disks spun up in resonse to the "find" command used by cache_dirs. 

 

It is easy to eliminate it from the cause of the spin-ups, don't use it, or invoke it with the -F and -v options to watch it as it scans in the foreground.  It typically scans a directory in a fraction of a second here (since it is all in RAM)

 

Joe L.

It does not reset the disk timer if the directory information is already in RAM, and no disk access is needed (in the linux disk buffer cache) since the disk is not accessed.

 

If something else is using your memory in an intense way, then it will be possible for all the memory to be used by it, and the disks spun up in resonse to the "find" command used by cache_dirs. 

 

It is easy to eliminate it from the cause of the spin-ups, don't use it, or invoke it with the -F and -v options to watch it as it scans in the foreground.  It typically scans a directory in a fraction of a second here (since it is all in RAM)

 

Joe L.

 

Okay, so, what I realize is:-

If the items are already in memory, when it tries to "read" them (To assign them to memory), it'll just read them of the memory and not require to spin up the disks.

 

But, what if a new file is written between the time of the last physical read to the disk and now, how will linux know that a new file has been added that's not in memory and it must spin up the disk to read it? Or, is it automatically added to memory the second it's written to?

It does not reset the disk timer if the directory information is already in RAM, and no disk access is needed (in the linux disk buffer cache) since the disk is not accessed.

 

If something else is using your memory in an intense way, then it will be possible for all the memory to be used by it, and the disks spun up in resonse to the "find" command used by cache_dirs. 

 

It is easy to eliminate it from the cause of the spin-ups, don't use it, or invoke it with the -F and -v options to watch it as it scans in the foreground.  It typically scans a directory in a fraction of a second here (since it is all in RAM)

 

Joe L.

 

Okay, so, what I realize is:-

If the items are already in memory, when it tries to "read" them (To assign them to memory), it'll just read them of the memory and not require to spin up the disks.

 

But, what if a new file is written between the time of the last physical read to the disk and now, how will linux know that a new file has been added that's not in memory and it must spin up the disk to read it? Or, is it automatically added to memory the second it's written to?

It is in memory once written, however, if you have 4Gig of ram (as an example) and subsequently play a 4Gig ISO image of a movie and nothing else running, all the ram would have been utilized to cache the blocks of data representing the movie.  The cache works by re-using the least recently referenced memory block.  In the case of a movie, it means you can rewind nearly instantly since the data is still in cache.   

 

cache_dirs simply accesses the directory information every few seconds by performing a "find" command on the directory tree.  Those data blocks then become the most recently used data blocks and will stay in the disk buffer cache if they do not become the least recently used (other data blocks, such as the beginning of the movie watched at some point int he past become the least recently used, and are therefore reused.)

 

Since typically cache_dirs re-scans the directories every few seconds, any new file written will be in memory immediately and its directory entries will be the same as any others in the directory tree.

It does not reset the disk timer if the directory information is already in RAM, and no disk access is needed (in the linux disk buffer cache) since the disk is not accessed.

 

If something else is using your memory in an intense way, then it will be possible for all the memory to be used by it, and the disks spun up in resonse to the "find" command used by cache_dirs. 

 

It is easy to eliminate it from the cause of the spin-ups, don't use it, or invoke it with the -F and -v options to watch it as it scans in the foreground.  It typically scans a directory in a fraction of a second here (since it is all in RAM)

 

Joe L.

 

Okay, so, what I realize is:-

If the items are already in memory, when it tries to "read" them (To assign them to memory), it'll just read them of the memory and not require to spin up the disks.

 

But, what if a new file is written between the time of the last physical read to the disk and now, how will linux know that a new file has been added that's not in memory and it must spin up the disk to read it? Or, is it automatically added to memory the second it's written to?

It is in memory once written, however, if you have 4Gig of ram (as an example) and subsequently play a 4Gig ISO image of a movie and nothing else running, all the ram would have been utilized to cache the blocks of data representing the movie.  The cache works by re-using the least recently referenced memory block.  In the case of a movie, it means you can rewind nearly instantly since the data is still in cache.   

 

cache_dirs simply accesses the directory information every few seconds by performing a "find" command on the directory tree.  Those data blocks then become the most recently used data blocks and will stay in the disk buffer cache if they do not become the least recently used (other data blocks, such as the beginning of the movie watched at some point int he past become the least recently used, and are therefore reused.)

 

Since typically cache_dirs re-scans the directories every few seconds, any new file written will be in memory immediately and its directory entries will be the same as any others in the directory tree.

 

Thanks for this, made a lot more sense and also made me drop my per-scan time from 900 seconds --> 1 second, as per I thought it did a physical scan on the drive each time.

 

Thanks, once again.

Thanks for this, made a lot more sense and also made me drop my per-scan time from 900 seconds --> 1 second, as per I thought it did a physical scan on the drive each time.

So you caused your own issue by messing with the default scan times.

 

If you just ignore the -m and -M options (do NOT use them at all) it will work as expected.  Let it figure out the times to use on its own.

 

With your 900 second delay between scans, it is EASILY possible for your playing of media to cause the directory inodes cached to be the least recently accessed, and therefore be reused. (and the disk subsequently spun-up once more when the 900 seconds was up.)

Thanks for this, made a lot more sense and also made me drop my per-scan time from 900 seconds --> 1 second, as per I thought it did a physical scan on the drive each time.

So you caused your own issue by messing with the default scan times.

 

If you just ignore the -m and -M options (do NOT use them at all) it will work as expected.  Let it figure out the times to use on its own.

 

With your 900 second delay between scans, it is EASILY possible for your playing of media to cause the directory inodes cached to be the least recently accessed, and therefore be reused. (and the disk subsequently spun-up once more when the 900 seconds was up.)

 

I didn't even think of default times too be honest, I read the manual and instantly started fiddling with stuff. I'm sorry 

Thanks for this, made a lot more sense and also made me drop my per-scan time from 900 seconds --> 1 second, as per I thought it did a physical scan on the drive each time.

So you caused your own issue by messing with the default scan times.

 

If you just ignore the -m and -M options (do NOT use them at all) it will work as expected.  Let it figure out the times to use on its own.

 

With your 900 second delay between scans, it is EASILY possible for your playing of media to cause the directory inodes cached to be the least recently accessed, and therefore be reused. (and the disk subsequently spun-up once more when the 900 seconds was up.)

 

OR... If you have an enormous amount of directories that cache_dirs is trying to get in memory it might also happen that the time between the "scans" of a particular directory are so far in between that cache_dirs actually -triggers- a disk spin up instead of preventing it... Right ?  I am guessing this is what is happening on my system, I have not messed with the parameters but with cache_dirs running I actually have more disks spun up then without it..

 

I have now excluded a lot of shares from getting scanned because they are:

 

- very irregularly used;

OR

- when viewing the dir will most certainly lead to the need for a file listing (like with my photos share, no sense in cache_dir'in it as windows will immediately start to show thumbnails together with the directory listing, or like with my music share for itunes, itunes will want to read the files on startup anyhow so there is again no sense in caching the rather large directory structure.

 

Basically I am now allmost only caching my media shares..  Wondering how this will work... !

And I can report it works beautifully... I see an allmost spun down array again after months... !