File Activity Plugin - How can I figure out what keeps spinning up my disks?


dlandon

Recommended Posts

This is a plugin that shows the file and directory activity on the array by disk.  This includes read and write activity on the array.  Read activity can cause disk spin ups.  It is recommended that you have the Dynamix Cache Directories plugin installed and running.

 

The plugin uses inotify to log read and write disk activity.  You will need to start File Activity to start the logging.  I don't recommend leaving it on all the time.  A server with a lot of file activity can fill the log space.

 

You can see which files and directories are being read and written and keeping the disks from spinning down.  The file activity is shown by disk, UD disks, and the cache and Pool drives.  The 'appdata', 'syslogs', and 'system' shares are excluded.

 

This is not the ultimate answer for disk spin ups, but I used it to find an issue with Windows 10 backing up files every hour.  It would write new files on the cache, but constantly modify the older file directory entries causing spin ups on disks.

 

The plugin is installed in the Tools menu under System Information.

 

The preferred way to install the plugin is with CA.

 

Or you can install the plugin by copying the following URL in the install plugin line of unRAID:

https://github.com/dlandon/file.activity/raw/master/file.activity.plg
 
  • Like 1
  • Thanks 1
  • Upvote 3
Link to comment

Just in short what exactly is the difference between this and the Open Files Plugin?

I know the Open File one gives you specific files that are currently open and this new one seems to give you a list of last accessed files.

 

Just looking for some clarification, but I will play with them both of course. ;)

Link to comment

The plugin has to query the disks to determine the last file and directory modify times.  There is currently no way to store this information in a log.  Linux has an audit ability to show all file activity, but the kernel has to be built with that ability enabled.  It may be worth the effort for me to investigate that further.

 

I am trying to help answer the questions about disks spinning up.

Link to comment

Hmmm ...  I wonder if this could be enhanced to do both?!    ;)

 

Perhaps a toggle to show only open, or only written, or both.

 

Not sure.  They kind of do different things and it might be confusing to combine them.

 

You're right, I didn't realize how different they are.  One is more of a search result, and the other a monitor(?).

Link to comment

Linux has an audit function that I am looking into that is a log of all file activity.  The problem is that the 'audit' functionality has to be built into the kernel.  Without it the 'audit' package I need to use won't work.  I will need LT support on enabling the 'audit' capability.  I'm not sure exactly where to go with this right now.

 

What I am trying to do is answer the age old question of why disks are spinning up and why doesn't cache_dirs prevent it (it can't on writes).  There is a lot of mis-understanding of what cache_dirs can and can't do, and some unrealistic expectations.  I am hoping to provide a tool to show users what is accessing disks and making them spin up.

 

The Open Files plugin shows currently open files and which processes have them open.  The Files Activity plugin shows the write history within the last two hours on the disk, cache, and UD shares.  Very different animals.

Link to comment

If you have installed this plugin, please use the following link to update the plugin.  Because of a bug in my plugin, it will not update through the plugins update.  Remove the previous version first.

 

https://github.com/dlandon/file.activity/raw/master/file.activity.plg

 

This will load the latest version that will update properly.

Link to comment

I have worked out how to log open read and write file events so the plugin will log read and write file activity.  It turns out that inotify is excessively chatty when tracking open file and directory events.  I managed to filter out all the extraneous open events that don't apply to disk spin ups.

 

Hopefully now all file activity that can cause disk spin ups will be logged.  I'll release a new version tomorrow after a bit more testing.

Link to comment

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

Link to comment

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

 

Which parts?

 

EDIT: So I'm thinking:

UD Disks

Cache

 

As the choices?

Link to comment

Which parts?

 

EDIT: So I'm thinking:

UD Disks

Cache

 

As the choices?

 

Don't know how easy it would be, but the obvious answer is to have an option to automatically exclude any filesystems residing on SSD media, since the title of the plugin is about spinning disks.  ;D

 

I'm not sure I'd automatically exclude SSD media but being able to exclude UD and/or Cache would be perfect.

 

Kevin.

 

Link to comment

New version available 2017.02.18a allows you to enable/disable UD and cache.

 

I don't like the idea of automatically excluding SSD disks.  There may be a reason someone wants to see disk activity on a SSD disk.  This way the user can make the decision on which ones to include/exclude.

Link to comment

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

 

The 'appdata' share is always excluded just for the reason you mentioned - too much activity and there is no value in tracking that.

Link to comment

I don't like the idea of automatically excluding SSD disks.  There may be a reason someone wants to see disk activity on a SSD disk.  This way the user can make the decision on which ones to include/exclude.

I agree, I was just trying to be funny, pointing out the title of the thread references spinning up disks, but not disk activity.  ::)

 

I suppose if you want to be pedantic the reference to spinning up something has been turned into a colloquialism meaning to start using it.

Link to comment

I get it.

 

The reason I added that to the title is because I have gotten tired of the constant "My disks are spinning up for no reason.  Cache_dirs must be broken."

 

Hopefully this plugin can show people what disk activity is happening and they can understand why disks spin up.

Link to comment

Version 2017.02.19 has been released.

 

I have learned that inotify is pretty chatty when a lot of disk activity is occurring and the Files Activity log file can get excessively large.  I have released a new version that will keep the log file to the last 20,000 events so it doesn't fill the log space.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.