Jump to content
dlandon

File Activity Plugin - How can I figure out what keeps spinning up my disks?

124 posts in this topic Last Reply

Recommended Posts

This is a plugin that shows the file and directory activity on the array by disk.  This includes read and write activity on the array.  Read activity can cause disk spin ups.  It is recommended that you have the Dynamix Cache Directories plugin installed and running.

 

The plugin uses inotify to log read and write disk activity.  You will need to start File Activity to start the logging.  I don't recommend leaving it on all the time.  A server with a lot of file activity can fill the log space.

 

You can see which files and directories are being read and written and keeping the disks from spinning down.  The file activity is shown by disk, UD disks, and the cache drive.  The 'appdata' directory is excluded.

 

This is not the ultimate answer for disk spin ups, but I used it to find an issue with Windows 10 backing up files every hour.  It would write new files on the cache, but constantly modify the older file directory entries causing spin ups on disks.

 

The plugin is for version unRAID 6.3 and later.  It is installed in the Tools menu under System Information.

 

The preferred way to install the plugin is with CA.

 

Or you can install the plugin by copying the following URL in the install plugin line of unRAID:

https://github.com/dlandon/file.activity/raw/master/file.activity.plg
 
Edited by dlandon

Share this post


Link to post

Just in short what exactly is the difference between this and the Open Files Plugin?

I know the Open File one gives you specific files that are currently open and this new one seems to give you a list of last accessed files.

 

Just looking for some clarification, but I will play with them both of course. ;)

Share this post


Link to post

This plugin shows files on the array that have been written or modified recently.  Open files shows currently open files.  These could be open for read or write access.

Share this post


Link to post

Hmmm ...  I wonder if this could be enhanced to do both?!    ;)

 

Perhaps a toggle to show only open, or only written, or both.

Share this post


Link to post

Hmmm ...  I wonder if this could be enhanced to do both?!    ;)

 

Perhaps a toggle to show only open, or only written, or both.

 

Not sure.  They kind of do different things and it might be confusing to combine them.

Share this post


Link to post

Ah recent activity app. Now I get it.

I like that idea.

 

I see there is a refresh button or is it not storing data in log files?

Share this post


Link to post

The plugin has to query the disks to determine the last file and directory modify times.  There is currently no way to store this information in a log.  Linux has an audit ability to show all file activity, but the kernel has to be built with that ability enabled.  It may be worth the effort for me to investigate that further.

 

I am trying to help answer the questions about disks spinning up.

Share this post


Link to post

Hmmm ...  I wonder if this could be enhanced to do both?!    ;)

 

Perhaps a toggle to show only open, or only written, or both.

 

Not sure.  They kind of do different things and it might be confusing to combine them.

 

You're right, I didn't realize how different they are.  One is more of a search result, and the other a monitor(?).

Share this post


Link to post

Linux has an audit function that I am looking into that is a log of all file activity.  The problem is that the 'audit' functionality has to be built into the kernel.  Without it the 'audit' package I need to use won't work.  I will need LT support on enabling the 'audit' capability.  I'm not sure exactly where to go with this right now.

 

What I am trying to do is answer the age old question of why disks are spinning up and why doesn't cache_dirs prevent it (it can't on writes).  There is a lot of mis-understanding of what cache_dirs can and can't do, and some unrealistic expectations.  I am hoping to provide a tool to show users what is accessing disks and making them spin up.

 

The Open Files plugin shows currently open files and which processes have them open.  The Files Activity plugin shows the write history within the last two hours on the disk, cache, and UD shares.  Very different animals.

Share this post


Link to post

Perhaps the inotifywait utility maybe useful here?

 

It can be set up to monitor file read/write activity and you would need to write some functionality to store the file names in some windowed fashion.

 

 

Share this post


Link to post

If you have installed this plugin, please use the following link to update the plugin.  Because of a bug in my plugin, it will not update through the plugins update.  Remove the previous version first.

 

https://github.com/dlandon/file.activity/raw/master/file.activity.plg

 

This will load the latest version that will update properly.

Share this post


Link to post

There is a new version of the file activity plugin.  I changed to using inotify to log the write disk activity.

Share this post


Link to post

I have worked out how to log open read and write file events so the plugin will log read and write file activity.  It turns out that inotify is excessively chatty when tracking open file and directory events.  I managed to filter out all the extraneous open events that don't apply to disk spin ups.

 

Hopefully now all file activity that can cause disk spin ups will be logged.  I'll release a new version tomorrow after a bit more testing.

Share this post


Link to post

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

Share this post


Link to post

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

 

Which parts?

 

EDIT: So I'm thinking:

UD Disks

Cache

 

As the choices?

Share this post


Link to post

Don't know how easy it would be, but the obvious answer is to have an option to automatically exclude any filesystems residing on SSD media, since the title of the plugin is about spinning disks.  ;D

Share this post


Link to post

Which parts?

 

EDIT: So I'm thinking:

UD Disks

Cache

 

As the choices?

 

Don't know how easy it would be, but the obvious answer is to have an option to automatically exclude any filesystems residing on SSD media, since the title of the plugin is about spinning disks.  ;D

 

I'm not sure I'd automatically exclude SSD media but being able to exclude UD and/or Cache would be perfect.

 

Kevin.

 

Share this post


Link to post

New version available 2017.02.18a allows you to enable/disable UD and cache.

 

I don't like the idea of automatically excluding SSD disks.  There may be a reason someone wants to see disk activity on a SSD disk.  This way the user can make the decision on which ones to include/exclude.

Share this post


Link to post

Is it possible to have settings for this so that each part of the array can be unselected.

 

My reasoning for this is that compared to Plex appdata on the cache drive and mv vm's on an unassigned device the main array gets very few read and write requests and I've found that they can get lost in the noise produced by the other two.

 

Kevin.

 

The 'appdata' share is always excluded just for the reason you mentioned - too much activity and there is no value in tracking that.

Share this post


Link to post

I don't like the idea of automatically excluding SSD disks.  There may be a reason someone wants to see disk activity on a SSD disk.  This way the user can make the decision on which ones to include/exclude.

I agree, I was just trying to be funny, pointing out the title of the thread references spinning up disks, but not disk activity.  ::)

 

I suppose if you want to be pedantic the reference to spinning up something has been turned into a colloquialism meaning to start using it.

Share this post


Link to post

I get it.

 

The reason I added that to the title is because I have gotten tired of the constant "My disks are spinning up for no reason.  Cache_dirs must be broken."

 

Hopefully this plugin can show people what disk activity is happening and they can understand why disks spin up.

Share this post


Link to post

Version 2017.02.19 has been released.

 

I have learned that inotify is pretty chatty when a lot of disk activity is occurring and the Files Activity log file can get excessively large.  I have released a new version that will keep the log file to the last 20,000 events so it doesn't fill the log space.

Share this post


Link to post

I use latest Unraid and latest File Activity plugin.

 

I set settings to Yes, No, No and apply,

but it doesn't show anything in file activity tab.

I tried to open a movie, browse folder but it doesn't show anything there.

Any idea of what I should try?

Share this post


Link to post

You turned off the cache and that is probably where your files are located.  Turin it on and try again.  Be sure the file activity shows as running.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.