(SOLVED) something is reading SMART data on all drives every half hour!

[CaptainTivo]

I was poking around in my newly updated server (6.11.5) and noticed in the system log that SMART data is being read on every drive every half hour!  By default, I spin down all drives after 30 minutes of idle (no access) so this means that they are all spinning about half of the time, 24/7!  This not good.

I run the Scrutiny docker so I thought that might be doing it, but I stopped that docker and that did not stop the SMART reads.

 

Apr  7 23:06:27 Tower  emhttpd: read SMART /dev/sdk
Apr  7 23:06:27 Tower  emhttpd: read SMART /dev/sdh
Apr  7 23:06:27 Tower  emhttpd: read SMART /dev/sde
Apr  7 23:06:27 Tower  emhttpd: read SMART /dev/sdf
Apr  7 23:06:27 Tower  emhttpd: read SMART /dev/sdc
Apr  7 23:06:38 Tower  emhttpd: read SMART /dev/sdg
Apr  7 23:06:38 Tower  emhttpd: read SMART /dev/sdl
Apr  7 23:07:31 Tower  emhttpd: read SMART /dev/sdj
Apr  7 23:07:31 Tower  emhttpd: read SMART /dev/sdd
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdj
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdk
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdh
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdd
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sde
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdf
Apr  7 23:22:32 Tower  emhttpd: spinning down /dev/sdc
Apr  7 23:23:02 Tower  emhttpd: spinning down /dev/sdg
Apr  7 23:23:02 Tower  emhttpd: spinning down /dev/sdl

Has anyone noticed this or it just my system?  Any ideas what is causing this and how to stop it?

 

As always, thanks for any help.

 

tower-diagnostics-20230427-1538.zip

[CaptainTivo]

Correction: my spindown time is 15 minutes.  This makes more sense when you read the log.  Sorry about that 🙂

[JonathanM]

When the system detects the drive has been spun up, it triggers a SMART read. So you need to figure out what's accessing the disks, the SMART read is a result of the spinup, not the cause.

 

Try safe mode to rule out plugins, disable dockers and VM's, audit network devices to rule out periodic share accesses. Obviously the nuclear approach of turning everything off and reenabling one set of things at a time until the problem occurs is the best method, but that might not be desirable, so you may have to just work on bits at a time.

 

The open files plugin may be helpful, but sometimes the accesses are so brief it's hard to catch that way.

[apandey]

It would be nice if unraid has an option to log the actual cause of drive spin up. I am assuming there is an explicit decision to spin up the drive at some point due to some requested IO, and given how frequently this question comes up, it would be good if there is some in-built way to troubleshoot this rather than indirect means. I am not sure what is feasible, but a better tool will probably be immensely useful

 

Right now, no indirect measurement is sufficient by itself to pin down the cause. It can be an already open file, or access to a not-open file, or directory listing, or some filesystem operation etc. And we need separate tools to observe each of these in isolation

[itimpi]

4 hours ago, apandey said:

It would be nice if unraid has an option to log the actual cause of drive spin up. I am assuming there is an explicit decision to spin up the drive at some point due to some requested IO, and given how frequently this question comes up, it would be good if there is some in-built way to troubleshoot this rather than indirect means. I am not sure what is feasible, but a better tool will probably be immensely useful

 

Right now, no indirect measurement is sufficient by itself to pin down the cause. It can be an already open file, or access to a not-open file, or directory listing, or some filesystem operation etc. And we need separate tools to observe each of these in isolation

I do not think that Unraid spins up the drive explicitly - it merely issues an I/O and the controller then spins up the drive.   I agree it would be nice to have more information, but I am sure if it was easy to provide it would have happened long before now.

[CaptainTivo]

After some investigation, I found the cause.  I had specified a network share on the server as the target for a Windows backup, including 

"Back up using File History" and I specified "Every hour" as the frequency.  The target is a share on the server which I 

I defined a Share on the server with "Included disks" set to "Disk1"  and "Excluded disks" set to empty (no entry).

My understanding is that you either use "Included disks" OR "Excluded disks" to define your share, not both.

So why did Unraid to spin up ALL the disks in the server if the share is only on one disk?  If I additionally excluded the other disks in the server would it then not spin them up?

 

Anyway, I will mark this SOLVED and ask the other question in a new thread.

Thanks.

[Kilrah]

16 minutes ago, CaptainTivo said:

So why did Unraid to spin up ALL the disks in the server if the share is only on one disk?

If you have enabled reconstruct writes in disk settings than any write to any disk will spin them all up. 

[CaptainTivo]

Thank you!  It had been so many years since I enabled reconstruct writes that I had forgotten the "side effect".

Mystery solved!

[Kilrah]

The Auto Turbo Write Mode plugin can toggle it automatically for you based on how many drives are already spinning, pretty convenient.