[Plugin] Spin Down SAS Drives


doron

Recommended Posts

Hi @doron

 

OK, the server is nearly ready and I also bought a couple of 2Tb Seagate SAS disks to inclued in the setup for you to test. Do you need the server set to HBA mode, or RAID mode? I usually set HP servers to HBA mode so the disks are visible directly from the O/S, in RAID mode you need each disk mounted individually as RAID 0 in order for UnRAID to see them.

 

Let me know and I should get this online for you in the next 24 hours - if you PM me when you want to connect I can set an open port to ILO for you and I will NAT the remote access ports so you can get to the UnRAID interface too ;-)

Link to comment
7 hours ago, SliMat said:

OK, the server is built and running... I have put 3 x 1Tb SAS disks, 2 x 2Tb SAS disks and a 146Gb SAS disk in (all Seagate) for you to take a look at. PM me when you are ready and I'll send you login details.

Wow, thank you for setting all this up! Greatly appreciated.

(Continuing via pm.)

Link to comment
  • 2 weeks later...
On 4/12/2022 at 12:14 AM, SliMat said:

Very welcome - glad to help if I can :) - Have sent a PM

Thanks again @SliMatfor setting up the testing lab! It is very much appreciated.

Unfortunately the results did not lead to a breakthrough - the controller / drives setup absolutely refused any attempt to put the drives into a "standby_z" (spindown) state. 

Net-net, I added the HP SAS controller to the exclusion list of the plugin.

 

Thanks!

Link to comment

The plugin spin down the SAS drive but after that i get a SMART read which spin it back up:

Apr 27 23:06:15 Tower-Oron emhttpd: spinning down /dev/sdb
Apr 27 23:06:16 Tower-Oron SAS Assist v0.86: Spinning down device /dev/sdb
Apr 27 23:07:23 Tower-Oron emhttpd: read SMART /dev/sdb

What am i missing? 

Unraid 6.9.2, HGST SAS drives and LSI SAS2008 HBA

Link to comment
56 minutes ago, orons said:

The plugin spin down the SAS drive but after that i get a SMART read which spin it back up:

Apr 27 23:06:15 Tower-Oron emhttpd: spinning down /dev/sdb
Apr 27 23:06:16 Tower-Oron SAS Assist v0.86: Spinning down device /dev/sdb
Apr 27 23:07:23 Tower-Oron emhttpd: read SMART /dev/sdb

What am i missing? 

Unraid 6.9.2, HGST SAS drives and LSI SAS2008 HBA

In the case above, it seems that the spin-up (read SMART) happens more than a full minute after the spin down. This might be a result of normal I/O (e.g. read) against the array.

Link to comment
8 hours ago, doron said:

In the case above, it seems that the spin-up (read SMART) happens more than a full minute after the spin down. This might be a result of normal I/O (e.g. read) against the array.

Hi Doron (from Israel?), it's only the SAS drives, SATA drives stays spun down. Also cleared statistic and in some of them there was no traffic and it's still up.

It's not the dockers (only cache drive and disabled it for testing), will try a reboot later today. 

Screen Shot 2022-04-28 at 8.51.44.png

Link to comment
1 hour ago, orons said:

it's only the SAS drives, SATA drives stays spun down.

  • Is the time span between spindown and "read SMART" always the same (about 1 min)?
  • Does the same thing happen both when you let the drive spin down "naturally" (i.e. following the Unraid spin down delay) and when you force spin down from the GUI?
  • Could you perhaps have some folder that lives mainly on these drives and sees frequent I/O?
  • Have you tried to upgrade to some 6.10 RC?

BTW all my SAS drives are HGST (HUH721212AL4200) with an (onboard) LSI controller and they remain spun down nicely.

Link to comment
17 minutes ago, Fuggin said:

I have noticed this using the RC versions. Otherwise SAS Spin Down works fine. Using all SAS drives connected through a SAS3008 HBA and Supermicro 836 backplane.

 

 

image.png

(sorry for the deleted post)

I just pushed out a newer version of the plugin (also with a more a-la-mode version numbering scheme). Should fix this issue - let me know if it does.

Link to comment
2 minutes ago, doron said:

Woah. That Should Not Occur™

Where and when exactly do you see it? One time or regularly? During boot or during normal operation? Please elaborate.

Towards the end of boot sequence. I don't get any other errors otherwise.

Link to comment
9 minutes ago, doron said:

(sorry for the deleted post)

I just pushed out a newer version of the plugin (also with a more a-la-mode version numbering scheme). Should fix this issue - let me know if it does.

That did the trick! Thanks for the quick fix!

Link to comment
6 hours ago, doron said:
  • Is the time span between spindown and "read SMART" always the same (about 1 min)?
  • Does the same thing happen both when you let the drive spin down "naturally" (i.e. following the Unraid spin down delay) and when you force spin down from the GUI?
  • Could you perhaps have some folder that lives mainly on these drives and sees frequent I/O?
  • Have you tried to upgrade to some 6.10 RC?

BTW all my SAS drives are HGST (HUH721212AL4200) with an (onboard) LSI controller and they remain spun down nicely.

  • Between 20 seconds to a minutes
  • Yes, happens both manually and after Unraid spin down delay. 
  • IT's movies so don't see a reason for frequent I/O...
  • Not yet

Reboot didn't solved it as well, don't think it's the plugin, but you have experience supporting people so thought you might seen this behavior before. 

Link to comment
13 minutes ago, orons said:
  • Between 20 seconds to a minutes
  • Yes, happens both manually and after Unraid spin down delay. 
  • IT's movies so don't see a reason for frequent I/O...
  • Not yet

Reboot didn't solved it as well, don't think it's the plugin, but you have experience supporting people so thought you might seen this behavior before. 

The moment i sent the replay i figured it out, it was the autofan plugin, for testing reason i configured it check temp every minute and for some reason it need to spin up the drives to check the temp in SAS, i excluded the SAS drives from the plugin and now it's working :) Thanks! 

Link to comment
11 minutes ago, orons said:

The moment i sent the replay i figured it out, it was the autofan plugin, for testing reason i configured it check temp every minute and for some reason it need to spin up the drives to check the temp in SAS, i excluded the SAS drives from the plugin and now it's working :) Thanks! 

That may be related to the "smartctl -n" issue spinning up drives. You may want to check 6.10-rc5, which per @SimonF includes an updated version of smartctl which has this issue solved (haven't seen it in the release notes btw).

Link to comment
3 hours ago, doron said:

That may be related to the "smartctl -n" issue spinning up drives. You may want to check 6.10-rc5, which per @SimonF includes an updated version of smartctl which has this issue solved (haven't seen it in the release notes btw).

Not sure why it is not in the release notes

 

root@computenode:~# uname -a
root@computenode:~# cat /etc/unraid-version
version="6.10.0-rc5"
root@computenode:~# uname -a
Linux computenode 5.15.35-Unraid #1 SMP Thu Apr 21 13:44:00 PDT 2022 x86_64 12th Gen Intel(R) Core(TM) i5-12600K GenuineIntel GNU/Linux
root@computenode:~# smartctl -V
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.15.35-Unraid] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

smartctl comes with ABSOLUTELY NO WARRANTY. This is free
software, and you are welcome to redistribute it under
the terms of the GNU General Public License; either
version 2, or (at your option) any later version.
See https://www.gnu.org for further details.

smartmontools release 7.3 dated 2022-02-28 at 16:33:40 UTC
smartmontools SVN rev 5338 dated 2022-02-28 at 16:34:26
smartmontools build host: x86_64-slackware-linux-gnu
smartmontools build with: C++11, GCC 11.2.0
smartmontools configure arguments: '--prefix=/usr' '--libdir=/usr/lib64' '--mandir=/usr/man' '--sysconfdir=/etc' '--docdir=/usr/doc/smartmontools-7.3' '--build=x86_64-slackware-linux' 'build_alias=x86_64-slackware-linux' 'PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/local/share/pkgconfig:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
root@computenode:~# 

 

Link to comment
3 hours ago, orons said:

The moment i sent the replay i figured it out, it was the autofan plugin, for testing reason i configured it check temp every minute and for some reason it need to spin up the drives to check the temp in SAS, i excluded the SAS drives from the plugin and now it's working :) Thanks! 

Spinup issue that is fixed in 7.3 of Smartctl was for Seagate drives as it was reading VPD before doing standby check.

Link to comment
  • 4 weeks later...
27 minutes ago, another_hoarder said:

@doron Hi - what's the correct way of editing this plugin to enable debug output in syslog?  I'd like to see what parent process is still driving spin ups / smart reads on my stubborn box (9.2).  Cheers.

Enabling debug on the plugin will not generate very much output on syslog, if at all (umm, I guess I need to change that).

Note however that if you are looking for debug info on spin up,  you'd probably not have found it here anyway - the spin ups / SMART reads occur outside of the plugin and unrelated to it.

 

If you want to explore debugging other parts of the system, you might want to wrap smartctl with some debug code, to track calls to it. Let me know if you want to explore that path, I can provide you with a neat little wrapper for that.

Link to comment
21 minutes ago, doron said:

Enabling debug on the plugin will not generate very much output on syslog, if at all (umm, I guess I need to change that).

Note however that if you are looking for debug info on spin up,  you'd probably not have found it here anyway - the spin ups / SMART reads occur outside of the plugin and unrelated to it.

 

If you want to explore debugging other parts of the system, you might want to wrap smartctl with some debug code, to track calls to it. Let me know if you want to explore that path, I can provide you with a neat little wrapper for that.


Ah, interesting.  I was guided here through this post which specifically referenced your script as potentially having the side super power of being able to point out the parent process that calls smartctl as part of your wrapper's functionality.  Unfortunately, I lack the skill to write a wrapper myself so was hoping to piggyback off someone else's abilities, even if it's a byproduct of their primary function.  

But if that other 'neat little wrapper' could be of some help in tracking my unending investigation into random spin up / read SMART / spin down sequences, I'd certainly be indebted.  And if not, no worries, I very much appreciate your reply in first place.
 

May 24 15:55:38 Tower emhttpd: read SMART /dev/sde
May 24 15:58:04 Tower emhttpd: read SMART /dev/sdj
May 24 15:58:16 Tower emhttpd: read SMART /dev/sdm
May 24 16:27:36 Tower emhttpd: spinning down /dev/sde
May 24 16:28:09 Tower emhttpd: read SMART /dev/sdh
May 24 16:33:05 Tower emhttpd: read SMART /dev/sdl
May 24 16:34:02 Tower emhttpd: read SMART /dev/sdk
May 24 16:35:41 Tower emhttpd: spinning down /dev/sdm
May 24 16:35:41 Tower emhttpd: spinning down /dev/sdj
May 24 16:49:39 Tower emhttpd: read SMART /dev/sde
May 24 16:57:46 Tower emhttpd: spinning down /dev/sdh

 

Link to comment
Posted (edited)

Here you go @another_hoarder.

To use it, put it anywhere convenient, align permissions (chmod +x generic.wrapper) and then use thusly:

generic.wrapper <CommandToBeWrapped>

In your case, the command to be wrapped would be smartctl.

To remove the wrapper, run the same command again.

 

No warranty, your own risk etc. 

generic.wrapper

Edited by doron
Link to comment

Do any of you use the HD location bash script that blinks the hd light to find the drive?  Since adding a expander case, some of my drive devices are 4 character long instead of three  ex sdg vs sdg1 and it no longer sees the 4 letter variants, any suggestions?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.