[Plugin] Spin Down SAS Drives


doron

Recommended Posts

14 minutes ago, decibelio99 said:

I installed a seagate constellation ES.2 drive and it didn't stop. I have 2 sas drives and one ata on a controller thread and they work fine, but this one that is seperated on another channel (thread) doesn't.

Can you specify the exact model of the drive, and the models of the two controllers - the one that "does the right thing" and the one that does not?

Link to comment
The controller is the same LSI 9211-8i
Ah I missed the fact they are all tied to a single controller. 
In that case we must conclude that the specific drive (ST33000650SS) does not interpret the spin down commands as expected. Unfortunately there is no "standard behavior" re this. Newer Seagate drives tend to do the right thing, quite a few older ones do not.
 
Your drive seems to have been pulled from an Adaptec system, which may or may not have to do with this issue.
 
There's a slight chance that updating the drive's firmware will help, but I can't guarantee it will (or that you won't end up with a bricked drive), so weigh your risk. If you try this, you may want to check out this. I have not tried it, and it seems to be specifically not intended for Adaptec OEM drives, but who knows.

  • Thanks 1
Link to comment
8 hours ago, doron said:

Ah I missed the fact they are all tied to a single controller. 

In that case we must conclude that the specific drive (ST33000650SS) does not interpret the spin down commands as expected. Unfortunately there is no "standard behavior" re this. Newer Seagate drives tend to do the right thing, quite a few older ones do not.

 

Your drive seems to have been pulled from an Adaptec system, which may or may not have to do with this issue.

 

There's a slight chance that updating the drive's firmware will help, but I can't guarantee it will (or that you won't end up with a bricked drive), so weigh your risk. If you try this, you may want to check out this. I have not tried it, and it seems to be specifically not intended from Adaptec OEM drives, but who knows.

Thanks a lot for the response.
I paid 20€ for this drive, money lost.😀
I will try update the firmware.

Link to comment
On 3/8/2022 at 9:23 AM, doron said:


 

 


If you do, please report your results here - it may help other users in the future.

Sent from my tracking device using Tapatalk
 

 

Hi,

I finally decided to upgrade the hard drive because some days it worked and others it didn't.
I have put the firmware of the link on 0005 but it does not work either.
any advice?

Captura de pantalla 2022-03-17 a las 11.28.39.png

Link to comment
Hi,
I finally decided to upgrade the hard drive because some days it worked and others it didn't.
I have put the firmware of the link on 0005 but it does not work either.
any advice?
348457050_Capturadepantalla2022-03-17alas11_28_39.thumb.png.6346f3c005aaf07561888247b2a60225.png
Unfortunately there's not much more I can suggest. Many Seagate drives have demonstrated erratic results wrt standby/spindown. I don't know of a reliable way around it.

Sent from my tracking device using Tapatalk

Link to comment
On 8/13/2021 at 2:26 PM, mike2246 said:

I have 8 of these 6TB SAS Drives ST6000NM0285 - none spin down with 6.9.2, all of my other SATA Drives (mixed array) spin down normally. anything I can do to try or just wait for 6.9.3?

I also have this drive and it will not spin down in 6.9.2 on spindown .86.  I have not added it to the array so i am attempting to spin down from unassigned devices.  Does that make a difference? here is sys log from time

 

Mar 24 18:01:47 lillis SAS Assist v0.86: Spinning down device /dev/sde
Mar 24 18:02:05 lillis emhttpd: read SMART /dev/sde
Mar 24 18:12:55 lillis emhttpd: spinning down /dev/sde
Mar 24 18:12:55 lillis SAS Assist v0.86: Spinning down device /dev/sde
Mar 24 18:13:13 lillis emhttpd: read SMART /dev/sde

 

None of my sas disks were spinning down until i installed .86 plugin in 6.9.2.  Now they all do but this one.  Again, its not in the array yet if that matters.

Edited by Timbiotic
Link to comment
On 3/17/2022 at 11:14 AM, doron said:

Unfortunately there's not much more I can suggest. Many Seagate drives have demonstrated erratic results wrt standby/spindown. I don't know of a reliable way around it.

Sent from my tracking device using Tapatalk
 

Hi doron,

 

I have been working on an issue with dlandon and have noticed that the version of smartctl has the bug which spins up Seagate devices, so this may be the cause of some issues you are seeing with Seagate spinners but may be un related.

 

smartctl r5155 is current installed version and fix for SAS was in r5179.

 

Regards

Simon.

Link to comment
Hi doron,
 
I have been working on an issue with dlandon and have noticed that the version of smartctl has the bug which spins up Seagate devices, so this may be the cause of some issues you are seeing with Seagate spinners but may be un related.
 
smartctl r5155 is current installed version and fix for SAS was in r5179.
 
Regards
Simon.
Thanks @SimonF , that's interesting. It may well be related to some of the cases.

I believe the plugin code already avoids smartctl for some of the functions due to something similar, I'll check when I get home.

It would be good if the patched smartctl finds itself in the next RC.

Sent from my tracking device using Tapatalk

Link to comment

Hi all,

 

I am trying to get my power consumption down as my server has 19 SAS disks permanently spinning. So, I tried using the SAS Spindown App. I tried once before with no success - so I have now upgraded to 6.9.2 and uninstalled/reinstalled the app.

 

I have just added another 1.2Tb SAS disk to my array, so thought before it has data on it I would set it to spindown after 15 minutes and see if it did spindown. It didnt. So, I clicked the green ball and manually set it to spindown and immediately it showed the disk in error and said content was being emulated. I couldnt spin it back up. The only way I could get it back into the array was by removing it from the array, restarting the array... stopping the array, re-introducing it and restarting the array again and letting a parity rebuild complete (well running right now).

 

Can anyone offer any advice on why it errors the disk - obviously I am worried in case by having this installed it tries to spin disks down and then shows them as failed :(

Link to comment
On 4/4/2022 at 6:13 AM, givmedew said:

Hi, I have a bunch of Seagate 3TB and 4TB drives and I was wondering if this Plug-In makes use of Seagate PowerChoice modes.  If it doesn’t is there a benefit to using those modes and could it be updated to use them?  

Interesting. I was not aware of this Seagate nomenclature. I've peeked at it now; it seems to be dealing with the power states this plugin is also attempting to deal with.

 

Seagate provides a set of tools of its own to interact with these modes. Specifically, to manipulate PowerChoice modes one needs to use a tool called SeaChest_PowerControl. While I did download and play with this tool, I do not have a single Seagate drive here so what's coming is mostly conjecture. 

 

If I had to guess, I'd say that this tool basically issues SCSI commands similar to those issued natively by the plugin. In other words, I'd guess it is a different wrapping for the same basic set of commands.

 

In essence, the problem we keep bumping our head into, mainly with some of the Seagate drives, is that we do manage to spin them down into STANDBY_Z mode; however Unraid i/o logic does not expect to have to explicitly "wake" the drive back up. It expects the drive to spin up automagically next time it gets an active i/o issued at it. And it seems that some drives (again, we've seen that mainly with Seagates but not only - the OEM world is complex as it is so no brand wars please), when they get read or write while spun down, they will return an i/o error (ASC/ASCQ 0x4/0x11 resp.) instead of spinning up. They expect an explicit "move back to ACTIVE" command. That in turn makes Unraid red-x them and boom.

 

If anyone has time and some non-array SG drives to work with me on this, as remote hands, I may be able to allocate some time for this. LMK. But the odds of this bet are slim I think - since it is mainly about whether the drive will auto-spin-up, not whether it will spin down successfully.

Link to comment
8 hours ago, SliMat said:

Can anyone offer any advice on why it errors the disk - obviously I am worried in case by having this installed it tries to spin disks down and then shows them as failed :(

What you're seeing is probably quite similar to what others are seeing with some brands of drives/controllers. See quite a few posts in this thread (including my post from a minute ago, just above).

If you provide some more info on the failure (drive and controller brand/model, syslog lines at the time of the failure) I'd be able to say more about whether this is a similar mishap (intervention required, drive in standby, ASC/ASCQ 0x4/0x11). Or post diagnostics. But I'm assuming it's the same. 

Link to comment
11 minutes ago, doron said:

If anyone has time and some non-array SG drives to work with me on this, as remote hands, I may be able to allocate some time for this. LMK. But the odds of this bet are slim I think - since it is mainly about whether the drive will auto-spin-up, not whether it will spin down successfully.

 

Hi @doron

 

I have a spare Proliant ML310e Gen8 with P420i/1Gb controller and probably have some 2.5" SAS disks - I need to look and see what manufacturer they actually are as they will be branded HP... But I was also considering buying a couple of 2Tb Seagate ST2000NX0273 Exos SAS disks (same as I have in my array on my production server) - if you want me to setup a non-production machine for you to access let me know and I will buy the Seagate disks, which I will use in my server when finished with :-)

Link to comment
3 minutes ago, SliMat said:

I have a spare Proliant ML310e Gen8 with P420i/1Gb controller and probably have some 2.5" SAS disks - I need to look and see what manufacturer they actually are as they will be branded HP... But I was also considering buying a couple of 2Tb Seagate ST2000NX0273 Exos SAS disks (same as I have in my array on my production server) - if you want me to setup a non-production machine for you to access let me know and I will buy the Seagate disks, which I will use in my server when finished with 🙂

Sure, we can do that. Let's validate first - do we know that these drives do exhibit that behavior? I mean, you did report a 1.2 drive  (I presume HP branded) getting this problem. Not all SG drives show this - some behave perfectly. If memory serves, for the most part we've seen this happen on Constellation ES.3 series drives (these are 3.5"). Not sure about these newer Exos series.

Link to comment
26 minutes ago, doron said:

Interesting. I was not aware of this Seagate nomenclature. I've peeked at it now; it seems to be dealing with the power states this plugin is also attempting to deal with.

 

Seagate provides a set of tools of its own to interact with these modes. Specifically, to manipulate PowerChoice modes one needs to use a tool called SeaChest_PowerControl. While I did download and play with this tool, I do not have a single Seagate drive here so what's coming is mostly conjecture. 

 

If I had to guess, I'd say that this tool basically issues SCSI commands similar to those issued natively by the plugin. In other words, I'd guess it is a different wrapping for the same basic set of commands.

 

In essence, the problem we keep bumping our head into, mainly with some of the Seagate drives, is that we do manage to spin them down into STANDBY_Z mode; however Unraid i/o logic does not expect to have to explicitly "wake" the drive back up. It expects the drive to spin up automagically next time it gets an active i/o issued at it. And it seems that some drives (again, we've seen that mainly with Seagates but not only - the OEM world is complex as it is so no brand wars please), when they get read or write while spun down, they will return an i/o error (ASC/ASCQ 0x4/0x11 resp.) instead of spinning up. They expect an explicit "move back to ACTIVE" command. That in turn makes Unraid red-x them and boom.

 

If anyone has time and some non-array SG drives to work with me on this, as remote hands, I may be able to allocate some time for this. LMK. But the odds of this bet are slim I think - since it is mainly about whether the drive will auto-spin-up, not whether it will spin down successfully.

I used  these options to support spin downs before we found the process to set power mode so my drives would spin down but unraid thought they where still active. I did use this doc as part of the looking for options.

 

So had to disable the poller. Below are the commands are results. It uses timers to move from one state to another. SZCT is length and STANDBY_Z=1 enables.

 

Sense shows standby by timer instead of standby by command use the plugin method.

 

sense: Standby condition activated by timer

 

I have one seagate SAS drive but my testing below was on HGST + HITACHI as didnt have the Seagate.

 

What test would you think of doing?

 

 

sdparm --flexible -6 -v -S -p po --set=STANDBY_Z=1 --set SZCT=5000 /dev/sdd


sdparm --flexible -6 -l --set SCT=18000 /dev/sdX
sdparm --flexible -6 -l --set STANDBY=1 /dev/sdX




sdparm --flexible -6 -l --set SZCT=18000 --set STANDBY_Z=1 /dev/sdd

sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sdc
sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sdd
sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sde
sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sdf
sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sdg
sdparm --flexible -6 -l -S --set SZCT=9000  /dev/sdi

sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sdc
sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sdd
sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sde
sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sdf
sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sdg
sdparm --flexible -6 -l -S --set STANDBY_Z=1 /dev/sdi

sdparm --clear=EN_BMS --save /dev/sdc
	
sdparm --flexible -6 -v -S -p po  /dev/sdg --set=STANDBY_Y=1
	

root@Tower:~# sg_requests -v --hex -s /dev/sdg
 00     70 00 00 00 00 00 00 18  00 00 00 00 00 00 00 00
 10     00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
root@Tower:~#  sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby condition activated by command
root@Tower:~# sg_requests -v --hex -s /dev/sdd
 00     70 00 00 00 00 00 00 18  00 00 00 00 5e 04 00 00
 10     00 00 00 00 18 38 00 00  00 00 00 00 00 00 00 00
root@Tower:~# 


root@Tower:~# sdparm --command=sense /dev/sdi
    /dev/sdi: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby_y condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdg
    /dev/sdg: HGST      HUS724030ALS640   A1C4
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdi
    /dev/sdi: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby_y condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sde
    /dev/sde: HGST      HUS724030ALS640   A1C4
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sde
    /dev/sde: HGST      HUS724030ALS640   A1C4
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdg
    /dev/sdg: HGST      HUS724030ALS640   A1C4
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Idle condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdd
    /dev/sdd: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdi
    /dev/sdi: HITACHI   HMRSK2000GBAS07K  3P02
Additional sense: Standby condition activated by timer
root@Tower:~# sdparm --command=sense /dev/sdg
    /dev/sdg: HGST      HUS724030ALS640   A1C4
Additional sense: Standby condition activated by timer
root@Tower:~# 

 

Link to comment

Hi @SimonF, thanks for this. Yes, we played quite extensively with the SCSI options etc. What was new to me was the Seagate toolset. So,

 

31 minutes ago, SimonF said:

What test would you think of doing?

 

Basically, things like 

SeaChest_PowerControl -d /dev/sgX --spinDown
SeaChest_PowerControl -d /dev/sgX --transitionPower standby_z
SeaChest_PowerControl -d /dev/sgX --transitionPower sleep

With each of which cases - 

(a) Test that the drive indeed spins down (sense)

(b) Issue a direct i/o - read, write - against it, see if it spins up or produces 0x4/0x11.

 

Something like that.

 

Again, my base hypothesis is that the behavior will remain the same - require an explicit "move-to-active" command and returning 4/11 without it - but, who knows, we may be surprised.

 

Edit:

./SeaChest_PowerControl -d /dev/sg8 --checkPowerMode
==========================================================================================
 SeaChest_PowerControl - Seagate drive utilities - NVMe Enabled
 Copyright (c) 2014-2021 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
 SeaChest_PowerControl Version: 3.0.2-2_2_3 X86_64
 Build Date: Jun 17 2021
 Today: Tue Apr  5 23:11:16 2022        User: root
==========================================================================================

/dev/sg8 - HUH721212AL4200 -  - SCSI
Device is in the Standby_Z state activated by host command

 

Edited by doron
Link to comment
31 minutes ago, doron said:

Sure, we can do that. Let's validate first - do we know that these drives do exhibit that behavior? I mean, you did report a 1.2 drive  (I presume HP branded) getting this problem. Not all SG drives show this - some behave perfectly. If memory serves, for the most part we've seen this happen on Constellation ES.3 series drives (these are 3.5"). Not sure about these newer Exos series.

@doron My production server which showed this problem with an HP 1.2Tb SAS disk is in a datacentre, so I can only check what the disk is from the ILO and this, I'm pretty sure, will only show the HP details and not the actual manufacturers details. So, I will try to get remote hands at the DC to physically pull out the drive which I had the issue with today and see if I have an identical one here which I can put in a sare server for you to play with. I have just logged on to the ILO of the machine in the DC and it shows as a 1.2Tb 12G 10k SAS disk... but I cant see if its WD/Seagate/Hitachi/Toshiba etc.

 

I will set up a trial version of UnRAID and as this machine has advanced ILO you should be able to get on to the machine to do whatever you need.

Link to comment
3 minutes ago, doron said:

Sounds good. Thanks for your help!

Welcome - I'll try to get this setup and running later this week.... or at the worst case early next week. Once its up and running I'll let you know. Do you need any array disks - or shall I just give you a clean UnRAID setup so you can set up what you want? Also - how many disks do you need in all?

Link to comment

@doron I have had a look at the disks I have locally to put in this server and I have HP/Seagate SAS disks in 1Tb, 500Gb and 146Gb flavours. So I will build a test rig with an assortment of disks and hopefully these will exibit the same problems. I will PM you with login details once I have it arranged ;-)

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.