doron

A keyfile could in fact be anything - just any file. Some people generate 32 random bytes from /dev/random, some use existing files. Just make sure you hold on to it 🙂

Sure. Once you have your keyfile generated (and backed up in a few places 🙂 ), place it somewhere accessible and then: unraid-newenckey - <keyfilepath> The dash indicates "prompt for key from console", which is what it will do.

Hmm. Certainly a possibility, yes. Currently it's second on my list of potential causes, only because the OEM manual of the Seagate described the drive's behavior in the Standby_Z state (this is what we're using to spin the drive down), and it explicitly stated that the drive will require an init op to spin up again. This is in contrast with the HGST/WD SAS drives, which (explicitly) specify that in the same state the drive is ready to spin back up on next I/O. The manual for your Hitachi is silent about this - does not say either - but the 04-11 sense makes me think it does it like the Seagate. You'd think that standards would be standards, and the drive behavior following a standard SAS/SCSI operation would be standard... go figure. It could still be the controller, the above is just my current thinking.

Thanks very much for this. Aha. That's the infamous/dreaded 02-04-11, same as we've seen on the Seagate. Will have to exclude this series as well 😞 A different drive? Was something else happening at the same time?

Thanks. So basically you're saying the result in your case is not consistent? Some (most?) of the time they do spin back up but at times they get the read errors? Could you paste syslog lines from the time of such error that red-x-ed a drive?

This manual for this drive (in fact, the entire Constellation ES.3 series) seems to indicate (sec 6.1) that an explicit NOTIFY needs to be sent to the device to recover from the spindown mode we're sending the device into (Standby_Z). This is in contrast with other devices tested (e.g. WD/HGST) that automatically spin up when sent to this state. Has anyone seen positive results with this drive and this plugin? We might end up having to enumerate the drive types where this works well vs. those that fail, and build a white list in the plugin. Nasty 😞 Can I get brief messages here from anyone who's using (or tried using) the plugin, reporting success/failure? Just a one liner with: <HDD Model> <Success/Failure> (<optional comment>) would be great. Example; HUH721212AL4200 Success PM would also work if you don't want to post. Thanks!

Indeed BUT @dansonamission if you try that, please do it only with the array stopped, to make sure you don't get a second red x. That would be really bad.

Ouch. That's not good. 1. Sorry! The command code being sent to the drive has been tested with multiple users and drives, up to now with no issue, let alone a red x. I guess one should always expect the unexpected. Please remove the plugin for now (I assume you already have). 2. Bottom line first - the best way to get the drive back into the array is to rebuild it from parity. There are multiple posts and guides as to how to do that (basically need to remove it from the array and reinsert it, but need to be precise, so please use one of the step-by-step guides). 3. I've had a red x happen during the initial tests (long before the plugin was written), when we sent a different op code to the drive - basically a STOP vs. a STANDBY. It might be that your particular drive/controller interpret the STANDBY op code differently (as STOP). 4. Can you post the drive model, and, if possible, the SAS controller model? Others - I'd probably hold off installing the plugin until we get to the bottom of this. Nobody wants a red x. This (sense 0x2 ASC 0x4) translates to the LU is spun down and needs an INIT command. This is usually the result of a STOP command, and "should not" be the result of a STANDBY command. This could match my assumption in #3 above - that for some reason the drive/controller took the STANDBY to mean STOP. @SimonF, any further insights?

Can you run, against a SAS drive that has just (allegedly..) spun down, these command (in this order) and post the result? sdparm --command=sense /dev/sdX smartctl -i /dev/sdX Should not cause an issue (many people, myself included, have a mix). The code checks for that and intervenes only if the drive is detected as SAS. The output from the two commands above will hopefully tell us more. Appreciate your help.

Thanks. The output file seems to indicate all's in order. Ah. Only remote - no local instance? If you do have a /var/log/syslog - can you look at that one?

That's not what should happen. Seems like the syslog filter is not triggered, for some reason. Can you run the following for me on your console: cat /etc/rsyslog.d/99-spinsasdown.conf &> /tmp/sas-dbg /etc/rc.d/rc.rsyslogd restart &>> /tmp/sas-dbg rsyslogd -dn &>> /tmp/sas-dbg and post (or pm) the resulting file /tmp/sas-dbg ? Mine are HUH721212AL4200 - close enough 🙂 Thank you!

Can you describe what you're doing, exactly? Are you pushing the green button (to initiate spin down) repeatedly? Could it be that your /dev/sdd is busy doing active i/o (so it keeps spinning back up immediately)? You can check that if you look at the "Reads" and "Writes" columns in the main panel - if they are changing, the disk is doing i/o so forcing it to spin down will not help. Also, can you share the disk and controller model. (Note, this plugin has not yet been tested in Unraid 6.9)

The plugin makes sure the SAS drive indeed spins down. It does not eliminate the error messages from mdcmd, as explained above. You can go to Settings -> Syslog Server and configure Local syslog rotation, to keep the log file from filling up.

Indeed. The next version (to be uploaded Soon) fixes that. Great. Please report any issues.

If you are referring to the messages that begin with "SAS Assist", they are by design - to log the functioning of the plugin. However you have other messages, such as: Oct 3 12:11:49 segator-unraid emhttpd: error: mdcmd, 2721: Input/output error (5): write Oct 3 12:11:49 segator-unraid emhttpd: error: mdcmd, 2721: Input/output error (5): write Oct 3 12:11:49 segator-unraid kernel: md: do_drive_cmd: disk7: ATA_OP e0 ioctl error: -5 Which seem to be unrelated to the plugin, and might indicate a problem. As a first step, I'd suggest you remove the plugin and see if these messages keep showing up (following a "spindown" message). EDIT: Okay seems like these messages have been plaguing some users with SAS drives - I'm seeing threads like this. I have never seen them, it's probably controller dependent. I guess you have been getting these and expected the plugin to make it stop. Because of how this plugin is triggered, the emhttp / mdcmd code runs just before it, and issues the ATA standby ops, so the plugin can't turn these off. We may want to filter these messages out of syslog, but gotta be careful not to filter too much (in other, non-spindown cases these messages might be important and indicate an issue). I'll look into this. At any rate, other than being a nuisance on the log, I believe these messages are harmless.

Hi @dotexe, thanks for reporting an installation issue (here). So far I have not been able to reproduce the issue. Can you run the following command for me, on the Unraid console, and post the results: /etc/rc.d/rc.rsyslogd restart EDIT: Also, please post your Unraid version, and an excerpt from your syslog around the time of the plugin installation.

Unraid does a nice job of controlling HDD's energy consumption (and probably longevity) by spinning down (mechanical) hard drives when idle for a set period of time. Unfortunately the technique used by Unraid to spin down an HDD, true to the time of writing this, works only for ATA drives. If you have SCSI/SAS hard drives, these drives do not spin down (although the UI will indicate they do). The drives continue spinning 24x7, expanding the energy footprint of your Unraid server. Following a long and fruitful discussion here, a solution is provided via this plugin. This is hopefully a temporary stopgap, until Limetech includes this functionality in the mainline Unraid, at which time this plugin will walk into the sunset. Essentially, this plugin complements the Unraid SATA spindown functionality with SAS-specific handling. In version 6.9 and upwards, it enhances the "sdspin" function (focal point for drive spin up/down) with support for SAS drives. In prior versions (up until 6.8.x) it does the following: 1. Install a script that spins down a SAS drive. The script is triggered by the Unraid syslog message reporting this drive's (intended) spin down, and actually spins it down. 2. Install an rsyslog filter that mobilizes the script in #1. 3. Monitor rsyslog configuration for changes, to make sure the filter in #2 above stays put across changes of settings. In addition, the plugin installs a wrapper for "smartctl", which works around smartctl's deficiency (in versions up to 7.1) of not supporting the "-n standby" flag for non-ATA devices (which leads to many unsolicited spin-ups for SAS drives). When this flag is detected, if the target device is SAS and is in standby (i.e. spun down), smartctl is bypassed. You can install this plugin via Community Applications (the recommended way), or by using this URL: https://raw.githubusercontent.com/doron1/unraid-sas-spindown/master/sas-spindown.plg In "Install Plugin" dialog. When you remove the plugin, original "vanilla" Unraid behavior is reinstated. As always, there is absolutely no warranty, use at your own risk. It works for me. With that said, please report any issue (or success stories...) here. Thanks and credit points go to this great community, with special mention to @SimonF and @Cilusse. EDIT: It appears that some combinations of SAS drives / controllers are not compatible with temporary spin-down. We've seen reports specifically re Seagate Constellation ES.3 and Hitachi 10KRPM 600GB but there are probably others. Plugin has been updated to exclude combinations known to misbehave, and to use a dynamic exclusion table so that other combinations can be added from time to time. 19-Nov-2020

and Precisely. To drive SAS drives, you need a SAS controller. This could be on-board (on server boards such as Supermicro's etc.) or a dedicated controller. The connector might be SATA style (e.g. many of SM boards have SAS-capable SATA-style connectors) but the protocol must be SAS. Connected to a SATA controller, a SAS drive will not spin up.

This used to work on older versions of Unraid, but not in recent versions.

Yeah, I'm using the G2. Old, can sometimes be had from ebay, but does have a solid GUID.

You're welcome, @gerard6110. Incidentally, a different way to solve the issue as you're framing it is to use an USB Card Reader instead of a USB stick. Some of the card readers have IDs that can be used by Limetech's licensing scheme. If the flash component (an SD card of sorts - usually a microSD) wears out, you just pull it out, install a new one and restore content from backup. ID remains the same so license remains intact. You can find out more about this here and here.

Sorry to be the bearer of bad news, but -- probably not; and in fact, the entire scheme, elegant as it may seem at first glance (which is what drew me in when I saw this thread, before stopping to think about it properly) cannot actually work. The only reason it kind-of-works for you right now is that your BOOTDISK is a USB flash drive, and not a hard drive. Had it been an HDD it would not have completed a full boot process (same as you saw earlier, one "bzoverlay" ago). Longer explanation: Unraid's OS, by design, places the various HDD controller (and many other) drivers as loadable kernel modules (not compiled in). Those modules are packaged separately (bzmodules) and are not part of the initramfs. Result is that during the early stages of the boot process, HDD's are not accessible - until bzmodules is mounted (and udev gets kicked to rediscover devices). Effectively, this means that if Unraid is booted from an HDD, there'll be a bootstrap deadlock, since bzmodules won't be accessible, hence not mountable, hence HDD can never be seen or mounted. Game over, insert coin. I believe the scheme proposed in this thread has never actually worked. The bzoverlay file that allegedly performs the cool trick has been never successfully unpacked by the kernel (cuz it wasn't properly packed). Therefore, boot worked and things were looking nice - but the truth of the matter was that no change actually took place, the original licensed USB flash was mounted r/w as /boot, and the boot drive was just lying there doing very little. Only after I gave you a properly packed bzoverlay, have we started to see it "functioning" - essentially making things break... Solving this properly would require repackaging of the Unraid OS, which, while perfectly doable, would change the scheme from elegant to terribly messy.

I think what happened is that your settings were saved to the UR drive, even when you booted from the BD drive. What happened during boot is quite clear... The fstab file from the original scheme could never have worked with recent Unraid versions; it misses some key mounts. I haven't set up a test machine yet, so if you're willing to keep playing, you can try the attached for size (again, just place it on the BOOTDISK instead of the previous one(s)). If the boot process completes successfully, please post results of "mount". Note this is a stopgap, not an elegant solution; if it actually moves us forward, I'll try to whip up something more elegant. bzoverlay

I was afraid of that. Seems like bzoverlay never gets opened, hence never used. Subsequently, BOOTDISK is mounted at /mnt/disks/BOOTDISK - this is actually done by UD and not by the original plan (it was supposed to be mounted at /boot, and the one with the license at /license). The original flash drive is mounted as /boot, read/write (so the desired result is not obtained at all). That's sad 🙂 This is probably the first time "bzoverlay" was actually opened and used. What happened? What do you see during boot?

Umm, not sure we're there yet. I'm concerned with this one: This might explain other stuff too. Can you paste the output of the command "mount" (before you try the suggested fix below)? Then, would you care to try the attached bzoverlay I made - just put it on your BOOTDISK instead of the previous one, and test again. Finally, paste the output of "mount" again (after the change). If this fix removes the above error message, you may want to enable the plugin you disabled previously, and see if you get any anomalies. bzoverlay