• [6.8.3] USB HDDs randomly spin up, but status stays unchanged


    mgutt
    • Closed

    I manually spun down all my disks. After some minutes the power consumption raised, so I executed "mdcmd spindown" through the terminal. As disk3 and disk6 reacted slowly to the command and the power consumption dropped after that, I could be sure that they were spinning, although the icon was grey:

     

    1309051213_2020-10-1601_28_15.thumb.png.8e96648f4f4988dbcaeef83ec6ed2e48.png

     

    I really like to help to find the problem, but I do not know what to search for. As it can't be a disk access thing, it must be something related to the USB device power management I think.

     

    P.S. I don't know if they stay spinning all the time or spin down after several minutes again. I will observe that.



    User Feedback

    Recommended Comments

    6 hours ago, trurl said:

    yet another

    I have no problems except of that. Full bandwidth, no disconnects.

    Link to comment
    Share on other sites

    At the moment I try to observe this. This are the things I found out:

     

    Status

     

    USB = sdb = disk3

    SATA = sdg = disk7

     

    USB

    smartctl --nocheck standby -i /dev/sdb
    
    smartctl 7.1 2019-12-30 r5022 [x86_64-linux-4.19.107-Unraid] (local build)
    Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
    
    CHECK POWER MODE: incomplete response, ATA output registers missing
    CHECK POWER MODE not implemented, ignoring -n option
    === START OF INFORMATION SECTION ===
    Model Family:     HGST Ultrastar DC HC520 (He12)
    Device Model:     HGST HUH721212ALE604
    Serial Number:    8CHUDHEE
    LU WWN Device Id: 5 000cca 26fd9a3b2
    Firmware Version: LEGNW3D0
    User Capacity:    12,000,138,625,024 bytes [12.0 TB]
    Sector Sizes:     512 bytes logical, 4096 bytes physical
    Rotation Rate:    7200 rpm
    Form Factor:      3.5 inches
    Device is:        In smartctl database [for details use: -P show]
    ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
    SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
    Local Time is:    Fri Oct 16 21:32:04 2020 CEST
    SMART support is: Available - device has SMART capability.
    SMART support is: Enabled

    SATA

    smartctl --nocheck standby -i /dev/sdg
    smartctl 7.1 2019-12-30 r5022 [x86_64-linux-4.19.107-Unraid] (local build)
    Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
    
    Device is in STANDBY mode, exit(2)

     

    USB

    hdparm -C /dev/sdb
    
    /dev/sdb:
    SG_IO: bad/missing sense data, sb[]:  70 00 01 00 00 00 00 0a 00 00 00 00 00 1d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     drive state is:  unknown

    SATA

    hdparm -C /dev/sdg
    
    /dev/sdg:
     drive state is:  standby

    Both commands do not wake up the disks.

    Monitoring

    - If the USB disks randomly spin up, they spin up both. Its never only one of it.

    - inotifywait does not return any file access

     

    Does Unraid periodically execute a command which checks the HDD status or similar?

     

    Next step is this command to log all processes:

    top -b -c -d 5 > /mnt/cache/top.log

    Sadly "lastcomm" is not available because of "accton: Function not implemented" which should mean the Linux Kernel of Unraid has not enabled accounting.

    Edited by mgutt
    Link to comment
    Share on other sites

    Ok, catched it:

    top - 22:56:25 up 22:11,  2 users,  load average: 0.01, 0.02, 0.00
    Tasks: 257 total,   1 running, 250 sleeping,   0 stopped,   6 zombie
    %Cpu(s):  0.6 us,  0.1 sy,  0.0 ni, 89.4 id,  9.7 wa,  0.0 hi,  0.0 si,  0.0 st
    MiB Mem :  64358.4 total,  20504.6 free,   9912.9 used,  33940.9 buff/cache
    MiB Swap:      0.0 total,      0.0 free,      0.0 used.  53088.3 avail Mem 
    
      PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
     4683 root      20   0  283604   4132   3416 S   0.2   0.0   3:24.90 /usr/local/sbin/emhttpd
     5256 root      20   0  149776   8300   3752 S   0.2   0.0   0:18.49 nginx: worker process
    17144 root      20   0       0      0      0 I   0.2   0.0   0:01.73 [kworker/3:3-events_freezable]
    25275 root      20   0    7464   4676   3620 D   0.2   0.0   0:00.01 /usr/sbin/smartctl -n standby -A /dev/sdb
    25276 root      20   0       0      0      0 Z   0.2   0.0   0:00.01 [smartctl] <defunct>
    25278 root      20   0    7464   4580   3524 D   0.2   0.0   0:00.01 /usr/sbin/smartctl -n standby -A /dev/sda
    25279 root      20   0       0      0      0 Z   0.2   0.0   0:00.01 [smartctl] <defunct>
        1 root      20   0    2468   1684   1576 S   0.0   0.0   0:10.52 init
    

     

    Let's verify it.

     

    Spin down USB... takes long time, so it's spinning down

    mdcmd spindown 6

    Check status of SATA... no delay

    /usr/sbin/smartctl -n standby -A /dev/sdg
    smartctl 7.1 2019-12-30 r5022 [x86_64-linux-4.19.107-Unraid] (local build)
    Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
    
    Device is in STANDBY mode, exit(2)

    Check status of USB... takes very long time, so it's spinning up

    /usr/sbin/smartctl -n standby -A /dev/sda
    smartctl 7.1 2019-12-30 r5022 [x86_64-linux-4.19.107-Unraid] (local build)
    Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
    
    CHECK POWER MODE: incomplete response, ATA output registers missing
    CHECK POWER MODE not implemented, ignoring -n option
    === START OF READ SMART DATA SECTION ===
    SMART Attributes Data Structure revision number: 16
    Vendor Specific SMART Attributes with Thresholds:
    ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
      1 Raw_Read_Error_Rate     0x000b   100   100   016    Pre-fail  Always       -       0
      2 Throughput_Performance  0x0005   132   132   054    Pre-fail  Offline      -       96
      3 Spin_Up_Time            0x0007   163   163   024    Pre-fail  Always       -       416 (Average 389)
      4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       401
      5 Reallocated_Sector_Ct   0x0033   100   100   005    Pre-fail  Always       -       0
      7 Seek_Error_Rate         0x000b   100   100   067    Pre-fail  Always       -       0
      8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
      9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       4336
     10 Spin_Retry_Count        0x0013   100   100   060    Pre-fail  Always       -       0
     12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       59
     22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
    192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       1683
    193 Load_Cycle_Count        0x0012   099   099   000    Old_age   Always       -       1683
    194 Temperature_Celsius     0x0002   187   187   000    Old_age   Always       -       32 (Min/Max 25/55)
    196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
    197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
    198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
    199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age   Always       -       0

     

    Bug Fix

    "smartctl" ignores "-n standby" as the USB device does not support power checks:

    CHECK POWER MODE: incomplete response, ATA output registers missing
    CHECK POWER MODE not implemented, ignoring -n option

    The same message is displayed if the "-i" option is used, but this does not spin up the drive, so I suggest to use this command, before executing "-A" (to obtain the temps/errors etc):

    /usr/sbin/smartctl --nocheck standby -i /dev/sdb

    Sadly this won't solve the active / inactive icon, temps, etc as Unraid seems to rely on the correct answer of the "-A" option.

     

    I was not able to find out if the disk is active or not. I tried to obtain the usb power:

    lsusb -v|egrep "^Bus|MaxPower"

    But it returns all the time "896mA", sleeping or not. Even fully disabling the USB port is not possible anymore, as "suspend" has been removed from recent Kernels:

    https://unix.stackexchange.com/a/166601/101920

     

     

    Edited by mgutt
    Link to comment
    Share on other sites

    Dirty hack

     

    Add this to the Go file:

    # -------------------------------------------------
    # USB HDD sleep bug fix
    # https://forums.unraid.net/bug-reports/stable-releases/683-usb-hdds-randomly-spin-up-but-status-stays-unchanged-r1091/?tab=comments#comment-11087
    # -------------------------------------------------
    sed -i -- 's/by %s -A/by %s -i/g' /usr/local/sbin/emhttpd

    Before that (which loads emhttpd):

     

    # -------------------------------------------------
    # Start the Management Utility
    # -------------------------------------------------
    /usr/local/sbin/emhttp &

     

    Note:

    • This will suppress displaying the temps!
    • And sadly it does not solve the icon issue. I think the active / inactive status is extracted from the "CHECK POWER MODE" status.
    • In very rare cases the USB disks still wake up. I need to investigate this further, but I think the USB adapter itself is the problem
    Edited by mgutt
    Link to comment
    Share on other sites

    I bought two new USB3 adapters from different brands through Amazon. Both use the same "ASM1051E" controller from ASMedia, but the second does not display the correct "iSerial":

    Bus 002 Device 003: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
    Device Descriptor:
      bLength                18
      bDescriptorType         1
      bcdUSB               3.00
      bDeviceClass            0 
      bDeviceSubClass         0 
      bDeviceProtocol         0 
      bMaxPacketSize0         9
      idVendor           0x174c ASMedia Technology Inc.
      idProduct          0x55aa ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
      bcdDevice            1.00
      iManufacturer           2 ASMedia
      iProduct                3 ASM105x
      iSerial                 1 8CHUDHEE            
    ...
    Bus 002 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
    Device Descriptor:
      bLength                18
      bDescriptorType         1
      bcdUSB               3.00
      bDeviceClass            0 
      bDeviceSubClass         0 
      bDeviceProtocol         0 
      bMaxPacketSize0         9
      idVendor           0x174c ASMedia Technology Inc.
      idProduct          0x55aa ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
      bcdDevice            1.00
      iManufacturer           2 CnMemory
      iProduct                3 USB3-SATA-Device
      iSerial                 1 123456789012

    But this seems not to be important as both forward the original disk ID:

    524558955_2020-10-1918_11_52.png.83072598160e8fcf1543531a0bc5763c.png

     

    But the most important part: Both support forwarding the standby mode which solves the issue:

     

    1226266321_2020-10-1918_04_50.png.ff61ca9c43dfaecfc0eaddd575515710.png

     

    sdb=disk3=bus002device003

    sda=disk6=bus002device002

     

    Performance is good as well:

    143150904_2020-10-1921_15_11.png.b20f322d8a4ebcd7874874d4a5f48d15.png

     

    As I was not able to find out how to check the sleep state if the usb adapter does not support the command, I close this bug report as its not solvable. The user needs to use a different adapter.

    Edited by mgutt
    Link to comment
    Share on other sites

    @JorgeB

    Maybe its useful to throw an error notification if the 

    /usr/sbin/smartctl --nocheck standby -A /dev/sdb

    returns this?

    CHECK POWER MODE not implemented

    So the user knows that he needs to use a different adapter.

    Link to comment
    Share on other sites

    I don't recommend using USB for array/pool devices, so don't think it's a high prioruty issue, still not my decision of course, it's up to LT.

    Link to comment
    Share on other sites
    On 10/17/2020 at 10:30 AM, mgutt said:

    I opened a ticket at smartmontools, so the tool hopefully changes its behaviour.

    Saw your ticket in Smartmontools, will take a look and look at a patch, as I think this would be good for UD USB devices, that dont support Power Mode

    Link to comment
    Share on other sites


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.