Jump to content

failing drive? - handle_stripe read error


Recommended Posts

Version: Unraid 4.7

Syslog is attached

 

I was moving some files from one spot on my server to another.  Mid-move, it stopped responding and became unavailable.  I attempted to shutdown -r now and it failed to shutdown.  I hard rebooted the machine and it booted up but the array did not fully mount, only the cache drive.

 

tower1x.png

 

Interface is responding very very slowly.  Unmenu, also responding very slowly, shows parity check in progress, going at something like 32k/sec and would take 132 days to complete.  Attempted to stop parity check through unmenu, and appeared to have worked.  Attempted to stop array through unmenu, but there was no change in the regular web interface.  Ended up hard resetting again.  Same situation the second time around.

 

Found several errors in the syslog.  Looks like maybe I have a bad parity drive?  I'm having a hard time discerning the lines, but these at the end appear to suggest a drive failure.

 

Aug  9 18:20:15 Tower kernel: ata9.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 (Errors)
Aug  9 18:20:15 Tower kernel: ata9.00: BMDMA stat 0x4 (Drive related)
Aug  9 18:20:15 Tower kernel: ata9.00: failed command: READ DMA (Minor Issues)
Aug  9 18:20:15 Tower kernel: ata9.00: cmd c8/00:08:c7:25:b1/00:00:00:00:00/e8 tag 0 dma 4096 in (Drive related)
Aug  9 18:20:15 Tower kernel:          res 51/40:00:c8:25:b1/00:00:00:00:00/e8 Emask 0x9 (media error) (Errors)
Aug  9 18:20:15 Tower kernel: ata9.00: status: { DRDY ERR } (Drive related)
Aug  9 18:20:15 Tower kernel: ata9.00: error: { UNC } (Errors)
Aug  9 18:20:20 Tower kernel: ata9.00: configured for UDMA/133 (Drive related)
Aug  9 18:20:20 Tower kernel: sd 9:0:0:0: [sdh] Unhandled sense code (Drive related)
Aug  9 18:20:20 Tower kernel: sd 9:0:0:0: [sdh] Result: hostbyte=0x00 driverbyte=0x08 (System)
Aug  9 18:20:20 Tower kernel: sd 9:0:0:0: [sdh] Sense Key : 0x3 [current] [descriptor] (Drive related)
Aug  9 18:20:20 Tower kernel: Descriptor sense data with sense descriptors (in hex):
Aug  9 18:20:20 Tower kernel:         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
Aug  9 18:20:20 Tower kernel:         08 b1 25 c8 
Aug  9 18:20:20 Tower kernel: sd 9:0:0:0: [sdh] ASC=0x11 ASCQ=0x4 (Drive related)
Aug  9 18:20:20 Tower kernel: sd 9:0:0:0: [sdh] CDB: cdb[0]=0x28: 28 00 08 b1 25 c7 00 00 08 00 (Drive related)
Aug  9 18:20:20 Tower kernel: end_request: I/O error, dev sdh, sector 145827272 (Errors)
Aug  9 18:20:20 Tower kernel: ata9: EH complete (Drive related)
Aug  9 18:20:20 Tower kernel: md: disk0 read error (Errors)
Aug  9 18:20:20 Tower kernel: handle_stripe read error: 145827208/0, count: 1 (Errors)

 

Thanks for the help!

syslog.txt

Link to comment

Apparently you interrupted the replay of the system journals.  (prior to their being mounted.)

 

You must be patient.  That process can cake as long as 15 minutes or more, especially if there is also a parity check in progress accessing the same disks.   During that process, they will show as "mounting"

 

If you had left them alone, they would have probably eventually mounted.

 

Now, the bulk of the errors are media errors. (un-readable sectors on the disk)  /dev/sdh

 

Joe L.

Link to comment

I was unable to get there via either other method, so I ran it from telnet.  These are the results from the smart report on /dev/sdh

 

root@Tower:~# smartctl -a -d ata /dev/sdh
smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green family
Device Model:     WDC WD20EADS-32S2B0
Serial Number:    WD-WCAVY1342243
Firmware Version: 01.00A01
User Capacity:    2,000,398,934,016 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Tue Aug  9 20:04:41 2011 CDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x84) Offline data collection activity
                                        was suspended by an interrupting command from host.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                 (39060) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x303f) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   186   186   051    Pre-fail  Always       -       90207
  3 Spin_Up_Time            0x0027   143   141   021    Pre-fail  Always       -       9833
  4 Start_Stop_Count        0x0032   098   098   000    Old_age   Always       -       2960
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   087   087   000    Old_age   Always       -       9570
10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       49
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       18
193 Load_Cycle_Count        0x0032   182   182   000    Old_age   Always       -       56430
194 Temperature_Celsius     0x0022   106   071   000    Old_age   Always       -       46
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   196   196   000    Old_age   Always       -       1463
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   199   199   000    Old_age   Offline      -       386

SMART Error Log Version: 1
ATA Error Count: 3731 (device log contains only the most recent five errors)
        CR = Command Register [HEX]
        FR = Features Register [HEX]
        SC = Sector Count Register [HEX]
        SN = Sector Number Register [HEX]
        CL = Cylinder Low Register [HEX]
        CH = Cylinder High Register [HEX]
        DH = Device/Head Register [HEX]
        DC = Device Command Register [HEX]
        ER = Error register [HEX]
        ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 3731 occurred at disk power-on lifetime: 9570 hours (398 days + 18 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 5f b8 d8 e8  Error: UNC 8 sectors at LBA = 0x08d8b85f = 148420703

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 5f b8 d8 e8 08      01:52:06.391  READ DMA
  ec 00 00 00 00 00 a0 08      01:52:03.416  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      01:52:03.416  SET FEATURES [set transfer mode]

Error 3730 occurred at disk power-on lifetime: 9570 hours (398 days + 18 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 5f b8 d8 e8  Error: UNC 8 sectors at LBA = 0x08d8b85f = 148420703

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 5f b8 d8 e8 08      01:51:58.543  READ DMA
  ec 00 00 00 00 00 a0 08      01:51:56.112  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      01:51:56.112  SET FEATURES [set transfer mode]

Error 3729 occurred at disk power-on lifetime: 9570 hours (398 days + 18 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 5f b8 d8 e8  Error: UNC 8 sectors at LBA = 0x08d8b85f = 148420703

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 5f b8 d8 e8 08      01:51:51.250  READ DMA
  ec 00 00 00 00 00 a0 08      01:51:48.817  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      01:51:48.817  SET FEATURES [set transfer mode]

Error 3728 occurred at disk power-on lifetime: 9570 hours (398 days + 18 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 5f b8 d8 e8  Error: UNC 8 sectors at LBA = 0x08d8b85f = 148420703

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 5f b8 d8 e8 08      01:51:44.002  READ DMA
  ec 00 00 00 00 00 a0 08      01:51:41.571  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      01:51:41.571  SET FEATURES [set transfer mode]

Error 3727 occurred at disk power-on lifetime: 9570 hours (398 days + 18 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 5f b8 d8 e8  Error: UNC 8 sectors at LBA = 0x08d8b85f = 148420703

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  c8 00 08 5f b8 d8 e8 08      01:51:36.702  READ DMA
  ec 00 00 00 00 00 a0 08      01:51:33.768  IDENTIFY DEVICE
  ef 03 46 00 00 00 a0 08      01:51:33.768  SET FEATURES [set transfer mode]

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      6102         -

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@Tower:~#

Link to comment

Overnight the remaining drives mounted.  This is the current status.  Should I wait a week for the parity check to finish?  New drive will be here before then.

 

tower1.png

 

tower2t.png

No... Stop the parity check, stop the array, power down, and replace the parity disk.

 

You will take the chances that the replacement is good, so do an initial parity calc, then a subsequent parity check.

Link to comment

I ordered another drive, but for future reference, is there an easy way to get the server back up and running with a 'full' array without replacing it?  I have enough free space.  What I am wondering is can I pick a new parity drive from my existing drives, have unraid move the contents to other disks and begin the parity calculation?  I am guessing I have to do the moving of the existing contents manually.

Link to comment

I ordered another drive, but for future reference, is there an easy way to get the server back up and running with a 'full' array without replacing it?  I have enough free space.  What I am wondering is can I pick a new parity drive from my existing drives, have unraid move the contents to other disks and begin the parity calculation?  I am guessing I have to do the moving of the existing contents manually.

Yes, you can move all the files off of a drive, then

stop the array

assign that drive as parity

set a new disk configuration by logging in and typing

initconfig

then start the array and let it build parity based on the new disk configuration.

 

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...