June 29, 20179 yr Hi, I just got notification that one of my disks has read errors but I am unsure where to go from here? is it a one-off? should I replace it? If anyone could give me any pointers I would appreciate it! :-)
June 29, 20179 yr Community Expert Pending sectors, replace it: 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 4
June 29, 20179 yr Wow! Some interesting finds with this drive's smart report: SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 197 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 168 160 021 Pre-fail Always - 6583 4 Start_Stop_Count 0x0032 094 094 000 Old_age Always - 6737 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0 9 Power_On_Hours 0x0032 043 043 000 Old_age Always - 42063 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 1922 192 Power-Off_Retract_Count 0x0032 199 199 000 Old_age Always - 1421 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 967932 194 Temperature_Celsius 0x0022 118 107 000 Old_age Always - 32 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 4 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 177 000 Old_age Always - 496592 200 Multi_Zone_Error_Rate 0x0008 182 174 000 Old_age Offline - 4923 Power on hours - 42000 = 4.8 years. Not a spring chicken. Load Cycle Count - ~1M! Wow!! But typical WD. There is a way to get it to not park the heads so frequently that I recommend setting to avoid excessive parking. Pending sectors - 4. Generally agree with @johnnie.black that replacement is probably indicated, although I'd usually suggest a few (non-correcting) parity checks and see if they clear or get no worse. It is nice to keep some of these historical reports to compare back and see if this has been this was for a long time, or if this just happened. UDMA CRC Error Count - Half million! Ding ding ding - we have a winner. This is a record I think. UDMA CRC errors are usually indicative of a cabling problem with the drive. This must have been resets forever! Not sure if this could have been a part of the read errors. Probably not. But CRC errors are not a reason to replace a drive. They are a reason to fix the cabling. BTW, once fixed this will stop incrementing, but the value will never reduce. So the cabling may have been fixed already and stopped incrementing. But with such a large number, I am doubting it. Multi Zone Error Rate - ~5000! Never seen so many! Normally we see 1 or 2, and are often associated with some weird behavior. I consider this a leading indicator of non-media (i.e, mechanical) related issues. But every manufacturer and even drive model may have a different internal meaning of this, and maybe for this drive (which is pretty ancient) this was normal. Given that the drive has lasted a long time, and I doubt this happened overnight, maybe not as serious an issue as I would typically believe. But I would call it a very bad sign. Overall I'd agree with @johnnie.black'[s conclusion. You could spend a little time messing with it, but I give a 98% chance that the problems will only get worse. FYI - i have one of these drives in my backup server. I once had 2 of them, but other failed a long time ago. This one still looks pretty healthy. Probably about the same physical age - but it has been in backup duty for at couple years and only infrequently powered on, so hours are less. No CRC errors because cabling has always been good. Far fewer LCCs (I think I did the update to reduce the parking). And no multi-zone errors. BTW, the other one that failed was still in my old disk box, and I was messing with it recently to see if it had any life left. It was a refurb as the original drive failed under warranty and was replaced. The controller it was plugged into would not get through POST (on 3 different controllers) with that drive attached. It is a gonner for sure. The 1T version of this series was great. Still have several of them and they work. But I was not that impressed with the 2T drives and this led me to look elsewhere and moved to Hitachi and HGST drives which were too good and never had a reason to come back to WD. 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 150 149 021 Pre-fail Always - 9491 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 491 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0 9 Power_On_Hours 0x0032 063 063 000 Old_age Always - 27202 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 287 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 115 193 Load_Cycle_Count 0x0032 183 183 000 Old_age Always - 52463 194 Temperature_Celsius 0x0022 126 114 000 Old_age Always - 26 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
June 29, 20179 yr Community Expert 2 minutes ago, bjp999 said: although I'd usually suggest a few (non-correcting) parity checks and see if they clear or get no worse Please don't forget to mention that if a users wants to try this it needs to be a non correcting check or the user risks corrupting parity, resulting then in a corrupt rebuild. Edited June 29, 20179 yr by johnnie.black
June 29, 20179 yr One of my similar drives for comparison Device Model: WDC WD20EARS-00MVWB0 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 535 3 Spin_Up_Time 0x0027 173 166 021 Pre-fail Always - 6341 4 Start_Stop_Count 0x0032 090 090 000 Old_age Always - 10495 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 024 024 000 Old_age Always - 56053 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 164 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 110 193 Load_Cycle_Count 0x0032 090 090 000 Old_age Always - 331914 194 Temperature_Celsius 0x0022 126 112 000 Old_age Always - 24 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 2 200 Multi_Zone_Error_Rate 0x0008 200 194 000 Old_age Offline - 0
June 29, 20179 yr Community Expert 23 minutes ago, bjp999 said: Load Cycle Count - ~1M! Wow!! But typical WD. There is a way to get it to not park the heads so frequently that I recommend setting to avoid excessive parking. Check this one (not mine), this is the highest LLC I've ever seen: Model Family: Western Digital Caviar Green (AF) Device Model: WDC WD20EARS-00MVWB0 Serial Number: WD-WCAZA1349441 LU WWN Device Id: 5 0014ee 20503e92b Firmware Version: 51.0AB51 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Size: 512 bytes logical/physical Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS (minor revision not indicated) SATA Version is: SATA 2.6, 3.0 Gb/s Local Time is: Thu Jun 1 01:29:17 2017 CDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART Status not supported: Incomplete response, ATA output registers missing SMART overall-health self-assessment test result: PASSED Warning: This result is based on an Attribute check. Extended self-test routine recommended polling time: ( 355) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_ FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 189 172 021 Pre-fail Always - 5541 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 61 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 063 063 000 Old_age Always - 27403 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 59 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 68 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 1357642 194 Temperature_Celsius 0x0022 120 108 000 Old_age Always - 30 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 13 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 1
June 29, 20179 yr While looking at the diagnostics file, I took a look at your other drives (identified by last three chars of the serial number below). Summary - Drive 51J needs to be replaced IMO. You are operating at the edge of failure according to manufacturer settings. Several (most) drives have multi-zone and/or CRC errors. (I didn't mention drive with fewer than 100 CRC errors). The CRC errors are indicative of cabling problems. I'd say you have a systemic problem with your cabling. The multizone errors I question how much to worry about them. My experience is that when they are present in conjunction with other issues, it raises my concern. These in isolation - I can't speak to. Since almost all of your drives have them, maybe best course is to just monitor. And run your non-correcting parity checks on a regular basis. I would NEVER run a correcting check on this array unless a non-correcting check indicated a small number of explainable errors (e.g., after a hard shutdown). Suggestions: 1. Replace 511 and then immediately 51J (in that order). 2. You won't want to hear this, but your array is full of a lot of older drives that are having issues. This might be a good time to do a drive replacement cycle. A few 8T drives would replace these older 2T and 1.5T drives 4+ for 1. Use these older ones for backups. Details: 51J: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 183 Runtime_Bad_Block 0x0032 001 001 000 Old_age Always - 811 187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 1935 195 Hardware_ECC_Recovered 0x001a 037 023 000 Old_age Always - 1940782640 Notice the Value, Worst, Thresh columns. These are normalized values. "Value" is nominally 100. "Worst" is worst "Value" that has been detected. "Thresh" is the point as which manufacturer says drive has failed. Notice that the "Value/Worst" for 2 of them are 1, and the thresholds are 0. This is extremely close to failure according to the manufacturer. And they tend to be conservative. These are not trivial attributes! This drive needs to be replaced IMO. The third one I list (ECC recovered) is also bad, but not quite as bad, but certainly nothing I'd want in my array even if the other attributes were normal. 015: Multi-zone - 420 UDMA - 248 951: Multi-zone - 580 968: Multi-zone - 1300 UDMA - 18600 594: Multi-zone - 8500 048: Multi-zone - 520 748: Multi-zone - 1024 160: Multi-zone - 419 UDMA - 240
June 29, 20179 yr Author Wow, Thanks for the detailed replies everyone! Really appreciated! I am currently moving the data from 511 now, probably the oldest drive in the array anyway! I will just keep it for backups of backups! Drive 51J is just a 1TB drive from my old PS3 that I recently put in, it cleared ok, so not sure how it has so many errors that were not brought to light before adding it to my array! Again, Thanks for all your replies, it made me all warm and fuzzy inside! :-D Cheers
June 29, 20179 yr I can offer more LCCs Its my WD3200BEVT working as unassigned device. Edited June 29, 20179 yr by Zonediver
June 29, 20179 yr Author Zonediver is now in the lead! :-D can anyone recommend a tool to clear/low level format & reset (if possible) some of these errors once the drives are removed from the pool? just out of interest really, they won't be going back in! :-D Lastly, is anyone using any plugins that monitor drive health on top of what unRAID already does? Cheers all
June 29, 20179 yr The idea of low level formatting went away a long time ago. There is no way to reset the SMART values. Not sure about monitoring tools. Dynamix already has some built in. I monitor mine before and after parity checks to see if any new issues are occurring. A little knowledge about the ones to watch out for is helpful, but it is not rocket science.
June 30, 20179 yr 2 hours ago, bjp999 said: There is no way to reset the SMART values. I heard a rumor on here a couple years ago that there are utility suites available that CAN mess with stuff like that, but that you can only get them from the manufacturer, and it's very large $$$. Typically used forensically. Or something like that.
June 30, 20179 yr I do know that when they refurb a drive the manufacturer resets the smart settings. So I it is possible. But if such a tool were available, someone on eBay could sell you a failing drive as new. I think such a feature would quickly become common knowledge and people would be screaming to not buy drives on eBay.
June 30, 20179 yr 7 minutes ago, bjp999 said: I do know that when they refurb a drive the manufacturer resets the smart settings. So I it is possible. But if such a tool were available, someone on eBay could sell you a failing drive as new. I think such a feature would quickly become common knowledge and people would be screaming to not buy drives on eBay. I think the up front investment and legal hoops necessary to obtain said tools would make it extremely unlikely that someone would jeopardize their business by resetting and reselling failed drives on eBay.
Archived
This topic is now archived and is closed to further replies.