Rebooted the array and the drive was unmountable with offline uncorrectable 111, current pending sector 112. Went from 1 two days ago and jumped up yesterday to 111. I have the community backup running it should have most of it. Tried once yesterday to rsync everything off , but got got tons of errors about 4 hours in.
I'm curious how much time I should invest into recovering data from the drive of if it's a lost cause.
When on the webGUI page for the drive if i click on btrfs check I get
parent transid verify failed on 1071661056 wanted 42552013 found 42546596
parent transid verify failed on 1071661056 wanted 42552013 found 42546596
Ignoring transid failure
ERROR: could not setup extent tree
ERROR: cannot open file system
Opening filesystem to check...
When I downloaded the smart info for the drive I got this:
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.10.28-Unraid] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar Black
Device Model: WDC WD2002FAEX-007BA0
Serial Number: WD-WMAY02******
LU WWN Device Id: 5 0014ee 65668b897
Firmware Version: 05.01D05
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 2.6, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sat Aug 7 13:25:12 2021 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is: Disabled
APM feature is: Unavailable
Rd look-ahead is: Enabled
Write cache is: Enabled
DSN feature is: Unavailable
ATA Security is: Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 121) The previous self-test completed having
the read element of the test failed.
Total time to complete Offline
data collection: (28860) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 294) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x3037) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
1 Raw_Read_Error_Rate POSR-K 200 200 051 - 0
3 Spin_Up_Time POS--K 253 253 021 - 8883
4 Start_Stop_Count -O--CK 079 079 000 - 21958
5 Reallocated_Sector_Ct PO--CK 200 200 140 - 0
7 Seek_Error_Rate -OSR-K 100 253 000 - 0
9 Power_On_Hours -O--CK 001 001 000 - 88502
10 Spin_Retry_Count -O--CK 100 100 000 - 0
11 Calibration_Retry_Count -O--CK 100 100 000 - 0
12 Power_Cycle_Count -O--CK 100 100 000 - 269
192 Power-Off_Retract_Count -O--CK 200 200 000 - 163
193 Load_Cycle_Count -O--CK 190 190 000 - 30593
194 Temperature_Celsius -O---K 108 094 000 - 44
196 Reallocated_Event_Count -O--CK 200 200 000 - 0
197 Current_Pending_Sector -O--CK 200 200 000 - 112
198 Offline_Uncorrectable ----CK 200 200 000 - 111
199 UDMA_CRC_Error_Count -O--CK 200 200 000 - 0
200 Multi_Zone_Error_Rate ---R-- 200 200 000 - 111
||||||_ K auto-keep
|||||__ C event count
||||___ R error rate
|||____ S speed/performance
||_____ O updated online
|______ P prefailure warning
General Purpose Log Directory Version 1
SMART Log Directory Version 1 [multi-sector log support]
Address Access R/W Size Description
0x00 GPL,SL R/O 1 Log Directory
0x01 SL R/O 1 Summary SMART error log
0x02 SL R/O 5 Comprehensive SMART error log
0x03 GPL R/O 6 Ext. Comprehensive SMART error log
0x06 SL R/O 1 SMART self-test log
0x07 GPL R/O 1 Extended self-test log
0x09 SL R/W 1 Selective self-test log
0x10 GPL R/O 1 NCQ Command Error log
0x11 GPL R/O 1 SATA Phy Event Counters log
0x80-0x9f GPL,SL R/W 16 Host vendor specific log
0xa0-0xa7 GPL,SL VS 16 Device vendor specific log
0xa8-0xb5 GPL,SL VS 1 Device vendor specific log
0xb6 GPL VS 1 Device vendor specific log
0xb7 GPL,SL VS 1 Device vendor specific log
0xbd GPL,SL VS 1 Device vendor specific log
0xc0 GPL,SL VS 1 Device vendor specific log
0xc1 GPL VS 24 Device vendor specific log
0xe0 GPL,SL R/W 1 SCT Command/Status
0xe1 GPL,SL R/W 1 SCT Data Transfer
SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
Device Error Count: 5
CR = Command Register
FEATR = Features Register
COUNT = Count (was: Sector Count) Register
LBA_48 = Upper bytes of LBA High/Mid/Low Registers ] ATA-8
LH = LBA High (was: Cylinder High) Register ] LBA
LM = LBA Mid (was: Cylinder Low) Register ] Register
LL = LBA Low (was: Sector Number) Register ]
DV = Device (was: Device/Head) Register
DC = Device Control Register
ER = Error register
ST = Status register
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 5 [4] occurred at disk power-on lifetime: 22965 hours (956 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
40 -- 51 00 08 00 00 b1 1f 05 98 40 00 Error: WP at LBA = 0xb11f0598 = 2971600280
Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
61 00 08 00 f8 00 00 20 00 00 40 40 08 10d+00:31:25.721 WRITE FPDMA QUEUED
61 00 08 00 f0 00 00 00 02 00 40 40 08 10d+00:31:25.721 WRITE FPDMA QUEUED
60 00 18 00 e8 00 00 09 ef a5 d0 40 08 10d+00:31:25.721 READ FPDMA QUEUED
60 00 28 00 e0 00 00 06 69 67 e8 40 08 10d+00:31:25.721 READ FPDMA QUEUED
60 00 08 00 d8 00 00 04 46 3d d8 40 08 10d+00:31:25.721 READ FPDMA QUEUED
Error 4 [3] occurred at disk power-on lifetime: 22965 hours (956 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
40 -- 51 00 08 00 00 b1 1f 05 98 40 00 Error: WP at LBA = 0xb11f0598 = 2971600280
Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
61 00 40 00 f0 00 00 00 1f f0 c0 40 08 10d+00:30:39.503 WRITE FPDMA QUEUED
61 00 20 00 d8 00 00 00 1f ec 60 40 08 10d+00:30:39.503 WRITE FPDMA QUEUED
61 00 20 00 d0 00 00 00 1f ec 20 40 08 10d+00:30:39.503 WRITE FPDMA QUEUED
61 00 20 00 c8 00 00 00 1f eb e0 40 08 10d+00:30:39.503 WRITE FPDMA QUEUED
61 00 60 00 c0 00 00 00 1f eb 60 40 08 10d+00:30:39.503 WRITE FPDMA QUEUED
Error 3 [2] occurred at disk power-on lifetime: 22965 hours (956 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
40 -- 51 00 08 00 00 b1 1f 05 98 40 00 Error: WP at LBA = 0xb11f0598 = 2971600280
Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
61 00 08 00 20 00 00 00 00 00 c0 40 08 10d+00:30:01.242 WRITE FPDMA QUEUED
60 00 08 00 18 00 00 b1 1f 06 10 40 08 10d+00:30:01.242 READ FPDMA QUEUED
60 00 08 00 10 00 00 b1 1f 05 98 40 08 10d+00:30:01.242 READ FPDMA QUEUED
60 00 08 00 08 00 00 04 46 3d c8 40 08 10d+00:30:01.242 READ FPDMA QUEUED
ea 00 00 00 00 00 00 00 00 00 00 e0 08 10d+00:30:01.163 FLUSH CACHE EXT
Error 2 [1] occurred at disk power-on lifetime: 22965 hours (956 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
40 -- 51 02 00 00 00 b1 1f 05 98 40 00 Error: UNC at LBA = 0xb11f0598 = 2971600280
Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
60 02 00 00 00 00 00 ad a0 a4 40 40 08 10d+00:29:23.096 READ FPDMA QUEUED
60 02 00 00 f8 00 00 ad a0 a2 40 40 08 10d+00:29:23.096 READ FPDMA QUEUED
60 02 00 00 f0 00 00 ad a0 a0 40 40 08 10d+00:29:23.096 READ FPDMA QUEUED
60 01 80 00 e8 00 00 b1 1f 0f 90 40 08 10d+00:29:23.096 READ FPDMA QUEUED
60 02 00 00 e0 00 00 b1 1f 0d 90 40 08 10d+00:29:23.096 READ FPDMA QUEUED
Error 1 [0] occurred at disk power-on lifetime: 22934 hours (955 days + 14 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
01 -- 51 0a 00 00 00 e8 8e bb 84 40 00 Error: AMNF at LBA = 0xe88ebb84 = 3901668228
Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
60 02 00 00 b8 00 00 e8 8e d1 c0 40 08 8d+17:00:22.280 READ FPDMA QUEUED
60 0a 00 00 b0 00 00 e8 8e c7 c0 40 08 8d+17:00:22.279 READ FPDMA QUEUED
60 0a 00 00 a8 00 00 e8 8e bd c0 40 08 8d+17:00:22.275 READ FPDMA QUEUED
60 0a 00 00 a0 00 00 e8 8e b3 c0 40 08 8d+17:00:22.275 READ FPDMA QUEUED
60 03 60 00 98 00 00 e8 8e b0 60 40 08 8d+17:00:22.193 READ FPDMA QUEUED
SMART Extended Self-test Log Version: 1 (1 sectors)
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed: read failure 90% 22966 2971600280
# 2 Extended offline Completed without error 00% 53900 -
# 3 Extended offline Completed without error 00% 37158 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
SCT Status Version: 3
SCT Version (vendor specific): 258 (0x0102)
Device State: Active (0)
Current Temperature: 44 Celsius
Power Cycle Min/Max Temperature: 30/44 Celsius
Lifetime Min/Max Temperature: 30/58 Celsius
Under/Over Temperature Limit Count: 0/0
SCT Temperature History Version: 2
Temperature Sampling Period: 1 minute
Temperature Logging Interval: 1 minute
Min/Max recommended Temperature: 0/60 Celsius
Min/Max Temperature Limit: -41/85 Celsius
Temperature History Size (Index): 478 (198)
Index Estimated Time Temperature Celsius
199 2021-08-07 05:28 45 **************************
... ..( 2 skipped). .. **************************
202 2021-08-07 05:31 45 **************************
203 2021-08-07 05:32 46 ***************************
... ..( 2 skipped). .. ***************************
206 2021-08-07 05:35 46 ***************************
207 2021-08-07 05:36 47 ****************************
... ..( 4 skipped). .. ****************************
212 2021-08-07 05:41 47 ****************************
213 2021-08-07 05:42 48 *****************************
... ..( 37 skipped). .. *****************************
251 2021-08-07 06:20 48 *****************************
252 2021-08-07 06:21 49 ******************************
... ..( 33 skipped). .. ******************************
286 2021-08-07 06:55 49 ******************************
287 2021-08-07 06:56 50 *******************************
... ..( 18 skipped). .. *******************************
306 2021-08-07 07:15 50 *******************************
307 2021-08-07 07:16 51 ********************************
... ..( 43 skipped). .. ********************************
351 2021-08-07 08:00 51 ********************************
352 2021-08-07 08:01 ? -
353 2021-08-07 08:02 30 ***********
354 2021-08-07 08:03 31 ************
355 2021-08-07 08:04 32 *************
356 2021-08-07 08:05 32 *************
357 2021-08-07 08:06 33 **************
358 2021-08-07 08:07 34 ***************
359 2021-08-07 08:08 34 ***************
360 2021-08-07 08:09 35 ****************
... ..( 2 skipped). .. ****************
363 2021-08-07 08:12 35 ****************
364 2021-08-07 08:13 36 *****************
365 2021-08-07 08:14 36 *****************
366 2021-08-07 08:15 37 ******************
367 2021-08-07 08:16 37 ******************
368 2021-08-07 08:17 37 ******************
369 2021-08-07 08:18 38 *******************
370 2021-08-07 08:19 38 *******************
371 2021-08-07 08:20 ? -
372 2021-08-07 08:21 30 ***********
373 2021-08-07 08:22 30 ***********
374 2021-08-07 08:23 31 ************
375 2021-08-07 08:24 32 *************
376 2021-08-07 08:25 32 *************
377 2021-08-07 08:26 33 **************
378 2021-08-07 08:27 34 ***************
379 2021-08-07 08:28 34 ***************
380 2021-08-07 08:29 35 ****************
381 2021-08-07 08:30 35 ****************
382 2021-08-07 08:31 35 ****************
383 2021-08-07 08:32 36 *****************
384 2021-08-07 08:33 36 *****************
385 2021-08-07 08:34 37 ******************
386 2021-08-07 08:35 37 ******************
387 2021-08-07 08:36 37 ******************
388 2021-08-07 08:37 38 *******************
389 2021-08-07 08:38 38 *******************
390 2021-08-07 08:39 39 ********************
391 2021-08-07 08:40 39 ********************
392 2021-08-07 08:41 39 ********************
393 2021-08-07 08:42 40 *********************
... ..( 6 skipped). .. *********************
400 2021-08-07 08:49 40 *********************
401 2021-08-07 08:50 41 **********************
... ..( 2 skipped). .. **********************
404 2021-08-07 08:53 41 **********************
405 2021-08-07 08:54 42 ***********************
... ..( 5 skipped). .. ***********************
411 2021-08-07 09:00 42 ***********************
412 2021-08-07 09:01 43 ************************
... ..( 6 skipped). .. ************************
419 2021-08-07 09:08 43 ************************
420 2021-08-07 09:09 44 *************************
... ..( 3 skipped). .. *************************
424 2021-08-07 09:13 44 *************************
425 2021-08-07 09:14 46 ***************************
... ..(117 skipped). .. ***************************
65 2021-08-07 11:12 46 ***************************
66 2021-08-07 11:13 45 **************************
... ..( 71 skipped). .. **************************
138 2021-08-07 12:25 45 **************************
139 2021-08-07 12:26 44 *************************
... ..( 7 skipped). .. *************************
147 2021-08-07 12:34 44 *************************
148 2021-08-07 12:35 43 ************************
... ..( 4 skipped). .. ************************
153 2021-08-07 12:40 43 ************************
154 2021-08-07 12:41 42 ***********************
... ..( 8 skipped). .. ***********************
163 2021-08-07 12:50 42 ***********************
164 2021-08-07 12:51 41 **********************
... ..( 26 skipped). .. **********************
191 2021-08-07 13:18 41 **********************
192 2021-08-07 13:19 42 ***********************
193 2021-08-07 13:20 43 ************************
194 2021-08-07 13:21 43 ************************
195 2021-08-07 13:22 43 ************************
196 2021-08-07 13:23 44 *************************
197 2021-08-07 13:24 44 *************************
198 2021-08-07 13:25 44 *************************
SCT Error Recovery Control command not supported
Device Statistics (GP/SMART Log 0x04) not supported
Pending Defects log (GP Log 0x0c) not supported
SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x0001 2 0 Command failed due to ICRC error
0x0002 2 0 R_ERR response for data FIS
0x0003 2 0 R_ERR response for device-to-host data FIS
0x0004 2 0 R_ERR response for host-to-device data FIS
0x0005 2 0 R_ERR response for non-data FIS
0x0006 2 0 R_ERR response for device-to-host non-data FIS
0x0007 2 0 R_ERR response for host-to-device non-data FIS
0x000a 2 1 Device-to-host register FISes sent due to a COMRESET
0x000b 2 0 CRC errors within host-to-device FIS
0x8000 4 3169 Vendor specific
Thanks for any time and help,
Bobby
storage-diagnostics-20210807-1402.zip