Hi all,
I found a few threads on similar problems, but none exactly matching my case.
I think I had a power cut which resulted in my server being shut down. I can't be sure, I've had some electrical work done and it might be the electrician cut the power to do some work without me realising while I was at work. Anyway, I don't know when it happened, but then I noticed one of the 8TB disks in the array gave the "Unmountable: Wrong or no file system" error.
I used Check Filesystem Status to run xfs_repair and it gave:
Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!!
attempting to find secondary superblock... followed by an unlimited number of periods.
After running for half a day or so, it finally gave:
Sorry, could not find valid secondary superblock Exiting now.
I ran an extended SMART test, and there didn't seem to be major errors, but likely I just don't understand what I'm looking at:
=== START OF INFORMATION SECTION ===
Model Family: Toshiba N300/MN NAS HDD
Device Model: TOSHIBA HDWG480
Serial Number:
LU WWN Device Id: 5 000039 b18c8a0ab
Firmware Version: 0601
User Capacity: 8,001,563,222,016 bytes [8.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: In smartctl database 7.3/5417
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Jun 18 19:33:38 2023 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is: Unavailable
APM level is: 128 (minimum power consumption without standby)
Rd look-ahead is: Enabled
Write cache is: Enabled
DSN feature is: Unavailable
ATA Security is: Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Unknown
=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.
General SMART Values:
Offline data collection status: (0x84) Offline data collection activity
was suspended by an interrupting command from host.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 120) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 698) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
1 Raw_Read_Error_Rate PO-R-- 100 100 050 - 0
2 Throughput_Performance P-S--- 100 100 050 - 0
3 Spin_Up_Time POS--K 100 100 001 - 11808
4 Start_Stop_Count -O--CK 100 100 000 - 1415
5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0
7 Seek_Error_Rate PO-R-- 100 100 050 - 0
8 Seek_Time_Performance P-S--- 100 100 050 - 0
9 Power_On_Hours -O--CK 078 078 000 - 9037
10 Spin_Retry_Count PO--CK 100 100 030 - 0
12 Power_Cycle_Count -O--CK 100 100 000 - 40
191 G-Sense_Error_Rate -O--CK 100 100 000 - 0
192 Power-Off_Retract_Count -O--CK 100 100 000 - 12
193 Load_Cycle_Count -O--CK 100 100 000 - 2099
194 Temperature_Celsius -O---K 100 100 000 - 34 (Min/Max 17/45)
196 Reallocated_Event_Count -O--CK 100 100 000 - 0
197 Current_Pending_Sector -O--CK 100 100 000 - 0
198 Offline_Uncorrectable ----CK 100 100 000 - 0
199 UDMA_CRC_Error_Count -O--CK 200 200 000 - 0
220 Disk_Shift -O---- 100 100 000 - 34734081
222 Loaded_Hours -O--CK 096 096 000 - 1775
223 Load_Retry_Count -O--CK 100 100 000 - 0
224 Load_Friction -O---K 100 100 000 - 0
226 Load-in_Time -OS--K 100 100 000 - 517
240 Head_Flying_Hours P----- 100 100 001 - 0
||||||_ K auto-keep
|||||__ C event count
||||___ R error rate
|||____ S speed/performance
||_____ O updated online
|______ P prefailure warning
General Purpose Log Directory Version 1
SMART Log Directory Version 1 [multi-sector log support]
Address Access R/W Size Description
0x00 GPL,SL R/O 1 Log Directory
0x01 SL R/O 1 Summary SMART error log
0x02 SL R/O 51 Comprehensive SMART error log
0x03 GPL R/O 5 Ext. Comprehensive SMART error log
0x04 GPL,SL R/O 8 Device Statistics log
0x06 SL R/O 1 SMART self-test log
0x07 GPL R/O 1 Extended self-test log
0x08 GPL R/O 2 Power Conditions log
0x09 SL R/W 1 Selective self-test log
0x0c GPL R/O 513 Pending Defects log
0x10 GPL R/O 1 NCQ Command Error log
0x11 GPL R/O 1 SATA Phy Event Counters log
0x24 GPL R/O 53248 Current Device Internal Status Data log
0x25 GPL R/O 53248 Saved Device Internal Status Data log
0x30 GPL,SL R/O 9 IDENTIFY DEVICE data log
0x80-0x9f GPL,SL R/W 16 Host vendor specific log
0xae GPL VS 25 Device vendor specific log
0xe0 GPL,SL R/W 1 SCT Command/Status
0xe1 GPL,SL R/W 1 SCT Data Transfer
SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged
SMART Extended Self-test Log Version: 1 (1 sectors)
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 9022 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
SCT Status Version: 3
SCT Version (vendor specific): 1 (0x0001)
Device State: SMART Off-line Data Collection executing in background (4)
Current Temperature: 34 Celsius
Power Cycle Min/Max Temperature: 22/38 Celsius
Lifetime Min/Max Temperature: 17/45 Celsius
Specified Max Operating Temperature: 55 Celsius
Under/Over Temperature Limit Count: 0/0
Vendor specific:
00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SCT Temperature History Version: 2
Temperature Sampling Period: 1 minute
Temperature Logging Interval: 1 minute
Min/Max recommended Temperature: 5/55 Celsius
Min/Max Temperature Limit: -40/70 Celsius
Temperature History Size (Index): 478 (463)
Index Estimated Time Temperature Celsius
464 2023-06-18 11:36 35 ****************
... ..(233 skipped). .. ****************
220 2023-06-18 15:30 35 ****************
221 2023-06-18 15:31 34 ***************
... ..(159 skipped). .. ***************
381 2023-06-18 18:11 34 ***************
382 2023-06-18 18:12 35 ****************
383 2023-06-18 18:13 34 ***************
384 2023-06-18 18:14 35 ****************
385 2023-06-18 18:15 34 ***************
... ..( 2 skipped). .. ***************
388 2023-06-18 18:18 34 ***************
389 2023-06-18 18:19 35 ****************
390 2023-06-18 18:20 34 ***************
391 2023-06-18 18:21 34 ***************
392 2023-06-18 18:22 34 ***************
393 2023-06-18 18:23 35 ****************
394 2023-06-18 18:24 34 ***************
395 2023-06-18 18:25 35 ****************
396 2023-06-18 18:26 35 ****************
397 2023-06-18 18:27 35 ****************
398 2023-06-18 18:28 34 ***************
399 2023-06-18 18:29 35 ****************
400 2023-06-18 18:30 34 ***************
401 2023-06-18 18:31 35 ****************
402 2023-06-18 18:32 35 ****************
403 2023-06-18 18:33 35 ****************
404 2023-06-18 18:34 34 ***************
405 2023-06-18 18:35 35 ****************
... ..( 11 skipped). .. ****************
417 2023-06-18 18:47 35 ****************
418 2023-06-18 18:48 34 ***************
419 2023-06-18 18:49 35 ****************
... ..( 8 skipped). .. ****************
428 2023-06-18 18:58 35 ****************
429 2023-06-18 18:59 34 ***************
430 2023-06-18 19:00 35 ****************
... ..( 27 skipped). .. ****************
458 2023-06-18 19:28 35 ****************
459 2023-06-18 19:29 34 ***************
... ..( 3 skipped). .. ***************
463 2023-06-18 19:33 34 ***************
SMART WRITE LOG does not return COUNT and LBA_LOW register
SCT (Get) Error Recovery Control command failed
Device Statistics (GP Log 0x04)
Page Offset Size Value Flags Description
0x01 ===== = = === == General Statistics (rev 3) ==
0x01 0x008 4 40 --- Lifetime Power-On Resets
0x01 0x010 4 9037 --- Power-on Hours
0x01 0x018 6 39736601792 --- Logical Sectors Written
0x01 0x020 6 46976063 --- Number of Write Commands
0x01 0x028 6 286789546719 --- Logical Sectors Read
0x01 0x030 6 164434116 --- Number of Read Commands
0x01 0x038 6 32533200000 --- Date and Time TimeStamp
0x02 ===== = = === == Free-Fall Statistics (rev 1) ==
0x02 0x010 4 0 --- Overlimit Shock Events
0x03 ===== = = === == Rotating Media Statistics (rev 1) ==
0x03 0x008 4 114 --- Spindle Motor Power-on Hours
0x03 0x010 4 582 --- Head Flying Hours
0x03 0x018 4 2099 --- Head Load Events
0x03 0x020 4 0 --- Number of Reallocated Logical Sectors
0x03 0x028 4 0 --- Read Recovery Attempts
0x03 0x030 4 0 --- Number of Mechanical Start Failures
0x03 0x038 4 0 --- Number of Realloc. Candidate Logical Sectors
0x03 0x040 4 12 --- Number of High Priority Unload Events
0x04 ===== = = === == General Errors Statistics (rev 1) ==
0x04 0x008 4 0 --- Number of Reported Uncorrectable Errors
0x04 0x010 4 0 --- Resets Between Cmd Acceptance and Completion
0x05 ===== = = === == Temperature Statistics (rev 1) ==
0x05 0x008 1 34 --- Current Temperature
0x05 0x010 1 33 N-- Average Short Term Temperature
0x05 0x018 1 34 N-- Average Long Term Temperature
0x05 0x020 1 45 --- Highest Temperature
0x05 0x028 1 17 --- Lowest Temperature
0x05 0x030 1 40 N-- Highest Average Short Term Temperature
0x05 0x038 1 21 N-- Lowest Average Short Term Temperature
0x05 0x040 1 34 N-- Highest Average Long Term Temperature
0x05 0x048 1 23 N-- Lowest Average Long Term Temperature
0x05 0x050 4 0 --- Time in Over-Temperature
0x05 0x058 1 55 --- Specified Maximum Operating Temperature
0x05 0x060 4 0 --- Time in Under-Temperature
0x05 0x068 1 5 --- Specified Minimum Operating Temperature
0x06 ===== = = === == Transport Statistics (rev 1) ==
0x06 0x008 4 243 --- Number of Hardware Resets
0x06 0x010 4 307 --- Number of ASR Events
0x06 0x018 4 0 --- Number of Interface CRC Errors
0x07 ===== = = === == Solid State Device Statistics (rev 1) ==
|||_ C monitored condition met
||__ D supports DSN
|___ N normalized value
Pending Defects log (GP Log 0x0c)
No Defects Logged
SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x0001 4 0 Command failed due to ICRC error
0x0002 4 0 R_ERR response for data FIS
0x0003 4 0 R_ERR response for device-to-host data FIS
0x0004 4 0 R_ERR response for host-to-device data FIS
0x0005 4 0 R_ERR response for non-data FIS
0x0006 4 0 R_ERR response for device-to-host non-data FIS
0x0007 4 0 R_ERR response for host-to-device non-data FIS
0x0008 4 0 Device-to-host non-data FIS retries
0x0009 4 4 Transition from drive PhyRdy to drive PhyNRdy
0x000a 4 3 Device-to-host register FISes sent due to a COMRESET
0x000b 4 0 CRC errors within host-to-device FIS
0x000d 4 0 Non-CRC errors within host-to-device FIS
0x000f 4 0 R_ERR response for host-to-device data FIS, CRC
0x0010 4 0 R_ERR response for host-to-device data FIS, non-CRC
0x0012 4 0 R_ERR response for host-to-device non-data FIS, CRC
0x0013 4 0 R_ERR response for host-to-device non-data FIS, non-CRC
What can I do to recover my data? I tried removing the drive from the array, unmounting it, reformatting it and then remounting it, hoping that the Data-Rebuild would fix it, but once it did, it was still showing as Unmountable: Wrong or no file system. So I'm guessing the parity drive backed it up in a corrupted state?
My worry is, if that's the case, then even if I get a replacement drive, then Data-Rebuild will just fill it with corrupted data? Or am I completely wrong? Does it look like the drive is bad and I should get rid of it? Any help is much appreciated.