mgworek Posted May 7, 2015 Share Posted May 7, 2015 i noticed last night one of my drives had a yellow triangle next to it. I rebooted the server without saving my syslog (i know i know). It came back up, and now the drive just has a gray triangle next to it. I can browse the drive from my computer. I don't have the buttons to do a parity check on the web interface so not sure where to go from here. I do have a piece of the my old log though. Before I rebooted, the drive in question had this in the log. ATA_OP e3 ioctl error: -5 After upgrading to 6.0-beta 15 I did notice this drive was the last drive to spin down for some reason. attached is my current syslog and here is my SMART report. === START OF INFORMATION SECTION === Model Family: Seagate Barracuda Green (AF) Device Model: ST2000DL003-9VT166 Serial Number: 5YD2JDKT LU WWN Device Id: 5 000c50 02fe230b0 Firmware Version: CC32 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Size: 512 bytes logical/physical Rotation Rate: 5900 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Thu May 7 09:21:29 2015 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 633) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 347) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x30b7) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 115 099 006 Pre-fail Always - 94717408 3 Spin_Up_Time 0x0003 093 073 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 091 091 020 Old_age Always - 9566 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 075 060 030 Pre-fail Always - 37110394 9 Power_On_Hours 0x0032 091 011 000 Old_age Always - 8333 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 228 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 071 049 045 Old_age Always - 29 (Min/Max 27/42) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 47 193 Load_Cycle_Count 0x0032 096 096 000 Old_age Always - 9593 194 Temperature_Celsius 0x0022 029 051 000 Old_age Always - 29 (0 19 0 0 0) 195 Hardware_ECC_Recovered 0x001a 021 009 000 Old_age Always - 94717408 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 240251880612439 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2616117722 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 3763453709 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 53090 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. root@Tower:/mnt# syslog.zip Quote Link to comment
RobJ Posted May 8, 2015 Share Posted May 8, 2015 Various comments - * Smart report looks great, except for one bazaar number, probably harmless. Power on hours is 8333, but it says you ran a SMART short test at 53090 hours! Disk 6 is probably fine. * The SATA support for 2 onboard SATA ports is configured in your BIOS settings to be in IDE emulation mode. The connected drives for those ports just happen to be your Parity drive and Disk 1! When you next reboot, go into the BIOS settings, locate the SATA configuration, and select a native SATA mode, preferably AHCI, anything but an IDE emulation mode. One of the bad things about IDE handling is that it treats disks in pairs per channel, and if something goes wrong with one drive on a channel, it may cause trouble for both drives. In your case, it is particularly confused, because it thinks the "IDE cables" to them are 40 wire, so it has slowed access to both drives down to UDMA/33, which will result in very slow reads and writes to those disks. Turning on a native SATA mode like AHCI will cause each drive to be independent of the others, and full speed. May 6 22:36:12 Tower logger: plugin: installing: /boot/config/plugins/dynamix.plg May 6 22:36:12 Tower logger: plugin: not installing older version * The above is sometimes evidence that there may be v5 files still installed, which may cause strange problems. You may want to review the upgrade instructions in the Upgrading to UnRAID v6 guide, to make sure that no remnants of v5 remain. Before I rebooted, the drive in question had this in the log. ATA_OP e3 ioctl error: -5 * Unfortunately, I need to see the context for the error, what subsystem is reporting it, and what devices are associated with it. I suspect it's the controller, but don't know for sure. SAS cards give us problems now and then. * There's something funny about this SAS controller, because it's taking way too long to initialize. The mvsas module loads very quickly, but the actual drive setup starts after everything else is complete, then takes quite awhile working on each drive. Disk 6 just happens to be connected to the 8th and last port on this controller, which means it was the last drive of all to be set up, almost too late to be included in the drive inventory! That's not its fault. I suspect if you reconnected it to an onboard port or one of the 2 port cards, it would show no problems at all. * The problem now is that it was dropped last time, for unknown reasons, but probably the controllers fault not the drives fault. So unRAID is simulating it, a virtual Disk 6. But since a drive is considered missing from the array, you cannot do a parity check, which requires all drives online. There are several methods to correct this. It's important now to know what writes have taken place to the array. If you are absolutely sure that nothing has been written or changed on Disk 6, then what I personally would do is note the current drive assignments, do a 'New Config', reassign the drives exactly as they are now, then click the 'Parity is valid' (or however it is worded) box, and the array should be fine. Then run a parity check (if not already running) to correct a few file system housekeeping bits (you should see a few parity corrections). HOWEVER, there are users who consider that dangerous. So if you suspect anything might have been written to the drive since it was disabled, and/or you want to handle this the safest way, then unassign Disk 6, start then stop the array, reassign Disk 6, then start the array and let unRAID rebuild Disk 6 onto itself. I'd wait a little, in case others have additional comments or corrections to anything I have suggested. Do tell us though whether Disk 6 may have been written to recently. Quote Link to comment
mgworek Posted May 8, 2015 Author Share Posted May 8, 2015 "* The SATA support for 2 onboard SATA ports is configured in your BIOS settings to be in IDE emulation mode. The connected drives for those ports just happen to be your Parity drive and Disk 1! When you next reboot, go into the BIOS settings, locate the SATA configuration, and select a native SATA mode, preferably AHCI, anything but an IDE emulation mode. One of the bad things about IDE handling is that it treats disks in pairs per channel, and if something goes wrong with one drive on a channel, it may cause trouble for both drives. In your case, it is particularly confused, because it thinks the "IDE cables" to them are 40 wire, so it has slowed access to both drives down to UDMA/33, which will result in very slow reads and writes to those disks. Turning on a native SATA mode like AHCI will cause each drive to be independent of the others, and full speed." Thanks for that info. I will do that. I didn't realize it was set to that. It has been like 5/6 years since I paid attention to the bios or the booting up process.... so that smart report, ignore it. It wasn't for the the drive in question which is disk14. I could have sworn I did the smart report for that drive but I guess I mistyped it and never noticed until you mentioned it. Sorry about that. I will get the correct report and post it. Quote Link to comment
mgworek Posted May 8, 2015 Author Share Posted May 8, 2015 when your not paying attention p and q look the same..... Here is the correct smart report, once again sorry for that. === START OF INFORMATION SECTION === Model Family: Seagate Desktop HDD.15 Device Model: ST4000DM000-1F2168 Serial Number: W30061FQ LU WWN Device Id: 5 000c50 03ced0d07 Firmware Version: CC54 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5900 rpm Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Fri May 8 09:52:07 2015 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 128) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 543) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x1085) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 113 099 006 Pre-fail Always - 50413025 3 Spin_Up_Time 0x0003 094 091 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 258 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 079 060 030 Pre-fail Always - 93891568 9 Power_On_Hours 0x0032 087 087 000 Old_age Always - 11575 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 90 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 099 099 000 Old_age Always - 1 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 0 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 069 052 045 Old_age Always - 31 (Min/Max 31/35) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 0 193 Load_Cycle_Count 0x0032 090 090 000 Old_age Always - 20371 194 Temperature_Celsius 0x0022 031 048 000 Old_age Always - 31 (0 20 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 8 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 5222h+56m+45.915s 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 57195478888 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 239191060360 SMART Error Log Version: 1 ATA Error Count: 1 CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 1 occurred at disk power-on lifetime: 11513 hours (479 days + 17 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 60 00 08 ff ff ff 4f 00 1d+21:40:22.947 READ FPDMA QUEUED 60 00 08 ff ff ff 4f 00 1d+21:40:22.947 READ FPDMA QUEUED 60 00 08 ff ff ff 4f 00 1d+21:40:22.947 READ FPDMA QUEUED 60 00 08 ff ff ff 4f 00 1d+21:40:22.946 READ FPDMA QUEUED 60 00 08 ff ff ff 4f 00 1d+21:40:22.946 READ FPDMA QUEUED SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. Quote Link to comment
trurl Posted May 8, 2015 Share Posted May 8, 2015 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 8 Do you have another drive you can rebuild this one onto? Quote Link to comment
mgworek Posted May 8, 2015 Author Share Posted May 8, 2015 I don't currently but I can get one. Would just need to wait the 2 days it takes to preclear it. I always said I need to buy an extra drive to keep as a spare but i never do. The drive was the last drive installed so it might be under warranty still. I never had a data drive fail yet. I had my parity drive fail on me before. Quote Link to comment
trurl Posted May 8, 2015 Share Posted May 8, 2015 I would preclear a new drive then rebuild. After you get everything working again, you can try preclearing the problem drive to see if the pending sectors and offline uncorrectable will clear up. If so then you may be able to use it again. If not then maybe you can return it under warranty. Have you ever done a rebuild? Quote Link to comment
mgworek Posted May 8, 2015 Author Share Posted May 8, 2015 I would preclear a new drive then rebuild. After you get everything working again, you can try preclearing the problem drive to see if the pending sectors and offline uncorrectable will clear up. If so then you may be able to use it again. If not then maybe you can return it under warranty. Have you ever done a rebuild? When I replaced my original parity drive a few months later my new parity drive failed so I think I had to do a rebuild then. That was like 2 years ago so don't really remember . Quote Link to comment
trurl Posted May 9, 2015 Share Posted May 9, 2015 I would preclear a new drive then rebuild. After you get everything working again, you can try preclearing the problem drive to see if the pending sectors and offline uncorrectable will clear up. If so then you may be able to use it again. If not then maybe you can return it under warranty. Have you ever done a rebuild? When I replaced my original parity drive a few months later my new parity drive failed so I think I had to do a rebuild then. That was like 2 years ago so don't really remember . Just want to mention that formatting a drive is NEVER part of the rebuild process. Somehow people have gotten the idea that the first thing they need to do with any disk before they can use it is format it. Several have managed to coax the gui into letting them format the drive they need to rebuild or rebuild onto, then they can't rebuild. If you have any questions, be sure to ask. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.