June 13, 200917 yr Hi, I'm pretty new to using UnRAID...first time poster. Brought a server online at the end of last year, and have been enjoying it ever since. I'm using version 4.3.3 and been really happy with it. I recently went to the webserver and noticed that my parity drive recorded 40 errors. I'm attaching the syslog, but I noticed a slew of errors occurring on June 5th. I think I was copying a slew of files over to the server from multiple machines that night. I just wanted some help in trying to understand the syslog a little bit more. Thanks for any help. I'm currently running a parity check on the system.
June 13, 200917 yr You are doing pretty well in understanding the syslog. there have been a series of errors when attempting to read the parity drive /dev/sda You should run a smartctl report on the drive to learn its health and post its output. smartctl -a -d ata /dev/sda It could be that you had a series of unreadable sectors on the disk, or it could be a loose/bad cable, or any number of other hardware issues. Since the errors have not continued there is some hope. (Although it does make it much easier to find and isolate the root cause if the errors were still occurring) If you get an error message stating a library file is missing when you run the "smartctl" command, see this post on how to install the missing library: http://lime-technology.com/forum/index.php?topic=2595.msg25343#msg25343 You do not need to wait for the parity check to finish before running the smartctl command. Post its output for analysis. You will be looking for reallocated sectors, and sectors pending reallocation. Both of those indicate the drive;s hardware found problems with a sector and relocated the data on a subsequent "write" to the drive, or is waiting for a subsequent write to reallocate the data. A parity drive with an unreadable sector will not be able to be used to recover other drives properly if they fail. A few reallocated sectors is OK, but pending reallocation is not (at least not on a parity drive, as nothing would write to those same sectors unless you deleted and re-loaded the involved sectors somehow.) If the sectors reallocate, and no more are detected, then you'll probably be fine. If you see the errors in the smartctl report continuing to increase in count (more sectors needing reallocation), you'll probably want to RMA the drive. Joe L.
June 13, 200917 yr Author Yeah, this is where I am now starting to get confused. After I let the server do a parity check, it reported under the "Disk Status" that the parity drive had 1473 errors , yet the check came up with 0 errors. I did as you suggested Joe with the smartctl command, and the drive reported a bunch of different errors shown below. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_ FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 115 099 006 Pre-fail Always - 90865157 3 Spin_Up_Time 0x0003 091 090 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 14 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 1 7 Seek_Error_Rate 0x000f 068 060 030 Pre-fail Always - 6514230 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 4062 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 1 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 8 184 Unknown_Attribute 0x0032 100 100 099 Old_age Always - 0 187 Unknown_Attribute 0x0032 001 001 000 Old_age Always - 154 188 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0 189 Unknown_Attribute 0x003a 100 100 000 Old_age Always - 0 190 Unknown_Attribute 0x0022 071 070 045 Old_age Always - 504889373 194 Temperature_Celsius 0x0022 029 040 000 Old_age Always - 29 (Lifetime Min/Max 0/15) 195 Hardware_ECC_Recovered 0x001a 046 016 000 Old_age Always - 90865157 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 SMART Error Log Version: 1 ATA Error Count: 154 (device log contains only the most recent five errors) CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 154 occurred at disk power-on lifetime: 4052 hours (168 days + 20 hours) When the command that caused the error occurred, the device was active or idle . After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 48d+12:59:25.442 READ DMA EXT 27 00 00 00 00 00 e0 00 48d+12:59:25.376 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 48d+12:59:25.356 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 48d+12:59:25.337 SET FEATURES [set transfer mode] 27 00 00 00 00 00 e0 00 48d+12:59:25.316 READ NATIVE MAX ADDRESS EXT Error 153 occurred at disk power-on lifetime: 4052 hours (168 days + 20 hours) When the command that caused the error occurred, the device was active or idle . After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 48d+12:59:22.361 READ DMA EXT 27 00 00 00 00 00 e0 00 48d+12:59:22.295 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 48d+12:59:22.275 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 48d+12:59:22.255 SET FEATURES [set transfer mode] 27 00 00 00 00 00 e0 00 48d+12:59:22.235 READ NATIVE MAX ADDRESS EXT Error 152 occurred at disk power-on lifetime: 4052 hours (168 days + 20 hours) When the command that caused the error occurred, the device was active or idle . After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 48d+12:59:19.280 READ DMA EXT 27 00 00 00 00 00 e0 00 48d+12:59:19.214 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 48d+12:59:19.193 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 48d+12:59:19.174 SET FEATURES [set transfer mode] 27 00 00 00 00 00 e0 00 48d+12:59:19.153 READ NATIVE MAX ADDRESS EXT Error 151 occurred at disk power-on lifetime: 4052 hours (168 days + 20 hours) When the command that caused the error occurred, the device was active or idle . After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 48d+12:59:16.198 READ DMA EXT 27 00 00 00 00 00 e0 00 48d+12:59:16.132 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 48d+12:59:16.112 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 48d+12:59:16.092 SET FEATURES [set transfer mode] 27 00 00 00 00 00 e0 00 48d+12:59:16.072 READ NATIVE MAX ADDRESS EXT Error 150 occurred at disk power-on lifetime: 4052 hours (168 days + 20 hours) When the command that caused the error occurred, the device was active or idle . After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 ff ff ff ef 00 48d+12:59:13.137 READ DMA EXT 27 00 00 00 00 00 e0 00 48d+12:59:13.071 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 00 48d+12:59:13.050 IDENTIFY DEVICE ef 03 46 00 00 00 a0 00 48d+12:59:13.040 SET FEATURES [set transfer mode] 27 00 00 00 00 00 e0 00 48d+12:59:12.980 READ NATIVE MAX ADDRESS EXT SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. I ended up clearing the statistics and did another parity check. This time I have 0 errors under the "Disk Status", so I don't know what to make of it, and the parity check came back with 0 errors. I've attached the updated syslog (syslog_130609.txt). I had to edit it to get the file size down, so I included only today's date. Also where alot of the errors are repetitive, I've left the beginning bunch and end bunch but took out the errors in between. Thanks for any more info anyone can provide. Right now this is a little over my head so it is very much appreciated.
June 13, 200917 yr There are two different kinds of "errors" involved in your report.. First are "read" errors associated with a specific disk drive. Those occur when a request to read a specific disk fails. When this occurs, from what I understand, unRAID will re-try to read the data. If un-successful, the data that should have been returned will be re-created by reading the parity disk and all the other data disks. It will then write the data to the disk it could not read. If the sector on the disk being read was un-readable, and the disk hardware detected that, it would normally mark the sector pending reallocation. Before reallocation, on the next "write" to that same sector it would first try to write to the sector it initially could not read and if it could then read back the data, it would not need to do anything more. If it could not read back the data it would relocate the data elsewhere on the disk as a relocated sector. Your disk's smart report shows 1 reallocated sector. you have 0 sectors pending reallocation. Together those seem to indicate a drive that is in pretty good shape. The other "RAW" values for Raw_Read_Error_Rate and Hardware_ECC (error correcting code) Recovered Data are only meaningful to the manufacturer. All disks rely on hardware error correction... it is only when the hardware error correction gets to where it cannot that a drive will fail SMART tests. You do have other errors listed though... All failures when trying to read data from the drive. To me, they could easily be an indication of a bad/poor quality cable to the drive. That would be my first suspect. The second type of error, "parity errors" occur when the value stored on the parity drive is not as expected based on reading the data drives. I've only ever had that type of error once on my server, when my power company dropped power to the area in a storm and the parity drive was not written to as I was writing to the array. You should never see "parity check" errors unless you have a improper shut-down. You can of course see "read" errors on any drive at any time. In the same way, you can have "write" errors on any drive at any time, but unRAID will take a drive off-line if a single write to it fails. You'll see the indicator next to a drive turn red when this occurs. (hopefully, you'll never see this... but alas, given enough time, and enough disk drives, one will fail.) Oh yes, I'm impressed about one thing... the drive seems to be reporting 169 days of up-time and only 8 power cycles... Apparently your power company is reliable and do not need to shut the server down very often. Joe L.
June 14, 200917 yr Author Gees Joe, thanks for the info. That's a great wealth of information you gave me....thank you. Curious, what cables would you suggest to use? Currently I'm using the standard off-the-shelf SATA interface cable....around 18" in length. Please don't say Monster (if they indeed even sell SATA cable)...I just can't go there . Thanks again for the excellent info. FYI, my UnRAID server consists of the following: Motherboard: ECS 945GCT-M/1333 Socket 775 w/video, and LAN CPU: Intel Celeron 440 RAM: Kingston ValueRAM 2x512MB KVR667D2N5K2/1G Cooler: Artic Cooling Alpine 7 Pro Power Supply: Antec EA-650 Hard Drives: 2x Seagate Barracuda ST3750630AS and 1x Seagate Barracuda ST3500418AS Planning on upgrading to most likely Pro one day.
June 14, 200917 yr Gees Joe, thanks for the info. That's a great wealth of information you gave me....thank you. Curious, what cables would you suggest to use? Currently I'm using the standard off-the-shelf SATA interface cable....around 18" in length. Please don't say Monster (if they indeed even sell SATA cable)...I just can't go there . Thanks again for the excellent info. FYI, my UnRAID server consists of the following: Motherboard: ECS 945GCT-M/1333 Socket 775 w/video, and LAN CPU: Intel Celeron 440 RAM: Kingston ValueRAM 2x512MB KVR667D2N5K2/1G Cooler: Artic Cooling Alpine 7 Pro Power Supply: Antec EA-650 Hard Drives: 2x Seagate Barracuda ST3750630AS and 1x Seagate Barracuda ST3500418AS Planning on upgrading to most likely Pro one day. Not monster... even if they did make one... I only have one monster product in my home, and it was a freebe gift... and it is not a cable. Try monoprice.com Their prices are very good, and from what I've read in this forum and others, the quality very good. Check out this link where I searched for sata2 and locking (the locking type tend to not get loose as easily as the non-locking type): http://www.monoprice.com/products/search.asp?keyword=sata2+locking&x=0&y=0 They are roughly 1.53 in single quantities, or $1.47 for 2 or more... dropping to $1.42 if you buy 10... I'd get 10 at that price... (and I'll bet that $14.20 for 10 18" locking cables is about as good a deal as you'll find, you can even choose the color.) Joe L.
Archived
This topic is now archived and is closed to further replies.