December 19, 201312 yr I've just powered off my server to have a little move around in the bedroom so on powering everything back up I've checked to make sure everything is ok but I've been presented with a Red dot and a DISK_DSBL error. I've done a little digging and seen the normal advise is to get a smart report from the drive but I don't fully understand what I'm looking at so I've included it below. From what I see on the understanding smart overviews, I should be looking to replace the drive pronto giving its Reallocated_Sector_Ct being so high? Its under warranty so replacement isn't a problem but I will buy a new drive to install straight away should I be advised to RMA. I'm gonna order a WD 3TB WD30EFRX RED as a replacement today as I don't want to wait and be left unable to get one over the xmas period Oh and I'm running on 4.7... smartctl -a -d ata /dev/sde smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: SAMSUNG HD204UI Serial Number: S2H7J9FZC18181 Firmware Version: 1AQ10001 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 6 Local Time is: Thu Dec 19 13:12:29 2013 GMT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (19920) seconds. Offline data collection capabilities: (0x5b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. SCT capabilities: (0x003f) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 100 100 051 Pre-fail Always - 206 2 Throughput_Performance 0x0026 252 252 000 Old_age Always - 0 3 Spin_Up_Time 0x0023 068 013 025 Pre-fail Always In_the_past 9922 4 Start_Stop_Count 0x0032 099 099 000 Old_age Always - 1909 5 Reallocated_Sector_Ct 0x0033 090 090 010 Pre-fail Always - 1240 7 Seek_Error_Rate 0x002e 252 252 051 Old_age Always - 0 8 Seek_Time_Performance 0x0024 252 252 015 Old_age Offline - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 2014 10 Spin_Retry_Count 0x0032 252 252 051 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 203 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 90 181 Program_Fail_Cnt_Total 0x0022 097 097 000 Old_age Always - 75552674 191 G-Sense_Error_Rate 0x0022 100 100 000 Old_age Always - 1399 192 Power-Off_Retract_Count 0x0022 252 252 000 Old_age Always - 0 194 Temperature_Celsius 0x0002 064 057 000 Old_age Always - 27 (Lifetime Min/Max 12/47) 195 Hardware_ECC_Recovered 0x003a 100 100 000 Old_age Always - 0 196 Reallocated_Event_Count 0x0032 090 090 000 Old_age Always - 1240 197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 1 198 Offline_Uncorrectable 0x0030 252 252 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0036 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x002a 100 100 000 Old_age Always - 3078 223 Load_Retry_Count 0x0032 100 100 000 Old_age Always - 203 225 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 2124 SMART Error Log Version: 1 ATA Error Count: 23 (device log contains only the most recent five errors) CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 23 occurred at disk power-on lifetime: 1744 hours (72 days + 16 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 68 70 d7 00 e0 Error: UNC 104 sectors at LBA = 0x0000d770 = 55152 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 b8 20 d7 00 e0 08 00:07:49.169 READ DMA 25 00 00 60 4c ec e0 08 00:07:49.169 READ DMA EXT 35 00 28 38 4c ec e0 08 00:07:49.169 WRITE DMA EXT 35 00 00 38 48 ec e0 08 00:07:49.169 WRITE DMA EXT 35 00 00 38 44 ec e0 08 00:07:49.169 WRITE DMA EXT Error 22 occurred at disk power-on lifetime: 1531 hours (63 days + 19 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 b0 40 17 00 e0 Error: UNC 176 sectors at LBA = 0x00001740 = 5952 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 d0 20 17 00 e0 08 00:45:15.252 READ DMA 25 00 00 48 ba 75 e0 08 00:45:15.252 READ DMA EXT 35 00 38 10 b9 75 e0 08 00:45:15.252 WRITE DMA EXT 35 00 00 10 b5 75 e0 08 00:45:15.252 WRITE DMA EXT 35 00 00 10 b1 75 e0 08 00:45:15.252 WRITE DMA EXT Error 21 occurred at disk power-on lifetime: 1176 hours (49 days + 0 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 30 98 21 00 e0 Error: UNC 48 sectors at LBA = 0x00002198 = 8600 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 e8 e0 21 00 e0 08 00:35:03.359 READ DMA ef 10 02 00 00 00 a0 08 00:35:03.359 SET FEATURES [Reserved for Serial ATA] 27 00 00 00 00 00 e0 08 00:35:03.359 READ NATIVE MAX ADDRESS EXT ec 00 00 00 00 00 a0 08 00:35:03.359 IDENTIFY DEVICE ef 03 46 00 00 00 a0 08 00:35:03.359 SET FEATURES [set transfer mode] Error 20 occurred at disk power-on lifetime: 1176 hours (49 days + 0 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 30 98 21 00 e0 Error: UNC 48 sectors at LBA = 0x00002198 = 8600 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 e8 e0 21 00 e0 08 00:35:03.347 READ DMA 35 00 88 e0 c4 b8 e0 08 00:35:03.347 WRITE DMA EXT 35 00 00 e0 c0 b8 e0 08 00:35:03.347 WRITE DMA EXT 35 00 00 e0 bc b8 e0 08 00:35:03.347 WRITE DMA EXT 35 00 00 e0 b8 b8 e0 08 00:35:03.347 WRITE DMA EXT Error 19 occurred at disk power-on lifetime: 1149 hours (47 days + 21 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 d8 f8 a8 00 e0 Error: UNC 216 sectors at LBA = 0x0000a8f8 = 43256 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 e8 e8 a8 00 e0 08 00:25:56.180 READ DMA 35 00 00 20 52 1f e0 08 00:25:56.180 WRITE DMA EXT 35 00 00 20 4e 1f e0 08 00:25:56.180 WRITE DMA EXT 35 00 00 20 4a 1f e0 08 00:25:56.180 WRITE DMA EXT 35 00 00 20 46 1f e0 08 00:25:56.180 WRITE DMA EXT SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 708 - # 2 Short offline Completed without error 00% 699 - # 3 Short offline Completed without error 00% 698 - # 4 Short offline Completed without error 00% 693 - # 5 Short offline Completed without error 00% 684 - # 6 Short offline Completed without error 00% 666 - # 7 Short offline Completed without error 00% 98 - # 8 Short offline Completed without error 00% 97 - Note: selective self-test log revision number (0) not 1 implies that no selective self-test has ever been run SMART Selective self-test log data structure revision number 0 Note: revision number not 1 implies that no selective self-test has ever been run SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Completed [00% left] (0-65535) 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. At the start of the smart report its indicates 'SMART overall-health self-assessment test result: PASSED' so I'm not sure if I should just go ahead and follow the advice here http://lime-technology.com/forum/index.php?topic=13908.0 from Joe to rebuild the data onto that disk or if there is anything else I should do. syslog-2013-12-19.txt
December 19, 201312 yr Author Works for me RMA is it... I've disabled all my downloaders etc so nothing should be wrote to the array and I've got a 3TB drive coming next day delivery so should be able to swap it out tomorrow sometime - infact might just power the server off just to be safe lol. Doh - Just realised I've got a 2TB Parity drive so the 3TB won't work as a direct swap for the 2TB failing drive... Bugger. Might have to pickup a 2TB drive too
December 19, 201312 yr There is a process where you could install the 3TB drive as the parity and use the old parity drive as your replacement for the failed drive. I've never used it as I have always just had a replacement of the same size available. I know others have used it and it worked for them. You may have to fail the drive to make it work. Don't want to get the name wrong so didn't post what I believe to be it's name - check wiki. The good thing about getting a 2TB replacement as well as the 3TB is that the old parity or the new replacement drive could become a cold spare so if you have a drive problem in the future you don't have to wait for a drive to arrive. I would also suggest running extensive tests on the new drive(s) before you first use it so that you can be sure it won't die after you have added data to it. I use preclear and run 3 cycles. For a 2 TB drive that could take a couple of days to complete. My 4TB drives were about 5 days - rough estimate.
December 19, 201312 yr Author Cheers for the advice there. I've called and added a 2TB drive to my order too so soon as it lands I'll preclear the 2TB drive over the weekend then once I've got the 2TB drive installed (yay for hotswap bays!) and the data rebuilt to it, I'll look to upgrade the Party to 3TB too. As the drive will be RMA'ed under warranty that should give me a spare drive as a swap over for the future and a 2nd drive so I can install another 2TB drive and gain a bit more space Its actually been a good kick up the backside as I haven't done any monitoring of the smart info so I've just checked all my other drives too (which look ok thankfully)
December 20, 201312 yr Author My replacement drives arrived this morning so I'm going to get that on a preclear this morning. The problem drive passed sea tools diagnostics but I'm still not happy at the error count, spin up retries and spin up time compared to all my other drives so I'm going to return for ram and see what they say. It's in warranty till March 2014 so I expect they will return it without replacement and argue until it's out of warranty so I'll see how it goes. Cheers for the help guys
December 20, 201312 yr not just spin-up-time. It actually FAILED SMART in the past when it was unable to spin-up in a timely manner. Spin_Up_Time 0x0023 068 013 025 Pre-fail Always In_the_past 9922
December 23, 201312 yr Author Oh yeah, cheers Joe L - I didn't even spot that. Lol this is why I always post up when I'm having issues! Well I've done 3x preclear passes and its come back all good on my new 2TB RED disk so its installed, I'm about 75% way through of my disk data rebuild so should be done in a couple hour. I'm looking to move to the latest Version 5 stable build very soon so is there anything I should be looking to do from a maintenance point of view before I look to move? I've also got a precleared ready to go 3TB drive to use as my new parity disk but I'm not sure if its best to upgrade first to v5 software or put the new parity in. My last complete parity check was the 1st month and checked without errors. I presume when I upgrade the parity drive it will be rebuilt from fresh using the data on the drives so is there anything special I should do to check the data disks? I've already checked the SMART on them all and everything looks good with the other drives in my system and I've not experiences any corruption or data loss that I've noticed as a result of the bad sector reallocation business which sparked this all off.
December 23, 201312 yr Oh yeah, cheers Joe L - I didn't even spot that. Lol this is why I always post up when I'm having issues! Well I've done 3x preclear passes and its come back all good on my new 2TB RED disk so its installed, I'm about 75% way through of my disk data rebuild so should be done in a couple hour. I'm looking to move to the latest Version 5 stable build very soon so is there anything I should be looking to do from a maintenance point of view before I look to move? I've also got a precleared ready to go 3TB drive to use as my new parity disk but I'm not sure if its best to upgrade first to v5 software or put the new parity in. My last complete parity check was the 1st month and checked without errors. I presume when I upgrade the parity drive it will be rebuilt from fresh using the data on the drives so is there anything special I should do to check the data disks? I've already checked the SMART on them all and everything looks good with the other drives in my system and I've not experiences any corruption or data loss that I've noticed as a result of the bad sector reallocation business which sparked this all off. Since 4.x series doesn't support 3TB drives you will need to update to 5.0.4 first. Then install the 3TB parity. If you want to make sure everything is ok - you should check with your backups of the files - or if you have none then with a checksum you did on each file before the problem. If you have neither then the only thing you can really do is look at smart reports for the drives. One thing you need to do after the drive is rebuilt is a parity check. The rebuild puts the data on the replacement disk. The following parity check ensures that there are no problems with how the data was written to the disk. Once that is done you've done all you can to ensure your data is fine before upgrading to latest version of unRAID. Once you have 5.0.4 up and running I would do one more parity check on 5.0.4 before upgrading the parity drive. That should give your drives one additional workout before the parity drive upgrade.
December 23, 201312 yr Author Thanks BobPhoenix, You just beat me to it on the 3TB front - I have another small issue I've noticed in my syslog: Dec 23 10:03:15 unraid kernel: attempt to access beyond end of device Dec 23 10:03:15 unraid kernel: sdb1: rw=0, want=625142384, limit=312592707 (Drive related) This is my cache drive which is just used for addon's so having read over a few thread on similar issues refering to gdisk I've noticed someone mention about the 2.2TB drive limit on 4.7. Thank you for the advice on the parity checks during the process, I figure it would be sensible to do something similar but wasn't sure what was the right time to check the parity. My media is all video and music, although I'd cry if I lost everything its not so important that I have a full backup or hash to compare with so its parity check only for me
December 23, 201312 yr Not going to be able to help with the error but since it is on your cache drive probably isn't a problem unless it shows up after the upgrade as well.
December 23, 201312 yr Author With it being the cache drive I may just backup everything on it and format it again. I'm toying with the idea of putting a 120gb SSD in there instead since the addons keep it spinning all the time and the SSD should speed up file operations for extracting and PAR checks etc. Now my data-rebuild has finished it says 'Parity - Last checked on 12/23/2013 8:01:57 PM, finding 0 errors' Does this mean it automatically runs a Parity check after the rebuild or is this just the time the rebuild finished? I've attached the syslog from booting with the new HDD to how things stand now... syslog-2013-12-23.txt
December 24, 201312 yr With it being the cache drive I may just backup everything on it and format it again. I'm toying with the idea of putting a 120gb SSD in there instead since the addons keep it spinning all the time and the SSD should speed up file operations for extracting and PAR checks etc. Now my data-rebuild has finished it says 'Parity - Last checked on 12/23/2013 8:01:57 PM, finding 0 errors' Does this mean it automatically runs a Parity check after the rebuild or is this just the time the rebuild finished? I've attached the syslog from booting with the new HDD to how things stand now... It is a BUG in the message being displayed... the process of computing parity and computing the contents of a replacement disk are exactly the same. unRAID probably should not use the same variable to track their completion, but it does, so you get the misleading message. (it is the time the rebuild completed) You probably should NOW perform a parity check, to make sure what you've re-written in the data-rebuild can be read.
December 24, 201312 yr Author Thanks for clearing that up Joe L. I've ran another parity check overnight and everything has come back clear with no errors. Also attached another syslog just in case. Think I know what the issue is with the cache drive which is giving me: Dec 23 10:03:15 unraid kernel: sdb1: rw=0, want=625142384, limit=312592707 (Drive related) I'm pretty sure when I moved from a 160GB to a 320GB cache drive I just cloned the smaller drive to the larger one but it never expanded the size to the whole disk and I had forgot about it in all honesty. Is the easiest way to resolve the issue to backup all the addons running from the drive and format it again, or is there some commandline tool I can use to expand the partition out to use the full available size of the disk? syslog-2013-12-24.txt
December 25, 201312 yr Thanks for clearing that up Joe L. I've ran another parity check overnight and everything has come back clear with no errors. Also attached another syslog just in case. Think I know what the issue is with the cache drive which is giving me: Dec 23 10:03:15 unraid kernel: sdb1: rw=0, want=625142384, limit=312592707 (Drive related) I'm pretty sure when I moved from a 160GB to a 320GB cache drive I just cloned the smaller drive to the larger one but it never expanded the size to the whole disk and I had forgot about it in all honesty. Is the easiest way to resolve the issue to backup all the addons running from the drive and format it again, or is there some commandline tool I can use to expand the partition out to use the full available size of the disk? I'm about to upgrade my cache drive on my N54L (later this week). I plan on just mounting the new drive manually then using midnight commander to copy my APPS folder to the manually mounted drive. Once the copy is complete I will just unmount the manually mounted drive. Then in the GUI I will change the cache drive from the old drive to the new drive and restart the array. I believe that is all I did before when I changed cache drives but if it doesn't work I will still have the old cache drive that I could remount again. I do plan on running mover to get all the files NOT in my apps folder off the cache drive first. Otherwise I will have to copy them too.
December 26, 201312 yr Author Thanks for the info. Its looking like I'm pretty much sorted. I've got my new drive in and I've done a data-rebuild, parity check, upgrade to latest v5 stable, parity check and everything has come back error free. Also all my apps on the cache drive /.custom area are working without issue (other then a few permissions issues but easily fixed). Running another pass of the precheck on the 3TB drive which will become the new parity so once this finished (maybe tomorrow) I'll add the new parity and should be good to go for the upcoming future. I do have the following highlighted in unmenu's syslog highlighted red: Dec 25 02:41:14 unraid kernel: ata17.00: exception Emask 0x32 SAct 0x0 SErr 0x0 action 0xe frozen (Errors) Dec 25 02:41:14 unraid kernel: ata17.00: irq_stat 0xffffffff, unknown FIS 00000000 00000000 00000000 00000000, host bus (Drive related) Dec 25 02:41:14 unraid kernel: ata17.00: failed command: READ DMA EXT (Minor Issues) Dec 25 02:41:14 unraid kernel: ata17.00: cmd 25/00:50:0f:ee:d0/00:00:53:00:00/e0 tag 0 dma 40960 in (Drive related) Dec 25 02:41:14 unraid kernel: res 50/00:00:5e:ee:d0/00:00:53:00:00/e0 Emask 0x32 (host bus error) (Errors) Dec 25 02:41:14 unraid kernel: ata17.00: status: { DRDY } (Drive related) Dec 25 02:41:14 unraid kernel: ata17: hard resetting link (Minor Issues) Dec 25 02:41:15 unraid kernel: ata17: SATA link up 3.0 Gbps (SStatus 123 SControl 300) (Drive related) Dec 25 02:41:15 unraid kernel: ata17.00: configured for UDMA/133 (Drive related) Dec 25 02:41:15 unraid kernel: ata17: EH complete (Drive related) Anything to worry about?
December 27, 201312 yr Thanks for clearing that up Joe L. I've ran another parity check overnight and everything has come back clear with no errors. Also attached another syslog just in case. Think I know what the issue is with the cache drive which is giving me: Dec 23 10:03:15 unraid kernel: sdb1: rw=0, want=625142384, limit=312592707 (Drive related) I'm pretty sure when I moved from a 160GB to a 320GB cache drive I just cloned the smaller drive to the larger one but it never expanded the size to the whole disk and I had forgot about it in all honesty. Is the easiest way to resolve the issue to backup all the addons running from the drive and format it again, or is there some commandline tool I can use to expand the partition out to use the full available size of the disk? there are tools, but if used incorrectly you could lose your data. Be very careful to NOT change the start of the first partition. You must FIRST expand the partition using fdisk, or cfdisk, (or something like them. ) to use the full physical size of the drive, and then invoke resize_reiserfs to have it expand the file-system to fill the expanded partition. http://linux.about.com/library/cmd/blcmdl8_resize_reiserfs.htm Again... if the first partition currently starts on sector 63, DO NOT CHANGE IT, as you'll lose the existing contents of the file-system. If it starts on sector 64, DO NOT CHANGE IT, or you'll lose the contents of the file-system... Basically, DO NOT CHANGE THE STARTING CYLINDER of the first partition. You can use the fdisk -lu /dev/sdX command to learn of the number of the 1st partition starting cylinder. (did I mention it should NOT be changed)
December 28, 201312 yr Author Does it repeat? I've been moving my kit around as my desk underneath was in silly need of a rearrange so I've built a bit of a rackmount case to house the UPS and switch. As part of this I had to power down the unraid box to move it out the way and on reconnecting / powering back up I've checked the log again and this time there is no instance of the error so I think I'll just need to keep an eye out. there are tools, but if used incorrectly you could lose your data. Thank Joe L. I think in this case I'll live with it then as all my downloaders are working and I don't use the cache drives as an actual 'cache' as I do very little copying to the unraid box. At some point early this year I'm going to look to replace this cache with a 240/256GB SSD so for now I'll live with it
Archived
This topic is now archived and is closed to further replies.