April 11, 201412 yr Hey JoeL I saw you updated the preclear.sh script to version 1.15. Does it include BJP's faster post-processing method, or the traditional method? I have not included his compiled module. It fixes the issue with unRAID 6.X. I have not had time to investigate the GPL requirements of distributing his compiled program, and I suspect it also requires distributing his source code which was based on "dd". Joe L.
April 11, 201412 yr Hey JoeL I saw you updated the preclear.sh script to version 1.15. Does it include BJP's faster post-processing method, or the traditional method? I have not included his compiled module. It fixes the issue with unRAID 6.X. I have not had time to investigate the GPL requirements of distributing his compiled program, and I suspect it also requires distributing his source code which was based on "dd". Joe L. Thanks for clarifying.
April 13, 201412 yr Help! I tried preclear_disk.sh -A /dev/sdc and I got this when running my preclear on my new HD: Elapsed Time: 15:10:25 Restarting udevd is STRONGLY discouraged and not supported. If you are sure you want to do this, use 'force-restart' instead. ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying the clear has been successful. = Elapsed Time: 15:10:35 ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying if the MBR is cleared. DONE = Elapsed Time: 15:10:40 ========================================================================1.15 == == SORRY: Disk /dev/sdc MBR could NOT be precleared == == out4= 00000 == out5= 00000 ============================================================================ 0000000 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00159971 s, 0.0 kB/s I then tried preclear_disk.sh -t /dev/sdc and got: Pre-Clear unRAID Disk /dev/sdc ################################################################## 1.15 Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4E1275395 LU WWN Device Id: 5 0014ee 2b4968c23 Firmware Version: 80.00A80 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes 255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table ######################################################################## failed test 1 failed test 2 00000 00000 00000 00000 failed test 3 00000 00000 00000 00000 failed test 5 failed test 6 ========================================================================1.15 == == Disk /dev/sdc is NOT precleared == 0 0 4294967295 ============================================================================ Is my drive bad? What should I do? Thanks. .
April 14, 201412 yr Help! I tried preclear_disk.sh -A /dev/sdc and I got this when running my preclear on my new HD: Elapsed Time: 15:10:25 Restarting udevd is STRONGLY discouraged and not supported. If you are sure you want to do this, use 'force-restart' instead. ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying the clear has been successful. = Elapsed Time: 15:10:35 ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying if the MBR is cleared. DONE = Elapsed Time: 15:10:40 ========================================================================1.15 == == SORRY: Disk /dev/sdc MBR could NOT be precleared == == out4= 00000 == out5= 00000 ============================================================================ 0000000 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00159971 s, 0.0 kB/s I then tried preclear_disk.sh -t /dev/sdc and got: Pre-Clear unRAID Disk /dev/sdc ################################################################## 1.15 Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4E1275395 LU WWN Device Id: 5 0014ee 2b4968c23 Firmware Version: 80.00A80 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes 255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table ######################################################################## failed test 1 failed test 2 00000 00000 00000 00000 failed test 3 00000 00000 00000 00000 failed test 5 failed test 6 ========================================================================1.15 == == Disk /dev/sdc is NOT precleared == 0 0 4294967295 ============================================================================ Is my drive bad? What should I do? Thanks. It appears as if the drive is no longer readable. You might have a loose cable. Stop your array, power down, re-seat the cables, and then try once more. Joe L. It appears as if the drive is no longer readable. You might have a loose cable. Stop your array, power down, re-seat the cables, and then try once more. Joe L.
April 16, 201412 yr I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Edited February 3, 20215 yr by Sean M.
April 16, 201412 yr I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Thanks, Sean Wow. A very interesting case. So the two big drive issues we monitor are reallocated sectors and pending reallocations. If there are pending reallocation, then the preclear procedure, as it enounters each one, should either clear it (drive decides no reallocation was needed after all), or reallocate it (using one of a limited number of spare sectors reserved for this purpose). Your before and afters are listed below: 368 pending, and 0 actual reallocations - begin 232 pending, and 0 actual reallocations - end It might have been predicted you'd end up with 0 pending, 368 reallocations If this had happened I'd advise you to preclear again and see if any more pending or reallocations occur. If they don't after a few tries, the drive may be ok to use. But if they keep shifting I would be nervous and RMA it. But that's not what happened. It may have been that during the preclear those 368 sectors pending reallocations were determined to be ok, but 232 others were found to be bad. Or maybe some of the 368 were found to be bad, but not bad enough to cause an actual reallocation. Not knowing exactly how the SMART logic works makes figuring out why this happened impossible. The good news is you don't have any actual reallocated sectors. I have seen some drives exhibit a confusing phenomenon similar to yours where a bunch of pending sectors develop, but then suddenly after a parity check, the pending count goes to 0, with no actual reallocations, and they stay at 0 and work great. I think this points to a bug of some kind in the SMART logic, but these drives I tend to trust. But I don't really trust this. Seagates don't have a great reputation. You could try another preclear cycle and see what happens. My guess is the pending sector counts will continue to change, and the jury is out if you are going to get any actual reallocations. I cannot tell you this is a healthy or an unhealthy situation - only that it would make me nervous. Short answer, I'd probably RMA this drive. That or watch it through a few preclears, and if nothing reallocates and there are no read errors in the logs, use it.
April 16, 201412 yr On 4/16/2014 at 1:09 PM, bjp999 said: Quote I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Wow. A very interesting case. So the two big drive issues we monitor are reallocated sectors and pending reallocations. If there are pending reallocation, then the preclear procedure, as it enounters each one, should either clear it (drive decides no reallocation was needed after all), or reallocate it (using one of a limited number of spare sectors reserved for this purpose). Your before and afters are listed below: 368 pending, and 0 actual reallocations - begin 232 pending, and 0 actual reallocations - end It might have been predicted you'd end up with 0 pending, 368 reallocations If this had happened I'd advise you to preclear again and see if any more pending or reallocations occur. If they don't after a few tries, the drive may be ok to use. But if they keep shifting I would be nervous and RMA it. But that's not what happened. It may have been that during the preclear those 368 sectors pending reallocations were determined to be ok, but 232 others were found to be bad. Or maybe some of the 368 were found to be bad, but not bad enough to cause an actual reallocation. Not knowing exactly how the SMART logic works makes figuring out why this happened impossible. The good news is you don't have any actual reallocated sectors. I have seen some drives exhibit a confusing phenomenon similar to yours where a bunch of pending sectors develop, but then suddenly after a parity check, the pending count goes to 0, with no actual reallocations, and they stay at 0 and work great. I think this points to a bug of some kind in the SMART logic, but these drives I tend to trust. But I don't really trust this. Seagates don't have a great reputation. You could try another preclear cycle and see what happens. My guess is the pending sector counts will continue to change, and the jury is out if you are going to get any actual reallocations. I cannot tell you this is a healthy or an unhealthy situation - only that it would make me nervous. Short answer, I'd probably RMA this drive. That or watch it through a few preclears, and if nothing reallocates and there are no read errors in the logs, use it. Well now, was not expecting it to be such a phenomenon that's for sure! Your insights are very appreciated. I believe this drive came with a 2 year limited warranty, purchased from amazon on 11.25.2012. I have used it to house my media for my plex server, inside my desktop, since its purchase so it definitely got some decent usage. It was almost full which was why I started looking for another solution and ended up with unRAID. The model number on this one is (ST3000DM001). I only ran one pre-clear as I wasn't expecting any problems. I am currently running a pre-clear on one of the two older models I planned to place in the build (ST33000651AS) which were purchased from amazon on 7.5.2011. I'm curious as to the responses from those drives as they were used in an external backup raid 1 case. I will definitely post the output of those as well but was only planning on running 1 pre-clear pass on each drive since they had all been used before. I haven't setup anything else up on my unRAID build, other than the preliminary add-on installs (unMENU & Control Panel) until these drives were ready to be installed, the pre-clear time is lengthy on 3TB drives to say the least so running it over and over again on 1 drive just continues to delay me. I'm not opposed to this as I am very much a draw twice cut once person. I have personally never had to RMA a HDD in the 3ish years I've been fiddling with computers personally now so I'm not quite sure what to expect or even if there would be any replacement based upon the time-frame and drives themselves. Could you elaborate a bit on if RMA'ing this drive or any of the three drives I have would be worth it? Seriously, thanks for all the help. These forums continually surprise me the more I read as to how helpful and knowledge people are. Edit: Did not realize the command output included drive model #, none the less I'll keep it included for continuity sake. Edited February 3, 20215 yr by Sean M.
April 20, 201412 yr SDE & SDC are a bit older drives but appear to have come through clear. Images Below: http://i.imgur.com/1N0YkUj.pnghttp://i.imgur.com/lpARRAr.png SDD is the one I posted about before that BJP gave some advice on, I have attached the second pre-clear. Just want to get some thoughts before I request an RMA since that drive appears to still be under warranty. Images Below: http://i.imgur.com/zjd2bE1.png Is there anything troublesome I should look out for? Edited February 3, 20215 yr by Sean M.
April 21, 201412 yr The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want.
April 21, 201412 yr The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want. Just submitted an RMA to Seagate to get rid of the questionable one. Thanks for the advice, greatly appreciated!
April 21, 201412 yr The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want. Bad cabling cannot cause reallocated or pending sectors. I have found there are 2 types of pending sectors - those that generate read errors (true bad sectors) and those that don't and invisibly waffle back and forth without any reallocations happening. The first situations unRaid will typically fix. The first read error will cause unRaid to reconstruct the sector, write it back, and the sector will reallocate. In theory this should be how pending sectors should behave. But these waffling pending sectors (which seem unique to Seagate but may happen on other brands too) are a mystery. My experience is until they actually start to reallocate they are just noise. I would run 4-5 preclear cycles and see what happens. Can't argue with RMAing this one with so many pendings though. We just don't have enough insight into what's happening in the drives firmware to understands the whys and wherefores.
May 27, 201412 yr I would like to add a new 4TB Hitachi drive to my system as a parity drive. My current parity is a 3TB WD red. I think that it would be quicker if I precleared the drive first. My system has been running stable for a few years and, when I upgraded to version 5.0 I kept the default partition format as MBR: unalighed. How should I proceed with the new drive? Should I preclear it as an ADVANCED FORMAT drive (-A) or leave all the same, (-a)? I am sorry if I am asking a question that has been already asked, but I cannot find the answer. Thank you, Lev
May 27, 201412 yr Do you plan to introduce the new drive to replace a smaller parity, and then add the old parity drive as a data drive to the array? Refer to the "Replace parity" how-to link in my sig for some instructions.
May 27, 201412 yr Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev
May 27, 201412 yr Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev With ANY drive over 2.2TB, both the -A and -a options are silently ignored. Use them as you like, but the disk MUST (and will) be partitioned with a GPT partition, and it will have a "protective" MBR partition installed starting on sector 1 to convince older utilities the disk is entirely allocated. Make sure you use the most current version of the preclear script. Older versions did not properly set the preclear signature on some versions of unRAID. You can type preclear_disk.sh -v to learn the version (current version as of this post is 1.15) Joe L.
May 27, 201412 yr Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev Did you look at my link/instructions about replacing parity? It answers your question about when to preclear (see step 0)
May 28, 201412 yr However, should I preclear the new drive prior to adding it in as a new parity? YesWill that make the parity rebuild faster?No, but you'll trust the disk more. (about 1 in 5 fail in the first hours of operation... do you feel lucky?) In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev If the drive is less than 2.2TB, and NOT a WD-EARS drive with a jumper added, use -A only if an older <=2TB WD-EARS drive with the jumper added should you use "-a" Joe L.
May 28, 201412 yr However, should I preclear the new drive prior to adding it in as a new parity? YesWill that make the parity rebuild faster?No, but you'll trust the disk more. (about 1 in 5 fail in the first hours of operation... do you feel lucky?) In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev If the drive is less than 2.2TB, and NOT a WD-EARS drive with a jumper added, use -A only if an older <=2TB WD-EARS drive with the jumper added should you use "-a" Joe L. Excellent! This is exactly what I was looking for. Thank you very much. Lev
May 30, 201412 yr just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. preclear_start_WD-WCC4E1579125_2014-05-29.txt preclear_finish_WD-WCC4E1579125_2014-05-29.txt preclear_rpt_WD-WCC4E1579125_2014-05-29.txt
May 31, 201412 yr just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. looks great.
May 31, 201412 yr just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. looks great. Thanks
June 6, 201412 yr Author Dumb question.... Do I need to pre-clear a new drive if it's going into the parity slot? I think the answer is no... but... I will pre-clear anyway to test the drive... but is it necessary? isn't the drive completely written anyway when it generates the new parity? Jim
June 6, 201412 yr Dumb question.... Do I need to pre-clear a new drive if it's going into the parity slot? I think the answer is no... but... I will pre-clear anyway to test the drive... but is it necessary? isn't the drive completely written anyway when it generates the new parity? Jim You do not need to, BUT, you should, as you said, to test the drive and to get past the early part of the "bathtub curve" About 1 in 5 drives seems to fail initially or in the first hours of operation. Do you feel lucky? Yes, the drive is completely written, but it is not read until you perform a subsequent parity check. You will not know of un-readable sectors until after you've performed those two steps. After the subsequent parity check you can get a SMART report. If you see any sectors pending re-allocation it is necessary to write again to the sectors marked as un-readable. This will either make them readable, or cause the SMART firmware to re-allocate them. Then you'll again need to perform another parity check to see if the re-allocations were all successful and readable after being re-allocated. Sounds like a lot of reading and writing... It is... and that is almost exactly what the preclear script does for you. Joe L.
June 6, 201412 yr Author That's what I figured. It was more of a "it won't save me any time" type of question. Like if I had a known good drive already that I had already used.. it doesn't save me anytime to pre-clear it! 9 hours to do the pre read on my 4TB disk! (122MB/s calculated) This is gonna take a while! :-) Jim
June 6, 201412 yr The last 4TB I did took about 100 hours. I'm sure this varies some from one system to another. It could save you a bunch of time if bad sectors are found though.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.