Joe L. Posted April 11, 2014 Share Posted April 11, 2014 Hey JoeL I saw you updated the preclear.sh script to version 1.15. Does it include BJP's faster post-processing method, or the traditional method? I have not included his compiled module. It fixes the issue with unRAID 6.X. I have not had time to investigate the GPL requirements of distributing his compiled program, and I suspect it also requires distributing his source code which was based on "dd". Joe L. Quote Link to comment
bkastner Posted April 11, 2014 Share Posted April 11, 2014 Hey JoeL I saw you updated the preclear.sh script to version 1.15. Does it include BJP's faster post-processing method, or the traditional method? I have not included his compiled module. It fixes the issue with unRAID 6.X. I have not had time to investigate the GPL requirements of distributing his compiled program, and I suspect it also requires distributing his source code which was based on "dd". Joe L. Thanks for clarifying. Quote Link to comment
W1nks Posted April 13, 2014 Share Posted April 13, 2014 Help! I tried preclear_disk.sh -A /dev/sdc and I got this when running my preclear on my new HD: Elapsed Time: 15:10:25 Restarting udevd is STRONGLY discouraged and not supported. If you are sure you want to do this, use 'force-restart' instead. ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying the clear has been successful. = Elapsed Time: 15:10:35 ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying if the MBR is cleared. DONE = Elapsed Time: 15:10:40 ========================================================================1.15 == == SORRY: Disk /dev/sdc MBR could NOT be precleared == == out4= 00000 == out5= 00000 ============================================================================ 0000000 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00159971 s, 0.0 kB/s I then tried preclear_disk.sh -t /dev/sdc and got: Pre-Clear unRAID Disk /dev/sdc ################################################################## 1.15 Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4E1275395 LU WWN Device Id: 5 0014ee 2b4968c23 Firmware Version: 80.00A80 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes 255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table ######################################################################## failed test 1 failed test 2 00000 00000 00000 00000 failed test 3 00000 00000 00000 00000 failed test 5 failed test 6 ========================================================================1.15 == == Disk /dev/sdc is NOT precleared == 0 0 4294967295 ============================================================================ Is my drive bad? What should I do? Thanks. . Quote Link to comment
Joe L. Posted April 14, 2014 Share Posted April 14, 2014 Help! I tried preclear_disk.sh -A /dev/sdc and I got this when running my preclear on my new HD: Elapsed Time: 15:10:25 Restarting udevd is STRONGLY discouraged and not supported. If you are sure you want to do this, use 'force-restart' instead. ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying the clear has been successful. = Elapsed Time: 15:10:35 ================================================================== 1.15 = unRAID server Pre-Clear disk /dev/sdc = cycle 1 of 1, partition start on sector 1 = Disk Pre-Clear-Read completed DONE = Step 1 of 10 - Copying zeros to first 2048k bytes DONE = Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE = Step 3 of 10 - Disk is now cleared from MBR onward. DONE = Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4 DONE = Step 5 of 10 - Clearing MBR code area DONE = Step 6 of 10 - Setting MBR signature bytes DONE = Step 7 of 10 - Setting partition 1 to precleared state DONE = Step 8 of 10 - Notifying kernel we changed the partitioning DONE = Step 9 of 10 - Creating the /dev/disk/by* entries DONE = Step 10 of 10 - Verifying if the MBR is cleared. DONE = Elapsed Time: 15:10:40 ========================================================================1.15 == == SORRY: Disk /dev/sdc MBR could NOT be precleared == == out4= 00000 == out5= 00000 ============================================================================ 0000000 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00159971 s, 0.0 kB/s I then tried preclear_disk.sh -t /dev/sdc and got: Pre-Clear unRAID Disk /dev/sdc ################################################################## 1.15 Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4E1275395 LU WWN Device Id: 5 0014ee 2b4968c23 Firmware Version: 80.00A80 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes 255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table ######################################################################## failed test 1 failed test 2 00000 00000 00000 00000 failed test 3 00000 00000 00000 00000 failed test 5 failed test 6 ========================================================================1.15 == == Disk /dev/sdc is NOT precleared == 0 0 4294967295 ============================================================================ Is my drive bad? What should I do? Thanks. It appears as if the drive is no longer readable. You might have a loose cable. Stop your array, power down, re-seat the cables, and then try once more. Joe L. It appears as if the drive is no longer readable. You might have a loose cable. Stop your array, power down, re-seat the cables, and then try once more. Joe L. Quote Link to comment
Sean M. Posted April 16, 2014 Share Posted April 16, 2014 (edited) I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Edited February 3, 2021 by Sean M. Quote Link to comment
SSD Posted April 16, 2014 Share Posted April 16, 2014 I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Thanks, Sean Wow. A very interesting case. So the two big drive issues we monitor are reallocated sectors and pending reallocations. If there are pending reallocation, then the preclear procedure, as it enounters each one, should either clear it (drive decides no reallocation was needed after all), or reallocate it (using one of a limited number of spare sectors reserved for this purpose). Your before and afters are listed below: 368 pending, and 0 actual reallocations - begin 232 pending, and 0 actual reallocations - end It might have been predicted you'd end up with 0 pending, 368 reallocations If this had happened I'd advise you to preclear again and see if any more pending or reallocations occur. If they don't after a few tries, the drive may be ok to use. But if they keep shifting I would be nervous and RMA it. But that's not what happened. It may have been that during the preclear those 368 sectors pending reallocations were determined to be ok, but 232 others were found to be bad. Or maybe some of the 368 were found to be bad, but not bad enough to cause an actual reallocation. Not knowing exactly how the SMART logic works makes figuring out why this happened impossible. The good news is you don't have any actual reallocated sectors. I have seen some drives exhibit a confusing phenomenon similar to yours where a bunch of pending sectors develop, but then suddenly after a parity check, the pending count goes to 0, with no actual reallocations, and they stay at 0 and work great. I think this points to a bug of some kind in the SMART logic, but these drives I tend to trust. But I don't really trust this. Seagates don't have a great reputation. You could try another preclear cycle and see what happens. My guess is the pending sector counts will continue to change, and the jury is out if you are going to get any actual reallocations. I cannot tell you this is a healthy or an unhealthy situation - only that it would make me nervous. Short answer, I'd probably RMA this drive. That or watch it through a few preclears, and if nothing reallocates and there are no read errors in the logs, use it. Quote Link to comment
Sean M. Posted April 16, 2014 Share Posted April 16, 2014 (edited) On 4/16/2014 at 1:09 PM, bjp999 said: Quote I apologize for not pasting the direct code output, I grabbed a screen-cap before leaving the house this AM so I could post it later. This is the first pre-clear and I wanted to check my results on the drive. 3TB Seagate barracuda, about 1+ year of previous usage, ran pre-clear script once. Output in the attached image. Wow. A very interesting case. So the two big drive issues we monitor are reallocated sectors and pending reallocations. If there are pending reallocation, then the preclear procedure, as it enounters each one, should either clear it (drive decides no reallocation was needed after all), or reallocate it (using one of a limited number of spare sectors reserved for this purpose). Your before and afters are listed below: 368 pending, and 0 actual reallocations - begin 232 pending, and 0 actual reallocations - end It might have been predicted you'd end up with 0 pending, 368 reallocations If this had happened I'd advise you to preclear again and see if any more pending or reallocations occur. If they don't after a few tries, the drive may be ok to use. But if they keep shifting I would be nervous and RMA it. But that's not what happened. It may have been that during the preclear those 368 sectors pending reallocations were determined to be ok, but 232 others were found to be bad. Or maybe some of the 368 were found to be bad, but not bad enough to cause an actual reallocation. Not knowing exactly how the SMART logic works makes figuring out why this happened impossible. The good news is you don't have any actual reallocated sectors. I have seen some drives exhibit a confusing phenomenon similar to yours where a bunch of pending sectors develop, but then suddenly after a parity check, the pending count goes to 0, with no actual reallocations, and they stay at 0 and work great. I think this points to a bug of some kind in the SMART logic, but these drives I tend to trust. But I don't really trust this. Seagates don't have a great reputation. You could try another preclear cycle and see what happens. My guess is the pending sector counts will continue to change, and the jury is out if you are going to get any actual reallocations. I cannot tell you this is a healthy or an unhealthy situation - only that it would make me nervous. Short answer, I'd probably RMA this drive. That or watch it through a few preclears, and if nothing reallocates and there are no read errors in the logs, use it. Well now, was not expecting it to be such a phenomenon that's for sure! Your insights are very appreciated. I believe this drive came with a 2 year limited warranty, purchased from amazon on 11.25.2012. I have used it to house my media for my plex server, inside my desktop, since its purchase so it definitely got some decent usage. It was almost full which was why I started looking for another solution and ended up with unRAID. The model number on this one is (ST3000DM001). I only ran one pre-clear as I wasn't expecting any problems. I am currently running a pre-clear on one of the two older models I planned to place in the build (ST33000651AS) which were purchased from amazon on 7.5.2011. I'm curious as to the responses from those drives as they were used in an external backup raid 1 case. I will definitely post the output of those as well but was only planning on running 1 pre-clear pass on each drive since they had all been used before. I haven't setup anything else up on my unRAID build, other than the preliminary add-on installs (unMENU & Control Panel) until these drives were ready to be installed, the pre-clear time is lengthy on 3TB drives to say the least so running it over and over again on 1 drive just continues to delay me. I'm not opposed to this as I am very much a draw twice cut once person. I have personally never had to RMA a HDD in the 3ish years I've been fiddling with computers personally now so I'm not quite sure what to expect or even if there would be any replacement based upon the time-frame and drives themselves. Could you elaborate a bit on if RMA'ing this drive or any of the three drives I have would be worth it? Seriously, thanks for all the help. These forums continually surprise me the more I read as to how helpful and knowledge people are. Edit: Did not realize the command output included drive model #, none the less I'll keep it included for continuity sake. Edited February 3, 2021 by Sean M. Quote Link to comment
Sean M. Posted April 20, 2014 Share Posted April 20, 2014 (edited) SDE & SDC are a bit older drives but appear to have come through clear. Images Below: http://i.imgur.com/1N0YkUj.pnghttp://i.imgur.com/lpARRAr.png SDD is the one I posted about before that BJP gave some advice on, I have attached the second pre-clear. Just want to get some thoughts before I request an RMA since that drive appears to still be under warranty. Images Below: http://i.imgur.com/zjd2bE1.png Is there anything troublesome I should look out for? Edited February 3, 2021 by Sean M. Quote Link to comment
itimpi Posted April 21, 2014 Share Posted April 21, 2014 The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want. Quote Link to comment
Sean M. Posted April 21, 2014 Share Posted April 21, 2014 The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want. Just submitted an RMA to Seagate to get rid of the questionable one. Thanks for the advice, greatly appreciated! Quote Link to comment
SSD Posted April 21, 2014 Share Posted April 21, 2014 The /dev/sdd device is looking problematical as the number of 'sectors pending reallocated' seems to continually be going up. You do not want to use in unRAID any drive that does not finish the pre-clear with 0 pending sectors. The only anomaly is that no sectors are actually being re-allocated, so it always possible that there is an external factor causing the pending sectors such as bad cabling at the power/SATA level. The large number of sectors pending reallocation is a good enough reason to RMA the drive The other two drives look fine. The key attributes relating to reallocated sectors are all 0 which is what you want. Bad cabling cannot cause reallocated or pending sectors. I have found there are 2 types of pending sectors - those that generate read errors (true bad sectors) and those that don't and invisibly waffle back and forth without any reallocations happening. The first situations unRaid will typically fix. The first read error will cause unRaid to reconstruct the sector, write it back, and the sector will reallocate. In theory this should be how pending sectors should behave. But these waffling pending sectors (which seem unique to Seagate but may happen on other brands too) are a mystery. My experience is until they actually start to reallocate they are just noise. I would run 4-5 preclear cycles and see what happens. Can't argue with RMAing this one with so many pendings though. We just don't have enough insight into what's happening in the drives firmware to understands the whys and wherefores. Quote Link to comment
levster Posted May 27, 2014 Share Posted May 27, 2014 I would like to add a new 4TB Hitachi drive to my system as a parity drive. My current parity is a 3TB WD red. I think that it would be quicker if I precleared the drive first. My system has been running stable for a few years and, when I upgraded to version 5.0 I kept the default partition format as MBR: unalighed. How should I proceed with the new drive? Should I preclear it as an ADVANCED FORMAT drive (-A) or leave all the same, (-a)? I am sorry if I am asking a question that has been already asked, but I cannot find the answer. Thank you, Lev Quote Link to comment
SSD Posted May 27, 2014 Share Posted May 27, 2014 Do you plan to introduce the new drive to replace a smaller parity, and then add the old parity drive as a data drive to the array? Refer to the "Replace parity" how-to link in my sig for some instructions. Quote Link to comment
levster Posted May 27, 2014 Share Posted May 27, 2014 Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev Quote Link to comment
Joe L. Posted May 27, 2014 Share Posted May 27, 2014 Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev With ANY drive over 2.2TB, both the -A and -a options are silently ignored. Use them as you like, but the disk MUST (and will) be partitioned with a GPT partition, and it will have a "protective" MBR partition installed starting on sector 1 to convince older utilities the disk is entirely allocated. Make sure you use the most current version of the preclear script. Older versions did not properly set the preclear signature on some versions of unRAID. You can type preclear_disk.sh -v to learn the version (current version as of this post is 1.15) Joe L. Quote Link to comment
SSD Posted May 27, 2014 Share Posted May 27, 2014 Yes, I may add the old parity in as a data disk, down the road. However, should I preclear the new drive prior to adding it in as a new parity? Will that make the parity rebuild faster? In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev Did you look at my link/instructions about replacing parity? It answers your question about when to preclear (see step 0) Quote Link to comment
Joe L. Posted May 28, 2014 Share Posted May 28, 2014 However, should I preclear the new drive prior to adding it in as a new parity? YesWill that make the parity rebuild faster?No, but you'll trust the disk more. (about 1 in 5 fail in the first hours of operation... do you feel lucky?) In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev If the drive is less than 2.2TB, and NOT a WD-EARS drive with a jumper added, use -A only if an older <=2TB WD-EARS drive with the jumper added should you use "-a" Joe L. Quote Link to comment
levster Posted May 28, 2014 Share Posted May 28, 2014 However, should I preclear the new drive prior to adding it in as a new parity? YesWill that make the parity rebuild faster?No, but you'll trust the disk more. (about 1 in 5 fail in the first hours of operation... do you feel lucky?) In the future, when preclearing new drives, should I stick with the "old" MBR format, or go the advanced route? Lev If the drive is less than 2.2TB, and NOT a WD-EARS drive with a jumper added, use -A only if an older <=2TB WD-EARS drive with the jumper added should you use "-a" Joe L. Excellent! This is exactly what I was looking for. Thank you very much. Lev Quote Link to comment
wgstarks Posted May 30, 2014 Share Posted May 30, 2014 just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. preclear_start_WD-WCC4E1579125_2014-05-29.txt preclear_finish_WD-WCC4E1579125_2014-05-29.txt preclear_rpt_WD-WCC4E1579125_2014-05-29.txt Quote Link to comment
Joe L. Posted May 31, 2014 Share Posted May 31, 2014 just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. looks great. Quote Link to comment
wgstarks Posted May 31, 2014 Share Posted May 31, 2014 just finished preclearing a new 4tb WD Red. Don't expect any problems, but I'd like to get opinions from someone better qualified than I am. Reports attached. looks great. Thanks Quote Link to comment
jbuszkie Posted June 6, 2014 Author Share Posted June 6, 2014 Dumb question.... Do I need to pre-clear a new drive if it's going into the parity slot? I think the answer is no... but... I will pre-clear anyway to test the drive... but is it necessary? isn't the drive completely written anyway when it generates the new parity? Jim Quote Link to comment
Joe L. Posted June 6, 2014 Share Posted June 6, 2014 Dumb question.... Do I need to pre-clear a new drive if it's going into the parity slot? I think the answer is no... but... I will pre-clear anyway to test the drive... but is it necessary? isn't the drive completely written anyway when it generates the new parity? Jim You do not need to, BUT, you should, as you said, to test the drive and to get past the early part of the "bathtub curve" About 1 in 5 drives seems to fail initially or in the first hours of operation. Do you feel lucky? Yes, the drive is completely written, but it is not read until you perform a subsequent parity check. You will not know of un-readable sectors until after you've performed those two steps. After the subsequent parity check you can get a SMART report. If you see any sectors pending re-allocation it is necessary to write again to the sectors marked as un-readable. This will either make them readable, or cause the SMART firmware to re-allocate them. Then you'll again need to perform another parity check to see if the re-allocations were all successful and readable after being re-allocated. Sounds like a lot of reading and writing... It is... and that is almost exactly what the preclear script does for you. Joe L. Quote Link to comment
jbuszkie Posted June 6, 2014 Author Share Posted June 6, 2014 That's what I figured. It was more of a "it won't save me any time" type of question. Like if I had a known good drive already that I had already used.. it doesn't save me anytime to pre-clear it! 9 hours to do the pre read on my 4TB disk! (122MB/s calculated) This is gonna take a while! :-) Jim Quote Link to comment
wgstarks Posted June 6, 2014 Share Posted June 6, 2014 The last 4TB I did took about 100 hours. I'm sure this varies some from one system to another. It could save you a bunch of time if bad sectors are found though. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.