Unassigned Devices Preclear - a utility to preclear disks before adding them to the array


dlandon

Recommended Posts

3 minutes ago, AinzOolGown said:

Hi there ! :)

 

I just shuked a new WD 12Tb drive and wanted to preclear it with this new plugin

First, there was no way to have the preclear option show, then i read that the disk must not contain any partition, so i deleted it by the cli.
Now the little preclear icon show on unassigned partition but the preclear fail everytime :/
And if i go on the plugin UI, it doesn't list the WD 12Tb at all :/

 

P.S.: i Tried to use the

wipefs -a /dev/sdl

cli (for my case) but fail again :/

 

Do you have any idea please ?

Thank you ! :)

You don't need to use the cli to clear the disk.  Enable Destructive Mode in UD settings and use UD to clear the disk.  Click on the red X next to the serial number.

 

Show a screen shot of the disk in UD.

  • Thanks 1
Link to comment
  • 2 weeks later...

Hello, I trying to preclear 3 8tb disks, 2 Seagates and 1 Western Digital (a bit slower than the Seagates). They were All in an Unraid pool previously with no issues with parity checks. So far, the 2 Seagates keep failing the "verifying if disk is zeroed" within 10 minutes of starting. I have been stopping the WD preclear to try to fix the issue.

I have been using the enhanced script and erase and clear disk options. I have uninstalled UD preclear. Uninstalled tmux in nerd pack, then reinstalled UD preclear twice. this is the third time I tried. how likely is it for both drives to fail in the same way.

I have resumed the WD's preclear to see if it passes the "verifying if disk is zeroed". we'll see in the morning.

 

SERVER-unassigned.devices.preclear-20220410-0207.zip SERVER-unassigned.devices.preclear-20220410-0027.zip SERVER-unassigned.devices.preclear-20220408-2324.zip

Link to comment
2 hours ago, marvin888 said:

I have uninstalled UD preclear. Uninstalled tmux in nerd pack, then reinstalled UD preclear twice.

This is a waste of your time and very rarely fixes anything.

 

The disk is failing because of this:

Apr 8 23:45:16: Error: shell_exec(/usr/sbin/smartctl --info --attributes -d auto '/dev/sdh' 2>/dev/null) took longer than 10s!

Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd command failed, exit code [141].
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 33827061760 bytes (34 GB, 32 GiB) copied, 141.91 s, 238 MB/s
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 17442+0 records in
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 17441+0 records out

 

2 hours ago, marvin888 said:

So far, the 2 Seagates keep failing the "verifying if disk is zeroed" within 10 minutes of starting.

So, you are running the "Clear Disk" option and when verifying the disks they are failing?

 

I'm not a disk expert, but my first impression is a disk cabling or controller issue.  Post the full SMART report for one of the disks that is failing.

 

Also, if you are running more than one preclear at a time, only run one and see if there is a difference.

Link to comment
4 hours ago, dlandon said:

This is a waste of your time and very rarely fixes anything.

 

The disk is failing because of this:

Apr 8 23:45:16: Error: shell_exec(/usr/sbin/smartctl --info --attributes -d auto '/dev/sdh' 2>/dev/null) took longer than 10s!

Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd command failed, exit code [141].
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 33827061760 bytes (34 GB, 32 GiB) copied, 141.91 s, 238 MB/s
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 17442+0 records in
Apr 09 23:46:19 preclear_disk_ZA1JXPN2_3743: Post-Read: dd output: 17441+0 records out

 

So, you are running the "Clear Disk" option and when verifying the disks they are failing?

 

I'm not a disk expert, but my first impression is a disk cabling or controller issue.  Post the full SMART report for one of the disks that is failing.

 

Also, if you are running more than one preclear at a time, only run one and see if there is a difference.

I was running "enhanced script" and "erase and clear disk" options.

 

All the hard drives are connected to 2 LSI SAS2008 flashed to IT mode (JBOD). I have 2 because I have a 4U 16 bay enclosure and at the time 3TB were the most economical to buy.

controller 1 has sdd - WD 8TB and sde - Seagate 8TB

controller 2 has sdh - Seagate 8TB 

 

I have other drives connected to both controllers

FYI: extended smart self test only got to 20% in 2 1/2 hours so I aborted it.

server-smart-20220410-0902.zip server-smart-20220410-0915.zip

Link to comment
1 hour ago, marvin888 said:

I was running "enhanced script" and "erase and clear disk" options.

While on the UD Preclear page, click on the Help icon or press F1.  It will explain the different options of the script.  Erase does not write zeroes to the disk.  It writes rendom patterns so a disk can be disposed of.  Only run 'Clear DIsk' once and you will be good to go.

Link to comment
5 minutes ago, marvin888 said:

if I remember correctly, the first time I tried to preclear with "start multiple sessions", it would not let me select just clear disk. It does now.

so, you think erase and preclear is causing the issue?

thank you for your help🙂👍

Probably.  Just clear the disk don't do an erase.  Erase is not a preclear.

Link to comment
22 hours ago, dlandon said:

Probably.  Just clear the disk don't do an erase.  Erase is not a preclear.

 

*** I Was wrong. It just failed the post read check ***

 

So yesterday, I ran a test on a 500gb drive with just clear and the post read option (to save time over a 8TB drive). The drive completed all 4 steps successfully. I started a preclear on one of the 8TB drives that was failing yesterday evening. So far everything is going good. It's about 20 min. in at 2%. in the past it wouldn't make it 5 min. and 0%. I'm hopeful your suggestion fixed it.

 

Thank you. 🍻

Edited by marvin888
Link to comment
On 2/28/2022 at 11:17 PM, zoggy said:

I have a stack of 24 ssd that were part of a linux array that I'm pre-clearing.

I figure this would be a good time to switch from the preclear plugin to UD w/UD prelear to test out.

 

Uninstalled precleared, installed UD and put in SSD and it show "There are no disks that can be precleared."

So normal preclear plugin normally would work fine as it doesn't care what was on the disk before.

 

UD Preclear wont touch it because it sees it was part of a raid and has existing filesystem as you noted above.

So installed UD+, and nuke partition on the disk and now I see I can preclear it.

 

So had to install UD, UD Preclear, UD+ to be able to preclear a drive.

 

Can we make it so the user doesn't need to install UD+ to nuke the drive partition in order to preclear a drive?

 

 

Holy crap, thanks for that. I would not have figured out the magic spell otherwise.

Link to comment

Sooo... one of my memory modules is bad. Ran memtest on all 4 and they failed instantly. took them all out and tested them individually and found one bad. Over 250 errors within the first 30 seconds. Im trying to RMA one set now. I'm running memtest on the other 2 modules now. 3 hours so far so good.

Link to comment

Hi! I did a Clear Disk + Post-read verification of two 2TB drives.

I was using the old plugin, and for some reason after the zeroing (success apparently), both drives failed and then disappeared from the system. I had to reboot in order for them to show up again.

I don't know what happened, but since finding out about this new version due to "issues" in the old plugin, I assume it was something like that.

 

Now, I'm assuming my disks are zeroed, so I'm running a "Verify Disk" on them.

Is that the same as the "post read verification"?

Thanks!

Edited by Robot
Link to comment
21 hours ago, trurl said:

Have you done memtest? RAM can cause the written or read data to be other than expected.

On 4/13/2022 at 11:42 AM, marvin888 said:

Sooo... one of my memory modules is bad. Ran memtest on all 4 and they failed instantly. took them all out and tested them individually and found one bad. Over 250 errors within the first 30 seconds. Im trying to RMA one set now. I'm running memtest on the other 2 modules now. 3 hours so far so good.

Yes, I have. I just finished a successful preclear. zeroing and post read. 29h 33m @ 150MB/s on a 8TB drive

 

Thanks to everyone in this thread (and forum) for your help.

 

Edited by marvin888
Link to comment

Hi,

I had a preclear fail on the last verifying Unraid's preclear signature step.  Screen shots and diagnostic log is attached. 

 

All software was up to date when I started the preclear, but there was an update to unassigned devices during the preclear run.  From searching, I found a few people with the same issue a few years ago w/recommended fix to update to the most recent versions.

 

any suggestions would be appreciated. This was on an 8TB drive that I'm adding as additional.  I also have a 4TB that I need to preclear before using to replace/expand an older smaller drive.   thanks for the input.

 

 

 

media-diagnostics-20220413-0913.zip

unraid_fail1.png

unraid_fail2.png

Edited by ginhead
formatted pictures
Link to comment
  • 2 weeks later...

Hello fellows!

 

I kinda regret setting up 3 cycles of the Preclear, given the time that it took to clear a 14TB disk.
Can I just cancel the process when the 2nd cycle starts? What could happen if I cancel the process, let's say, during Step 1?

Thanks!

  

 

Screen Shot 2022-04-28 at 10.54.00 AM.png

Edited by eltonk
Link to comment

Hi All,

 

I'm planning to upgrade\increase size of some of the disks in my UNRAID array (regular SATA Drives). I know the process but am reviewing the pre-clear option. I'm running the current stable version of UNRAID V 6.9.2 & the pre-clear I have installed is no lonoger supported on this version.  I did some more digging on the forums\youtube and apparently there is no need to pre-clear as when the new drive is added to the array UNRAID automatically does this (pre-clear) in the background and re-populates the new disk and allows you to still access the data using parity and the data on the other disks.  

 

came across this thread and I see that some of you are recommending using  to install UD, UD Preclear, UD+ to be able to preclear a drive.... at 1st I thought this was as old thread, but I see posts are recent.  Just curious as to why one would need to pre-clear if V 6.9.2 does it automatically. 

 

Again I'm still quite a newb to UNRAID and all my questions\facts is based on what I troll, so please correct me if I am wrong.

 

My question is, is this all (manual Pre-clear) necessary ? and if so what are the Pros ? Cons ?

 

I'm just trying to find the best method which works for me.

Link to comment
3 minutes ago, E-ManN said:

apparently there is no need to pre-clear as when the new drive is added to the array UNRAID automatically does this (pre-clear) in the background and re-populates the new disk and allows you to still access the data using parity and the data on the other disks.  

Unraid will clear (not preclear) a disk when it requires a clear disk if it has not already been precleared.

 

A clear disk is required when adding a disk to a new data slot in an array that already has valid parity. This is so parity will remain valid since a clear disk (all zeros) has no effect on parity. This is the only scenario where Unraid requires a clear disk.

 

If you are replacing a disk to rebuild, it doesn't matter at all what is on the replacement disk since it will be completely overwritten by the rebuild.

 

Preclear was invented because old versions of Unraid would take the array offline while it cleared a disk. These days preclear is really just for testing disks. I never preclear, but I keep my array in good condition and have good (enough) backups.

Link to comment
25 minutes ago, trurl said:

Unraid will clear (not preclear) a disk when it requires a clear disk if it has not already been precleared.

 

A clear disk is required when adding a disk to a new data slot in an array that already has valid parity. This is so parity will remain valid since a clear disk (all zeros) has no effect on parity. This is the only scenario where Unraid requires a clear disk.

 

If you are replacing a disk to rebuild, it doesn't matter at all what is on the replacement disk since it will be completely overwritten by the rebuild.

 

Preclear was invented because old versions of Unraid would take the array offline while it cleared a disk. These days preclear is really just for testing disks. I never preclear, but I keep my array in good condition and have good (enough) backups.

 

Thanks for responding trurl  ..  I get it now.  I was a ltitle confused but your explanation is clear. 

 

I too keep my array in good shape and have backups as well.

 

AT work Im IT SYS admin\analyst and we only do stress tests on new standalone disks which have server OS installed even though these disks are mirrored at RAID 1 this the only place we do drive stress test\burnin.  But the data drives in the SAN we never stress test, we just pop them in the array.

 

I will go with not pre-clearing as if the drive fails I will simply replace it.  Most new drives are pretty stable and very few arrive DOA unless they have been dropped in transit\delivery.  My stress test will be the re-population of the data on the new drive.  🙂

 

 

 

 

thanks again,

 

E

 

Thanks so much.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.