Preclear plugin


Recommended Posts

On 5/2/2020 at 11:18 AM, Fifthy said:

 

Thanks for clearing things up, greatly appreciated.

 

I did a preclear because of your reasons 1 and 2, and although it takes ages to finnish i feel its worth it in the end.

 

I guess the fuss about preclear is because unraid already does a zeroing when adding drives to the array? 

 

Glad it helped clarify the situation for you. My concern with letting unRAID do the zero is for those few cases where errors are encountered. If a drive has errors while unRAID itself is zeroing the drive, it may or may not get added to the array pool.

 

Early on in my unRAID usage I had numerous errors that were cabling/host controller related. That led to some drives failing with multiple UDMA CRC errors while unRAID finished zeroing and adding them to the pool. Some of the drives went into failure just after finishing the zero and others only after some data had been copied to the array.

 

This meant I had to replace the drive as unRAID was now running in degraded mode. The replacement drive(s) would be rebuilt from parity, taking a considerable amount of time as it would need to reconstruct any data copied to the drive(s) and then zero the rest. And because my problem was really cabling/HBA related, the rebuild from parity can and did occasionally fail as well.

 

Needless to say, it helped me troubleshoot the cabling/HBA issues as I setup another system temporarily just to run preclears. None of the drives actually had any real failures - running the preclear for at least 2 full passes on the other system showed the drives to be OK. That helped me narrow it down to the cabling/HBA.

 

So to clarify, adding an untested or new drive and letting unRAID do the zero is only one pass over the hard drive, whereas a full preclear evaluates each sector 3x per pass - once for the pre-read, once for the zero and once for the post-read. While one pass is better than none, I feel more comfortable doing at least one full preclear. Yes, it takes 3 times as long as letting unRAID do the zero but worth it to me for peace of mind.

 

  • Like 1
Link to comment

Does anyone know if you can run preclear on a different system, then put it in another system and it know that a preclear has already been run?

 

I'm asking as my unRAID server only has USB 2.0, which is PAINFULLY slow. Bought an 8TB MyBook and hooked it up to my server, currently running a preclear on it. It's been running for 5 hours and it's only at 7% of the pre-read EEKS.

 

Would it be possible to throw unRAID on a USB key, boot from it on my main PC, which has USB 3.0, run a preclear on it to make sure it's all good, shuck it, then throw it in my server. Would the unRAID server know it has already been precleared? I don't know if something is written to the drive itself to indicate that it's been run.

 

Thanks for any info.

Link to comment
5 hours ago, unRAIDuser7 said:

Does anyone know if you can run preclear on a different system, then put it in another system and it know that a preclear has already been run?

 

I'm asking as my unRAID server only has USB 2.0, which is PAINFULLY slow. Bought an 8TB MyBook and hooked it up to my server, currently running a preclear on it. It's been running for 5 hours and it's only at 7% of the pre-read EEKS.

 

Would it be possible to throw unRAID on a USB key, boot from it on my main PC, which has USB 3.0, run a preclear on it to make sure it's all good, shuck it, then throw it in my server. Would the unRAID server know it has already been precleared? I don't know if something is written to the drive itself to indicate that it's been run.

 

Thanks for any info.

Yes it does. I am doing the same thing right now and I'm guessing it writes the drives preclear signature after the zero process, I aborted a preclear in the Post Read phase and it still appeared as precleared on the other Unraid

Link to comment
3 hours ago, guru69 said:

Yes it does. I am doing the same thing right now and I'm guessing it writes the drives preclear signature after the zero process, I aborted a preclear in the Post Read phase and it still appeared as precleared on the other Unraid

 

New feature explained on the OP:

 

Q) I lost communication with the webgui, can I manage preclear sessions from the terminal?

A) Yes, you can. If you lost communication with the webgui or want to use the command line interface to manage your preclear sessions, you just need to type preclear on your terminal to start/stop or observe a preclear session.

Link to comment

Got what is hopefully an easy question. Probably answered but not found while searching the forums.

 

In the process of upgrading my parity drive to larger and adding a disk, now I have the former 3T parity drive sitting in Unassigned devices. All good and as expected. Array is started and nominal. Therefore I wanted to do a preclear on the 3T parity, currently sitting on sde1. If I start preclear with default options the UI says "starting" but no stats appear and it does not appear to be working the disk. Checking the disk log I see:

 

May 4 17:10:58 Floater emhttpd: WDC_WD30EFRX-68AX9N0_WD-WMC1T2998971 (sde) 512 5860533168
May 5 05:41:22 Floater emhttpd: WDC_WD30EFRX-68AX9N0_WD-WMC1T2998971 (sde) 512 5860533168
May 5 05:41:51 Floater root: WARNING: adding device /dev/sdi1 gen 225 but found an existing device /dev/sde1 gen 234
May 5 05:41:56 Floater emhttpd: shcmd (10913): /sbin/btrfs device delete /dev/sdi1 gen 224 but found an existing device /dev/sde1 gen 234 /mnt/cache &

 

Not sure what I should do here. Any hints?

 

Thanks!

Link to comment

Hello.

 

I updated this morning to 2020.05.05a and now I am seeing this in the logs:

 

May 6 10:32:14 legion preclear_disk[29247]: error encountered, exiting...
May 6 10:34:49 legion preclear_disk[32017]: error encountered, exiting...
May 6 10:35:45 legion preclear_disk[660]: error encountered, exiting...

 

Since I had performed an update, I also tried completely removing the plugin and reinstalling it, but I am getting the same result.

  • Thanks 1
Link to comment
1 hour ago, trurl said:

Why? Seems like it has already had a good workout and that is the only purpose of preclear.

I should have noted, I tried multiple settings. My first path was to just have it zero the drive without the pre/post tests. Agreed it is already known to be good, reliable and nominal SMART stats.

 

Regardless, the preclear should work, so my question about the warning and behavior still stands. Your support is much appreciated!

 

Added:

 

Actually, I see a different but related issue in the array. I decided to start writing some media files and I notice that Disk 2&3 are greenballed yet the parity disk is not showing as spun up. That should not be possible. Something seems to be fishy in the enumeration at this time.

Edited by pjneder
Link to comment
6 hours ago, pjneder said:

I should have noted, I tried multiple settings. My first path was to just have it zero the drive without the pre/post tests. Agreed it is already known to be good, reliable and nominal SMART stats.

 

Regardless, the preclear should work, so my question about the warning and behavior still stands. Your support is much appreciated!

 

Added:

 

Actually, I see a different but related issue in the array. I decided to start writing some media files and I notice that Disk 2&3 are greenballed yet the parity disk is not showing as spun up. That should not be possible. Something seems to be fishy in the enumeration at this time.

I got this fixed. It was due to repeating UUIDs, as I had not wiped the signature from the disks and they were in unassigned but conflicted with pool disks. All sorted now.

 

Thanks!

Link to comment
8 hours ago, draeh said:

Hello.

 

I updated this morning to 2020.05.05a and now I am seeing this in the logs:

 


May 6 10:32:14 legion preclear_disk[29247]: error encountered, exiting...
May 6 10:34:49 legion preclear_disk[32017]: error encountered, exiting...
May 6 10:35:45 legion preclear_disk[660]: error encountered, exiting...

 

Since I had performed an update, I also tried completely removing the plugin and reinstalling it, but I am getting the same result.

I noticed the same thing, and tried the same.  Aside from the above error message, I also noticed this in the installation log:

 

plugin: installing: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.md5 ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/readvz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/dialog-1.3_20200327-x86_64-1.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/preclear_disk_20.sh ... done
tmux version 3.0a is lower than the installed version (3.1.0), aborting...
libevent version 2.1.11 is greater or equal than the installed version (2.1.11), installing...

+==============================================================================
| Skipping package libevent-2.1.11-x86_64-1 (already installed)
+==============================================================================


utempter version 1.1.6 is lower than the installed version (1.1.6.20191231), aborting...

+==============================================================================
| Installing new package /boot/config/plugins/preclear.disk/preclear.disk-2020.05.05a.txz
+==============================================================================

Verifying package preclear.disk-2020.05.05a.txz.
Installing package preclear.disk-2020.05.05a.txz:
PACKAGE DESCRIPTION:
Package preclear.disk-2020.05.05a.txz installed.

-----------------------------------------------------------
preclear.disk has been installed.
Copyright 2015-2020, gfjardim
Version: 2020.05.05a
-----------------------------------------------------------

plugin: installed

Updating Support Links

preclear.disk --> http://lime-technology.com/forum/index.php?topic=39985.0


Finished Installing. If the DONE button did not appear, then you will need to click the red X in the top right corner

#

 

I will assume that in my case the  Nerd Tools package upgraded some pre-requisites to a version higher than the script likes.
 

Link to comment
1 hour ago, YellowSnowArtisan said:

I will assume that in my case the  Nerd Tools package upgraded some pre-requisites to a version higher than the script likes.

I also have nerd tools installed. The only thing I installed was perl to do the system temperature devices scan. When perl installed it also installed the newer tmux or atleast the nerd tools page shows tmux as installed.

 

Either way, this is some new incompatibility with preclear as the previous version was working fine before the update.

 

EDIT:

I spoke too soon. While tmux 3.1.0 was listed on the nerd tools page, my installed tmux is 3.0a

Edited by draeh
Link to comment
15 hours ago, YellowSnowArtisan said:

I noticed the same thing, and tried the same.  Aside from the above error message, I also noticed this in the installation log:

 

plugin: installing: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.md5 ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/readvz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/dialog-1.3_20200327-x86_64-1.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/preclear_disk_20.sh ... done
tmux version 3.0a is lower than the installed version (3.1.0), aborting...
libevent version 2.1.11 is greater or equal than the installed version (2.1.11), installing...

+==============================================================================
| Skipping package libevent-2.1.11-x86_64-1 (already installed)
+==============================================================================


utempter version 1.1.6 is lower than the installed version (1.1.6.20191231), aborting...

+==============================================================================
| Installing new package /boot/config/plugins/preclear.disk/preclear.disk-2020.05.05a.txz
+==============================================================================

Verifying package preclear.disk-2020.05.05a.txz.
Installing package preclear.disk-2020.05.05a.txz:
PACKAGE DESCRIPTION:
Package preclear.disk-2020.05.05a.txz installed.

-----------------------------------------------------------
preclear.disk has been installed.
Copyright 2015-2020, gfjardim
Version: 2020.05.05a
-----------------------------------------------------------

plugin: installed

Updating Support Links

preclear.disk --> http://lime-technology.com/forum/index.php?topic=39985.0


Finished Installing. If the DONE button did not appear, then you will need to click the red X in the top right corner

#

 

I will assume that in my case the  Nerd Tools package upgraded some pre-requisites to a version higher than the script likes.
 

I am getting the same log spamming, on multiple operations.

Link to comment
On 5/6/2020 at 10:37 AM, draeh said:

Hello.

 

I updated this morning to 2020.05.05a and now I am seeing this in the logs:

 


May 6 10:32:14 legion preclear_disk[29247]: error encountered, exiting...
May 6 10:34:49 legion preclear_disk[32017]: error encountered, exiting...
May 6 10:35:45 legion preclear_disk[660]: error encountered, exiting...

 

Since I had performed an update, I also tried completely removing the plugin and reinstalling it, but I am getting the same result.

I am seeing the same error in log since I updated the plug-in yesterday

Link to comment
I noticed the same thing, and tried the same.  Aside from the above error message, I also noticed this in the installation log:
 
plugin: installing: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.05.05a.md5 ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/readvz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/dialog-1.3_20200327-x86_64-1.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/source/packages/preclear_disk_20.sh ... done
tmux version 3.0a is lower than the installed version (3.1.0), aborting...
libevent version 2.1.11 is greater or equal than the installed version (2.1.11), installing...

+==============================================================================
| Skipping package libevent-2.1.11-x86_64-1 (already installed)
+==============================================================================


utempter version 1.1.6 is lower than the installed version (1.1.6.20191231), aborting...

+==============================================================================
| Installing new package /boot/config/plugins/preclear.disk/preclear.disk-2020.05.05a.txz
+==============================================================================

Verifying package preclear.disk-2020.05.05a.txz.
Installing package preclear.disk-2020.05.05a.txz:
PACKAGE DESCRIPTION:
Package preclear.disk-2020.05.05a.txz installed.

-----------------------------------------------------------
preclear.disk has been installed.
Copyright 2015-2020, gfjardim
Version: 2020.05.05a
-----------------------------------------------------------

plugin: installed

Updating Support Links

preclear.disk --> http://lime-technology.com/forum/index.php?topic=39985.0


Finished Installing. If the DONE button did not appear, then you will need to click the red X in the top right corner
#
 
I will assume that in my case the  Nerd Tools package upgraded some pre-requisites to a version higher than the script likes.
 
The install script will check if there isn't a newer version of a package installed, avoiding the installation of an older version.

Enviado de meu SM-N970F usando o Tapatalk

Link to comment

I'm about to add 4 2tb drives to the array later today.  With the precelar app can i do multiple drives at once? Do you just select multiple drives from the settings menu (i haven't installed them yet, so i dont know what the plugin will show)?  

I'm on a 7700k with 16gb ram.  

Edited by danimal86
Link to comment
16 minutes ago, danimal86 said:

I'm about to add 4 2tb drives to the array later today.  With the precelar app can i do multiple drives at once? Do you just select multiple drives from the settings menu (i haven't installed them yet, so i dont know what the plugin will show)?  

I'm on a 7700k with 16gb ram.  

Yes, you can preclear all 4 simultaneously. The average speed I've seen on my systems is a little over 1.1 hours per TB. So each pass over your 2TB drives will take about 2.2 hours. If you do a single pass of the full preclear, that 3 x 2.2 so 6.6hrs to complete. If you trust the drives and don't want to wait that long, just do the zero stage which writes the preclear signature.

Link to comment

You can clear any number of drives you want. Just go to Tools > Preclear Disk and click the icon Start Multiple Sessions.
433e53b1c77303832b251df8e132060c.jpg

If you hit any hardware limit, you can limit the number of concurrent sessions using the queue:

c5c0d61a1c7604760ae96b7ecb0dc58c.jpg

Enviado de meu SM-N970F usando o Tapatalk

Link to comment
6 hours ago, gfjardim said:

You can clear any number of drives you want. Just go to Tools > Preclear Disk and click the icon Start Multiple Sessions.
433e53b1c77303832b251df8e132060c.jpg

If you hit any hardware limit, you can limit the number of concurrent sessions using the queue:

c5c0d61a1c7604760ae96b7ecb0dc58c.jpg

Enviado de meu SM-N970F usando o Tapatalk
 

 

6 hours ago, AgentXXL said:

Yes, you can preclear all 4 simultaneously. The average speed I've seen on my systems is a little over 1.1 hours per TB. So each pass over your 2TB drives will take about 2.2 hours. If you do a single pass of the full preclear, that 3 x 2.2 so 6.6hrs to complete. If you trust the drives and don't want to wait that long, just do the zero stage which writes the preclear signature.

Thanks for the assistance!  I could only do 3 of the 4 disks since i only have the 12 drive license (so far).  

image.png.b37c8a20026ccedaa863ef7e58ce622c.png

Edited by danimal86
Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.