Preclear plugin

[cybrnook 92]

unRAID version (6.8.1) not supported.
plugin: run failed: /bin/bash retval: 1

Being marked for deletion, and cannot reinstall on 6.8.1

[pluginCop 10]

2 hours ago, cybrnook said:

unRAID version (6.8.1) not supported.
plugin: run failed: /bin/bash retval: 1

Being marked for deletion, and cannot reinstall on 6.8.1

Confirmed.  Will however give @gfjardim time to fix this before any additional action is taken.

[cybrnook 92]

Just now, pluginCop said:

Confirmed.  Will however give @gfjardim time to fix this before any additional action is taken.

I was just looking at the source, and I think it's due to the "less than":

elif is_unraid_version '>=' '6.8.0-b1' && is_unraid_version '<' '6.8.1'; then

Perhaps there is a dependency on the kernel version in 6.8.1 and what is being brought down for tmux, libevent and utemper. But I think likely the same packages from 6.8.0-b1 would work, will see 🙂

[pluginCop 10]

10 minutes ago, BRiT said:

Well the previous versions of PreClear Plugin still did questionable things on 6.8 such as replacing libevents with years older versions, so it probably shouldn't even be listed on 6.8.

 

Quote

Bugs within applications can (and do) happen.  This is outside of the control of the moderators of CA.  Depending upon the circumstances, the application may be subject to moderation due to the bug.  This moderation may be mild or in the cases where the bug could cause data-loss severe resulting in the possible blacklisting of the application.  In most cases, the author is given time to rectify the bug before moderation happens.  Minor issues with any application will tend to not have any moderation applied.  As a general rule, it is recommended to always keep your applications (especially plugins and unRaid itself) up to date.  In the case of egregious software errors, the moderators of CA will err on the side of the user instead of the side of the author.

 

[cybrnook 92]

33 minutes ago, BRiT said:

Well the previous versions of PreClear Plugin still did questionable things on 6.8 such as replacing libevents with years older versions, so it probably shouldn't even be listed on 6.8.

I think libevent was the only thing, rather than a few things, right? I can confirm that libevent is installed by default, but perhaps the included tmux for example can't find the default? Note sure.. @gfjardim ?

root@noah:/usr/lib64# ls -l libevent*
lrwxrwxrwx 1 root root     21 Jan 11 03:15 libevent-2.1.so.7 -> libevent-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 354048 Aug  2 14:22 libevent-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     21 Jan 11 03:15 libevent.so -> libevent-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     26 Jan 11 03:15 libevent_core-2.1.so.7 -> libevent_core-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 229752 Aug  2 14:22 libevent_core-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     26 Jan 11 03:15 libevent_core.so -> libevent_core-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     27 Jan 11 03:15 libevent_extra-2.1.so.7 -> libevent_extra-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 143264 Aug  2 14:22 libevent_extra-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     27 Jan 11 03:15 libevent_extra.so -> libevent_extra-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     29 Jan 11 03:15 libevent_openssl-2.1.so.7 -> libevent_openssl-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  31336 Aug  2 14:22 libevent_openssl-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     29 Jan 11 03:15 libevent_openssl.so -> libevent_openssl-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     30 Jan 11 03:15 libevent_pthreads-2.1.so.7 -> libevent_pthreads-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  14296 Aug  2 14:22 libevent_pthreads-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     30 Jan 11 03:15 libevent_pthreads.so -> libevent_pthreads-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  14192 Dec 18 13:34 libevents-samba4.so*

 

Edited January 11, 2020 by cybrnook

[aim60 5]

3 hours ago, cybrnook said:

unRAID version (6.8.1) not supported.
plugin: run failed: /bin/bash retval: 1

Being marked for deletion, and cannot reinstall on 6.8.1

Its for this kind of issue that binhex isolated preclear in a docker.

 

 

[Frank1940 563]

2 minutes ago, aim60 said:

Its for this kind of issue that binhex isolated preclear in a docker.

 

 

I agree with you up to a point.  The problem with the Docker is that it requires the use of the command line.  While its usage is much more straight forward than Joe L's original preclear script, it still requires the use of a good cheat sheet.  (While I am comfortable using the command line, it seems that many folks are really afraid of even trying to use it.) 

 

What is really needed it to combine the Interface of this Preclear Plugin into a Docker app.  While it would not be running (almost transparently) in the Unraid GUI, it would probably be using a NOvnc browser tab with a GUI wrapper interface to launch the underlying script.  That way the preclear script would always have the proper Linux dependencies and libraries even when Unraid's Linux is updated. 

[DarphBobo 12]

My take on the situation

 

The original script worked flawlessly.  By extension, the binhex container should also work flawlessly.

 

Unfortunately, the track record of the plugin has been mixed, and there's too many posts that suggest to uninstall it to fix any problems with the underlying OS.

 

"Pre" clearing a drive hasn't actually been necessary for quite a while now, and while people do still use it to see if a new drive is ok or not, other alternatives exist (ie: extended smart test) that will definitely not affect anything else on the server.

 

Personally, I haven't actually precleared a drive in years.  Toss a new one in, have unRaid clear it (or rebuild onto it if expanding an existing drive).  The SMART reports will pretty much tell you everything you need to know.  

 

Besides, when was the last time anyone actually ran Preclear on a drive that is destined for their desktop?  If you don't particularly care about it there, why would you care on your server?

 

To each their own

Edited January 11, 2020 by DarphBobo

[cybrnook 92]

Well, personally I preclear every drive I add to my unraid server prior to making it a part of my array 🙂 While "most" drives come up uneventful, I did just two weeks ago shuck and add a handful of 8TB's to my backup unraid server and low and behold preclearing saved me ahead of time on a disk that started to mark and reallocate bad sectors. Easier to identify ahead of time, rather than add a disk, put data on the disk, and then in time realize through smart somethings happening.

 

For me, I love the integration of this plugin into UD (UD+), and being able to preclear at the click of a button.

[zoggy 19]

using latest version on 6.7.2, inserted a new disk, going to preclear it but first I wanted to pull the smart details to record.

/Tools/Preclear/New?name=sdn

clicking on "Download SMART report:" results in nothing/this invalid url:

/Tools/Preclear/%3Cbr%20/%3E%3Cb%3EWarning%3C/b%3E:%20%20copy():%20Filename%20cannot%20be%20empty%20in%20%3Cb%3E/usr/local/emhttp/plugins/dynamix/include/Download.php%3C/b%3E%20on%20line%20%3Cb%3E24%3C/b%3E%3Cbr%20/%3E/husky-smart-20200111-1706.zip

 

[TechMed 11]

Ya know, I wasn’t going to chime in here as I am just beginner. However, like Cybrnook I too

2 hours ago, cybrnook said:

I preclear every drive I add to my unraid server prior to making it a part of my array

 

2 hours ago, cybrnook said:

For me, I love the integration of this plugin into UD (UD+), and being able to preclear at the click of a button.

There have been a number of occasions where I thank those of you (to an empty room or one of my other personalities 😊) who take the time to write these sorts of programs and keep them up to date.

 

Unfortunately for me, just today I discovered there is a Docker for preclear. Now I need to figure out which to use… plugin or Docker. Does anyone know of thread discussing the pros and cons so I can make a decision on that? 🤔

 

Again, just my two cents about preclearing and a thanks to a great supportive Community.

[itimpi 890]

The big advantage of docker based solutions is that they are (in principle at least) isolated from and independent of the Unraid release that is hosting it, and will not be affected by Unraid updates.   As such a docker based solution is always the preferred approach over the same capability via a plugin.

 

[TechMed 11]

I read in Binhex's FAQ that the Docker

2 minutes ago, itimpi said:

isolated from and independent of the Unraid release

which makes perfect sense. My ponder is if it is seasoned enough.

 

I just need to read the posts about each for a bit. Like most of you, I will probably DL the Docker to my test machine and see how it does.

Thing is, while I do not have any issues with using the CLI, there are those that do.

Frank1940 made a great point about those that are not comfortable with it:

2 hours ago, Frank1940 said:

requires the use of a good cheat sheet

This way folks could copy and paste, for the most part, and reduce the likelihood of trashing one of their existing data drives.

 

Dunno... just need to 'give it a think' and play with it (Docker) for a while. The past couple of days have been a great educational conversation though!

[gfjardim 91]

5 hours ago, cybrnook said:

I think libevent was the only thing, rather than a few things, right? I can confirm that libevent is installed by default, but perhaps the included tmux for example can't find the default? Note sure.. @gfjardim ?

root@noah:/usr/lib64# ls -l libevent*
lrwxrwxrwx 1 root root     21 Jan 11 03:15 libevent-2.1.so.7 -> libevent-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 354048 Aug  2 14:22 libevent-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     21 Jan 11 03:15 libevent.so -> libevent-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     26 Jan 11 03:15 libevent_core-2.1.so.7 -> libevent_core-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 229752 Aug  2 14:22 libevent_core-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     26 Jan 11 03:15 libevent_core.so -> libevent_core-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     27 Jan 11 03:15 libevent_extra-2.1.so.7 -> libevent_extra-2.1.so.7.0.0*
-rwxr-xr-x 1 root root 143264 Aug  2 14:22 libevent_extra-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     27 Jan 11 03:15 libevent_extra.so -> libevent_extra-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     29 Jan 11 03:15 libevent_openssl-2.1.so.7 -> libevent_openssl-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  31336 Aug  2 14:22 libevent_openssl-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     29 Jan 11 03:15 libevent_openssl.so -> libevent_openssl-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     30 Jan 11 03:15 libevent_pthreads-2.1.so.7 -> libevent_pthreads-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  14296 Aug  2 14:22 libevent_pthreads-2.1.so.7.0.0*
lrwxrwxrwx 1 root root     30 Jan 11 03:15 libevent_pthreads.so -> libevent_pthreads-2.1.so.7.0.0*
-rwxr-xr-x 1 root root  14192 Dec 18 13:34 libevents-samba4.so*

 

Sometimes Limetech uses stripped versions of packages, for an example, ncurses doesn't have the correct terminal library (screen) for tmux, so or I change the terminal type or I overwrite the stock package. Libevent too.

 

I've become strict about Unraid versioning because of these stock packages, but I'll implement a better way of dealing with those incompatibilities.

 

By the way, 6.8.1 support is implemented.

[cybrnook 92]

@gfjardim thanks a ton for the quick turn around time, and for looking into package handling. I really do like this plugin, and have gotten used to the UD + Preclear integration, don't wanna change... MmmmmmmMmm

[gfjardim 91]

To proper handle unsupported status, I'm thinking in display a message in the webui and disable the plugin functionality until proper support. What do you guys think?

[TechMed 11]

Not sure if this is relevant to the discussion, but yesterday I started preclearing two drives and did something stupid shortly after midnight… I updated the plugin in the middle of the process.; rookie move, I know. Needless to say, the preclear halted. So I figured, ‘what the heck, they are new drives’ and put one into the array (the second is being used for Parity so I wanted that thoroughly checked first).

 

At this point I stopped the array, added one of the drives to array and started the array, knowing unRAID would 'clear' the drive for me. Well, the array would not start. Since I had already screwed up from being tired, I quit there and just left everything running as is until today. However, before I hit the sack I noticed that the second drive picked up the preclear process where it left off; array still showing ‘starting’; it is not though. Is this an expected chain of events?

 

Now, looking at the event times it appears that the array did eventually start, but the time indicates it was after the second drive preclear had finished ‘zeroing’ and was starting the 'Post-Read'. Now that the array was finally started, the first drive I had added to the array began the unRAID ‘clearing’ process.

 

To recap, preclearing two drives, updated preclear plugin during, hosed the preclear process, added one drive to array, array would not start, but second drive restarted the preclear process where it stopped (it is currently in Post-Read), first drive started ‘clearing’ about the time when second drive finished ‘zeroing’ and is continuing the unRAID clearing process at this time.

 

As of right now, everything appears to be as it should; we’ll see later today. Should I just let it all finish and see what the end result is? I am posting all this in case there is useful info for anyone. Not sure if the logs are necessary, but attached them anyway. Thanks!!!

5000cca23b05c734.resume 5000cca23b063a1c.resume preclear.disk-2020.01.11b.md5 preclear.disk-2020.01.11b.txz tmux-3.0a-x86_64-1.txz thedarkvault-diagnostics-20200112-1006.zip

[Squid 2986]

4 hours ago, gfjardim said:

To proper handle unsupported status, I'm thinking in display a message in the webui and disable the plugin functionality until proper support. What do you guys think?

Would probably save some needless posts...

[gfjardim 91]

29 minutes ago, TechMed said:

Not sure if this is relevant to the discussion, but yesterday I started preclearing two drives and did something stupid shortly after midnight… I updated the plugin in the middle of the process.; rookie move, I know. Needless to say, the preclear halted. So I figured, ‘what the heck, they are new drives’ and put one into the array (the second is being used for Parity so I wanted that thoroughly checked first).

 

 

You added a preclearing drive to your array.

 

Jan 11 15:25:12 - the existing plugin is installed.

Jan 11 15:47:33 - start a preclear session to disk 2EG35HVJ

Jan 11 15:47:47 - start a preclear session to disk 2EG3E52J

Jan 11 15:47:55 - stopped the preclear session of disk 2EG35HVJ
Jan 11 15:48:02 - stopped the preclear session of disk 2EG3E52J

Jan 11 15:48:44 - installed plugin version 2020.01.11a

Jan 11 15:49:11  - start a new preclear session to disk 2EG35HVJ

Jan 11 15:49:20 - start a new preclear session to disk 2EG3E52J

 

Jan 12 05:30:42 - you added a disk to the array, apparently 2EG35HVJ. At that moment the disk was between zero writing and MBR writing, and after that it began to post-read at 05:32:39. I see no more activity after that.

 

You can see what's happening in the background by typing preclear at the web terminal.

[TechMed 11]

Thanks @gfjardim I was unaware of the terminal command; I will look into it shortly.

I was more concerned about if I completely hosed up the currently running preclear/clear, despite their continuing to chug along nicely. And if the event might prove useful somehow.

 

Based on my readings here this may need to be posted elsewhere, but it would be nice if the unRAID 'clearing' had feedback on the GUI like yours does. Just a thought.

 

Thanks for your help and continuing to support the plug-in.

So at this point in time, are we to use the plug-in, assuming we want to of course, or is it being shelved for a bit? From what I have been following here it seems to be still up in the air?

[gfjardim 91]

The plugin is fully functional; there were problems during Unraid 6.8.1 update because the plugin wasn't updated for it, but it's working right now.

[TechMed 11]

Cool!

Thanks for the quick response.

Have a good day!

[jedimstr 8]

Both version 2020.01.12 and 2020.01.13 are showing the "unsupported" warning even though it was updated for 6.8.1 support.

 

[gfjardim 91]

6 minutes ago, jedimstr said:

Both version 2020.01.12 and 2020.01.13 are showing the "unsupported" warning even though it was updated for 6.8.1 support.

 

Please remove the plugin, install it again and send me all the content from the installation process.

[jedimstr 8]

3 hours ago, gfjardim said:

Please remove the plugin, install it again and send me all the content from the installation process.

Removed and re-installed from CA.  Here's the installation output:
 

plugin: installing: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.01.13.txz ... done
plugin: downloading: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/archive/preclear.disk-2020.01.13.md5 ... done
tmux version 3.0a is greater or equal than the installed version (3.0a), installing...

+==============================================================================
| Skipping package tmux-3.0a-x86_64-1 (already installed)
+==============================================================================

libevent version 2.1.11 is greater or equal than the installed version (2.1.11), installing...

+==============================================================================
| Skipping package libevent-2.1.11-x86_64-1 (already installed)
+==============================================================================

utempter version 1.1.6 is lower than the installed version (1.1.6.20191231), aborting...

+==============================================================================
| Installing new package /boot/config/plugins/preclear.disk/preclear.disk-2020.01.13.txz
+==============================================================================

Verifying package preclear.disk-2020.01.13.txz.
Installing package preclear.disk-2020.01.13.txz:
PACKAGE DESCRIPTION:
Package preclear.disk-2020.01.13.txz installed.

-----------------------------------------------------------
preclear.disk has been installed.
Copyright 2015-2020, gfjardim
Version: 2020.01.13
-----------------------------------------------------------

plugin: installed

Updating Support Links

preclear.disk --> http://lime-technology.com/forum/index.php?topic=39985.0