Jump to content
binhex

[Support] binhex - Preclear

88 posts in this topic Last Reply

Recommended Posts

Overview: Support for Docker image arch-preclear in the binhex repo.

Application: Preclear - https://forums.unraid.net/topic/2732-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/

Docker Hub: https://hub.docker.com/r/binhex/arch-preclear/

GitHub: https://github.com/binhex/arch-preclear

Documentation: https://github.com/binhex/documentation

 

If you appreciate my work, then please consider buying me a beer 😁
 

btn_donate_SM.gif&key=57f6b617e37a194c35

 

For other Docker support threads and requests, news and Docker template support for the binhex repository please use the "General" thread here

Edited by binhex
  • Like 1
  • Thanks 1

Share this post


Link to post

PLEASE read the FAQ for this image before posting:-

https://github.com/binhex/documentation/blob/master/docker/faq/preclear.md

 

IMPORTANT:- This container MUST run as user and group 'root' (defined via PUID 0 and PGID 0), it also MUST run as 'Privileged'. Do NOT modify any of the paths on the container or host side for the 'ro' volume mounts, these are required to correctly read the status of drives in the array.

Edited by binhex

Share this post


Link to post
8 minutes ago, hypyke said:

What is the advantage of this docker over the existing plugin?

Wow. Just curious, what was your motivation for posting this question?

 

1. Trying to be funny?

2. Too lazy to read the FAQ?

3. Honestly didn't understand what you read in the FAQ?

 

If the answer is 3, I am truly sorry for my attitude here, and apologize profusely. What part of the FAQ explanation needs more work?

Share this post


Link to post

I missed the faq post because I misread it as specifically about the technical aspects of running the docker based on the comment below the link.  That was indeed my mistake.  I do appreciate all the kind comments though.

Share this post


Link to post

Simply put and taken from the FAQ and a HUGE advantage IMHO. 😀

 

Q2. Why has this docker image been created, as we already have a unraid preclear plugin?.

 

A2. Due to the fact that plugins rely on the underlying OS (Slackware) in order to run, any changes made by Limetech can potentially lead to a breakage with the preclear plgin, this has historically happened a number of times and is unfortunately a fact of life. So how do we try and mitigate this from happening?, by using Docker, this then gives us a known platform from which to run the preclear script, and should reduce the chances of this happening.

 

So if LimeTech makes system changes the Docker can stay in tact and still perform like intended.

 

 

Share this post


Link to post

First off, thank you!  Read the FAQ, and the execution of the pre-clear, while CL, seems very straight forward.  I've installed and will give this a try with some used disks I picked up recently.

Share this post


Link to post

Please excuse the possibly idiot question but can I close the Docker command prompt window while the preclear is running (like you could with Screen) or must it remain open in order for the script to continue? On an 8Tb drive it will of course need to run for quite a while and this was not mentioned in the FAQ.

Share this post


Link to post
8 minutes ago, DanielCoffey said:

Please excuse the possibly idiot question but can I close the Docker command prompt window while the preclear is running (like you could with Screen) or must it remain open in order for the script to continue? On an 8Tb drive it will of course need to run for quite a while and this was not mentioned in the FAQ.

its a good question and one i was/am going to add to the FAQ :-), in short yes you can close the novnc web ui window and the preclear script will carry on with no issues whatsoever. The only gotchas here is don't perform a docker backup of this container, as might interupt the running container and thus the preclear procees, and obviously dont stop the docker service :-), other than that you can do what you want and just re-connect to the web ui whenever you want to see progress.

 

edit - just read your wording, so just to be clear here im talking about the novnc web ui, NOT the terminal window running in novnc web ui, that must remain running, but you can minimise it if you wanted.

Edited by binhex

Share this post


Link to post

That is just what I needed, thank you. So just as with other Dockers, we can close the Docker window and the process inside it will carry on running. We can pop back later and reopen the WebUI and take a fresh peek at how it is getting on. Once it is completely finished we are free to stop the Docker if we want.

 

Cheers.

Share this post


Link to post
Just now, DanielCoffey said:

That is just what I needed, thank you. So just as with other Dockers, we can close the Docker window and the process inside it will carry on running. We can pop back later and reopen the WebUI and take a fresh peek at how it is getting on. Once it is completely finished we are free to stop the Docker if we want.

 

Cheers.

EXACTLY! 🙂

Share this post


Link to post

Here's a tip based on personal experience... make sure the array is not due to go to sleep while the Preclear is running. In my case it watched the array for 30 minutes, decided all was quiet, ignored the unassigned devices and shut down. Oh poop! Time to start over I guess.

 

EDIT : well colour me impressed! When I woke the array, the Preclear docker simply resumed from where it had left off and was up and ticking by the time I reopened the WebUI.

Edited by DanielCoffey

Share this post


Link to post

Is there any way to copy/paste text from the terminal window? Wanted to save the help text so I wouldn't have to run it every time but can't get it to save to a clip board outside the window. Alternatively, if I knew the location of the script I could cd to it and run help.

Share this post


Link to post
10 minutes ago, wgstarks said:

Is there any way to copy/paste text from the terminal window? Wanted to save the help text so I wouldn't have to run it every time but can't get it to save to a clip board outside the window. Alternatively, if I knew the location of the script I could cd to it and run help.

Nevermind. Found the button to open the clipboard in NoVNC.

Share this post


Link to post

So has anybody managed to complete a preclear on a disk yet using this image?, i have obviously done this as part of my tests but its always nice to know its working as intended for the end user :-)

Share this post


Link to post
13 hours ago, binhex said:

So has anybody managed to complete a preclear on a disk yet using this image?, i have obviously done this as part of my tests but its always nice to know its working as intended for the end user 🙂

I've got a new 8TB drive on its way and I'll be using the docker version as soon as it gets here. I'll post back and let you know.

 

I understand and get the point of the docker vs potentially breaking the plug-in, but, even though I grew up in the DOS days and used *nix in the 90s, I've gotten lazy and do like the GUIness of the plug-in. I'll live. :)

 

Oh, and welcome back!

Edited by FreeMan

Share this post


Link to post

The drive just arrived today - shockingly fast considering the "SmartPost" delivery. Of course, it looks like the box was kicked 1/2 way across the country...

 

Following the instructions on GitHub, I got this error

Quote

[root@c7771003f836 /]# preclear_binhex.sh -l
====================================1.18
 Disks not assigned to the unRAID array
  (potential candidates for clearing)
========================================
     /dev/sdj = id/ata-ST8000NM0055-1RM112_ZA1FS9VW
[root@c7771003f836 /]# preclear_binhex.sh -f /dev/sdj

Sorry: /dev/sdj does not exist as a block device
Clearing will NOT be performed

 

I'm not sure what "does not exist as a block device" means.

 

I have double (nay, triple) checked that this is the correct drive, and it's the only on on the machine not already assigned to data/parity/cache, so that is the correct one (plus the model/serial match).

 

At the moment, I've got it sitting on top of the server connected via a USB3.0 -> SATA adapter. I've used that for my last 2 or 3 preclears, though those were with the plugin, not the docker, if that matters, without issue.

 

I've gotta run for a couple of hours (I was hoping to give it a head start). If no response when I return, I'll dig into the preclear instructions to see if they have anything to say...

Share this post


Link to post
1 hour ago, FreeMan said:

I'm not sure what "does not exist as a block device" means.

Me either, but my WAG is that the docker can't get the direct access to the USB interface that it needs. Is there any way you can attempt the same operation directly attached to a SATA port to see if the situation changes?

Share this post


Link to post
Me either, but my WAG is that the docker can't get the direct access to the USB interface that it needs. Is there any way you can attempt the same operation directly attached to a SATA port to see if the situation changes?
That was my thought, too. Well, sort of...

I will give it a try plugged in internally. I just didn't have time right then.

Sent from Tapatalk

Share this post


Link to post
1 hour ago, FreeMan said:

That was my thought, too. Well, sort of...

I will give it a try plugged in internally. I just didn't have time right then.

Sent from Tapatalk
 

Yup, definitely something to do with the USB<->SATA adapter. I shut down the server, plugged the drive in to an internal SATA port and she's up and running 1 cycle.

 

Running between 208 & 222MB/s right now. I think that's the fastest I've ever seen on my system!

 

@binhex - you may want to modify the answer to Q6 - Can I preclear via USB. I would list that as a definite maybe... May work for some, not for others. Don't know if you had success doing so, it may just be my configuration.

Also, having uninstalled gfjardim's plugin, on reboot, Fix Common Problems reminded me to uninstall the Statistics plugin that went along with it. Do you have any plans to implement any sort of statistics gathering? (Does anyone know if there were any results of gfjardim's statistics gathering ever published?)

Edited by FreeMan

Share this post


Link to post
11 hours ago, FreeMan said:

I'm not sure what "does not exist as a block device" means.

interesting!, ok i got a question for you regards this, firstly did you start the container BEFORE connecting the drive via the adapter to your USB port?. if you did start the container before connecting the drive, any chance you could try restarting the container AFTER the drive is connected, just incase its a detection issue.

 

8 hours ago, FreeMan said:

Running between 208 & 222MB/s right now. I think that's the fastest I've ever seen on my system!

ive never seen a drive preclear at that speed, that is impressive!.

 

8 hours ago, FreeMan said:

you may want to modify the answer to Q6 - Can I preclear via USB. I would list that as a definite maybe... May work for some, not for others. Don't know if you had success doing so, it may just be my configuration.

i will do, i would love to know if a restart of the container fixes it first though, and yes i did have success preclearing a drive using a sata to usb adapter, it ran without issue for me (cant remember if i connected the drive first or started the container first) , so i would guess either your sata to usb adapter is a bit funky and sometimes works and sometimes doesnt, or its something to do with the order of things, as in usb drive needs to be connected and then container started/restarted for it to correctly identify the device as being a 'block device'.

 

8 hours ago, FreeMan said:

Do you have any plans to implement any sort of statistics gathering?

absolutely not, not now and not in the future :-)

 

i appreciate the feedback btw, really useful.

Edited by binhex

Share this post


Link to post
12 hours ago, binhex said:

interesting!, ok i got a question for you regards this, firstly did you start the container BEFORE connecting the drive via the adapter to your USB port?. if you did start the container before connecting the drive, any chance you could try restarting the container AFTER the drive is connected, just incase its a detection issue.

The container was up and running prior to plugging in the adapter at either end. I'll be happy to try plugging in with the docker running prior.

 

12 hours ago, binhex said:

ive never seen a drive preclear at that speed, that is impressive!.

I've never seen a drive do anything at that speed in my machine! Usually, I've done preclears via the USB<->SATA adapter, so I don't get anywhere near that kind of speed, so this is a bit of a revelation for me! It's a Seagate EXOS 8TB drive - maybe they're actually really fast! It does run at 7200 RP<

 

12 hours ago, binhex said:

i would guess either your sata to usb adapter is a bit funky and sometimes works and sometimes doesnt, or its something to do with the order of things

I've not had issues with the adapter with anything else I've tried to use it for on either unRAID server or connected to my Win10 desktop, so I'd hazard a guess that it's got more to do with the order of connection/activation.

 

12 hours ago, binhex said:

absolutely not, not now and not in the future 🙂

Okey dokey then... steps back from the keyboard...   :)

 

One issue: After starting the PC last night, I checked on it this morning. noVNC had disconnected. Based on the FAQ, I figured "no problem". I clicked the reconnect button and... that was 12 hours ago, I don't remember exactly. I think I ended up closing the browser tab and relaunching the WebGUI again from the docker icon. When it came back up, I've got a terminal session, but no indication of the currently running preclear. 

 

ps -ef | grep pre gives me one result - itself, so it appears to no longer be running. Is there anything else I can do to check to see if it's still running on a background thread somewhere, or do I just presume it managed to die on me and restart it from scratch? preclear_binhex.sh -l lists the drive, but that seems like a reasonable thing for it to have done since it hasn't been added to the array yet. Any suggestions on this?

Share this post


Link to post

I've ended up restarting the preclear. It started from 0 at the pre-read. it is ripping right along on the preclear, though:

 

Quote

================================================================== 1.18
=                unRAID server Pre-Clear disk /dev/sdc
=               cycle 1 of 1, partition start on sector 1
= Disk Pre-Read in progress: 0% complete
= ( 7,603,404,800  bytes of  8,001,563,222,016  read ) 213 MB/s

 

Share this post


Link to post
I've ended up restarting the preclear. It started from 0 at the pre-read. it is ripping right along on the preclear, though:
 
================================================================== 1.18
=                unRAID server Pre-Clear disk /dev/sdc
=               cycle 1 of 1, partition start on sector 1
= Disk Pre-Read in progress: 0% complete
= ( 7,603,404,800  bytes of  8,001,563,222,016  read ) 213 MB/s
 
No idea why the script had exited, I can only assume the container was restarted, what is the uptime for the container right now?


Sent from my CLT-L09 using Tapatalk

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.