Preclear plugin


Recommended Posts

Thanks ... anything to save days if not weeks of lost time will be appreciated. I don't have UD installed, should i have? Also, my scenario is slightly different inasmuch as i am preclearing an unassigned disk not yet part of the array. I am assuming that this is ok and that i dont need to assign it until AFTER it has been successfully cleared. J BB, looks like yours is preclearing AFTER you've assigned it, yes? Got to admit i'm only anle to view your jpeg from my ohone at present and my eyes are not good

 

If you assign a disk to your array that hasn't been pre-cleared, unRaid will pre-clear prior to starting the array. Obviously, this would mean that your array would be totally unusable for an extended period until the pre-clear was finished. This was the original purpose of this script. To allow pre-clearing drives before they're added to the array so that there is no downtime. Not sure about the new beta version but Joe L's version would not pre-clear a drive that was already assigned.

Link to comment

I didn’t want to stop the 3 disks that are preclearing on that server so I was trying it on my test server, and I see now that it only hanged because I assigned the disk to array and started it after starting the preclear.

 

As long as I’m not starting a new array for the first time I can start the array with a preclear going, so this is a very specific use case, still attached log in case you want to have a look.

 

BTW, is there a command I can use to see the progress of the 3 preclears that I have going on the hanged server?

 

preclear.disk.beta.txt

Link to comment

Can i come back i here? I was NOT preassigning disk to the array. It initially showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Link to comment

Can i come back i here? I was NOT preassigning disk to the array. It inutiall showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Do what gfjardim asked a few posts above and upload your log.

Link to comment

I didn’t want to stop the 3 disks that are preclearing on that server so I was trying it on my test server, and I see now that it only hanged because I assigned the disk to array and started it after starting the preclear.

 

As long as I’m not starting a new array for the first time I can start the array with a preclear going, so this is a very specific use case, still attached log in case you want to have a look.

 

BTW, is there a command I can use to see the progress of the 3 preclears that I have going on the hanged server?

 

Can i come back i here? I was NOT preassigning disk to the array. It inutiall showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Evaluating the log you posted, this is a unRAID's problem. Nothing in the plugin code has blocked the webui. Maybe a sync command that take ages because the drive is being written to.

 

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

Link to comment

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

 

That sounds like a good idea, because normally stopping the array during a preclear sometimes also takes a long time.

Link to comment

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

 

That sounds like a good idea, because normally stopping the array during a preclear sometimes also takes a long time.

 

Is this happen only on writes or on reads too?

Link to comment

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

 

When it hangs stoping the array, status line on the WebGUI displays "syncing..."

 

Yep, I had to kill the preclear session to be able to start the array:

 

Mar 20 11:13:05 Servidor emhttp: shcmd (1466): sync
Mar 20 11:15:32 Servidor sshd[31777]: Accepted password for root from 192.168.0.60 port 64225 ssh2
Mar 20 11:18:05 Servidor emhttp: shcmd (1467): mkdir /mnt/user0

 

I tried to use nice to schedule dd command to a lower priority, but that wasn't enough...

 

Will try to add a pause function on the script, if possible.

Link to comment

I have found that if you do anything that will cause a Linux 'sync' command to be issued while preclearing (e.g. stopping the array, adding new disks to the array) then the 'sync' effectively last forever (I presume because the preclear is continually filling the Linux disk buffers so they never completely get flushed to disk).  When the sync finishes (e.g. by stopping the preclears) then everything comes back to life).

 

I am not sure there is a workaround for this - it is just an interaction at a low level between the Linux sync command and the disk activity currently taking place.

Link to comment

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

 

When it hangs stoping the array, status line on the WebGUI displays "syncing..."

 

New update 2016.03.20a which pause the preclear while starting/stopping the array.

 

Please let me know if it helps.

Link to comment

New update 2016.03.20a which pause the preclear while starting/stopping the array.

 

Please let me know if it helps.

 

Yes it did! With a preclear going it just took 1 or 2 seconds more than usual to start the array after a new config.

 

Thanks! Great work!  :)

 

Link to comment

Can i give this a go? I've got an 8TB still to clear. How do i get the new version?

 

<Edit> ... I've got it and it works a treat!!!

 

Stopped and restarted my array a few times with no ill effects to the preclear (none expected) and more importantly the webgui returned after around 10 seconds instead of hanging which was the point at which i had been power cycling unable to confirm that the preclear was still ongoing.

 

The only confusion was with the download script which still shows the old date/timestamp and version on the plugins screen (2015.11.18 ) which was why i asked where i could get it from earlier.

 

Great work and thanks a lot gfjardim  :)>

 

Link to comment

That's expected, since the beta plugin is not compatible with Unassigned Devices. Please go to Settings>Preclear Disk Beta

 

gfjardim ... was that reply for me?

 

If so i do not understand what you are trying to tell me. I do not have a preclear disk beta tab within settings. FYI i downloaded the 'amended' version by 'search'ing for 'preclear' on the CA apps tab. It downloaded the new version (dated today) and the install script confirmed this but the plugin it installed still shows old timestamps and version number against it and hasn't got 'beta' anywhere to be seen. You know much better than me how these things work. I'm just happy to confirm that it does!!

Link to comment

It was a reply to me.  I discovered my error and deleted my post and he must have seen it before I removed it.

 

Excellent script.  Works as expected when I try to run it from the proper spot. :)

 

That's expected, since the beta plugin is not compatible with Unassigned Devices. Please go to Settings>Preclear Disk Beta

 

gfjardim ... was that reply for me?

 

If so i do not understand what you are trying to tell me. I do not have a preclear disk beta tab within settings. FYI i downloaded the 'amended' version by 'search'ing for 'preclear' on the CA apps tab. It downloaded the new version (dated today) and the install script confirmed this but the plugin it installed still shows old timestamps and version number against it and hasn't got 'beta' anywhere to be seen. You know much better than me how these things work. I'm just happy to confirm that it does!!

Link to comment

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

fEO5Tst.png

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

Link to comment

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

fEO5Tst.png

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

I hope you realize that for CA to display the beta version, you're going to have to fork the plugin repo (sent you a PM a while back) https://github.com/Squidly271/gfjardims-plugin-repository/  (should probably do it anyways as you can then take advantage of what CA offers)
Link to comment

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

fEO5Tst.png

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

I hope you realize that for CA to display the beta version, you're going to have to fork the plugin repo (sent you a PM a while back) https://github.com/Squidly271/gfjardims-plugin-repository/  (should probably do it anyways as you can then take advantage of what CA offers)

 

I honestly missed that. I need to fork everything or just add a new xml file on it?

Link to comment
  • Squid featured, unfeatured and pinned this topic
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.