Thanks ... anything to save days if not weeks of lost time will be appreciated. I don't have UD installed, should i have? Also, my scenario is slightly different inasmuch as i am preclearing an unassigned disk not yet part of the array. I am assuming that this is ok and that i dont need to assign it until AFTER it has been successfully cleared. J BB, looks like yours is preclearing AFTER you've assigned it, yes? Got to admit i'm only anle to view your jpeg from my ohone at present and my eyes are not good [emoji1]

 

If you assign a disk to your array that hasn't been pre-cleared, unRaid will pre-clear prior to starting the array. Obviously, this would mean that your array would be totally unusable for an extended period until the pre-clear was finished. This was the original purpose of this script. To allow pre-clearing drives before they're added to the array so that there is no downtime. Not sure about the new beta version but Joe L's version would not pre-clear a drive that was already assigned.

I'll not muddy the waters here and let you guys sort things out ... 

I didn’t want to stop the 3 disks that are preclearing on that server so I was trying it on my test server, and I see now that it only hanged because I assigned the disk to array and started it after starting the preclear.

 

As long as I’m not starting a new array for the first time I can start the array with a preclear going, so this is a very specific use case, still attached log in case you want to have a look.

 

BTW, is there a command I can use to see the progress of the 3 preclears that I have going on the hanged server?

 

preclear.disk.beta.txt

Can i come back i here? I was NOT preassigning disk to the array. It initially showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Can i come back i here? I was NOT preassigning disk to the array. It inutiall showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Do what gfjardim asked a few posts above and upload your log.

I didn’t want to stop the 3 disks that are preclearing on that server so I was trying it on my test server, and I see now that it only hanged because I assigned the disk to array and started it after starting the preclear.

 

As long as I’m not starting a new array for the first time I can start the array with a preclear going, so this is a very specific use case, still attached log in case you want to have a look.

 

BTW, is there a command I can use to see the progress of the 3 preclears that I have going on the hanged server?

 

Can i come back i here? I was NOT preassigning disk to the array. It inutiall showed below as an unassigned disk. I the started a preclear using the plugin which started and ran aok UNTIL i tried to 'start' the array, which i assume is allowed? Webgui at that point becomes inaccessible. Similarly if i 'start' the array BEFORE the preclear which again runs fine if i then choose to 'stop' the array the webgui freezes. In both cases the preclear 'seems' to be continuing but hcw can i tell? More importantly the array is useless from that point so i just power cycle costing me days!

 

Evaluating the log you posted, this is a unRAID's problem. Nothing in the plugin code has blocked the webui. Maybe a sync command that take ages because the drive is being written to.

 

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

 

That sounds like a good idea, because normally stopping the array during a preclear sometimes also takes a long time.

What I could do is add a "pause" function to read/write operations of preclear disk script that could be triggered while array is starting/stopping. What do you think?

 

That sounds like a good idea, because normally stopping the array during a preclear sometimes also takes a long time.

 

Is this happen only on writes or on reads too?

That might do it, pausing the preclear until the array either stops or starts.

 

Hopefully the webgui will  remain responsive.

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

 

When it hangs stoping the array, status line on the WebGUI displays "syncing..."

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

 

When it hangs stoping the array, status line on the WebGUI displays "syncing..."

 

Yep, I had to kill the preclear session to be able to start the array:

 

Mar 20 11:13:05 Servidor emhttp: shcmd (1466): sync
Mar 20 11:15:32 Servidor sshd[31777]: Accepted password for root from 192.168.0.60 port 64225 ssh2
Mar 20 11:18:05 Servidor emhttp: shcmd (1467): mkdir /mnt/user0

 

I tried to use nice to schedule dd command to a lower priority, but that wasn't enough...

 

Will try to add a pause function on the script, if possible.

I have found that if you do anything that will cause a Linux 'sync' command to be issued while preclearing (e.g. stopping the array, adding new disks to the array) then the 'sync' effectively last forever (I presume because the preclear is continually filling the Linux disk buffers so they never completely get flushed to disk).  When the sync finishes (e.g. by stopping the preclears) then everything comes back to life).

 

I am not sure there is a workaround for this - it is just an interaction at a low level between the Linux sync command and the disk activity currently taking place.

Is this happen only on writes or on reads too?

 

For me only while writing, no issues when it’s reading.

 

Pretty sure is the sync command hanging the webui.

 

When it hangs stoping the array, status line on the WebGUI displays "syncing..."

 

New update 2016.03.20a which pause the preclear while starting/stopping the array.

 

Please let me know if it helps.

New update 2016.03.20a which pause the preclear while starting/stopping the array.

 

Please let me know if it helps.

 

Yes it did! With a preclear going it just took 1 or 2 seconds more than usual to start the array after a new config.

 

Thanks! Great work! 

 

Can i give this a go? I've got an 8TB still to clear. How do i get the new version?

 

<Edit> ... I've got it and it works a treat!!!

 

Stopped and restarted my array a few times with no ill effects to the preclear (none expected) and more importantly the webgui returned after around 10 seconds instead of hanging which was the point at which i had been power cycling unable to confirm that the preclear was still ongoing.

 

The only confusion was with the download script which still shows the old date/timestamp and version on the plugins screen (2015.11.18 ) which was why i asked where i could get it from earlier.

 

Great work and thanks a lot gfjardim  >

 

That's expected, since the beta plugin is not compatible with Unassigned Devices. Please go to Settings>Preclear Disk Beta

That's expected, since the beta plugin is not compatible with Unassigned Devices. Please go to Settings>Preclear Disk Beta

 

gfjardim ... was that reply for me?

 

If so i do not understand what you are trying to tell me. I do not have a preclear disk beta tab within settings. FYI i downloaded the 'amended' version by 'search'ing for 'preclear' on the CA apps tab. It downloaded the new version (dated today) and the install script confirmed this but the plugin it installed still shows old timestamps and version number against it and hasn't got 'beta' anywhere to be seen. You know much better than me how these things work. I'm just happy to confirm that it does!!

It was a reply to me.  I discovered my error and deleted my post and he must have seen it before I removed it.

 

Excellent script.  Works as expected when I try to run it from the proper spot.

 

That's expected, since the beta plugin is not compatible with Unassigned Devices. Please go to Settings>Preclear Disk Beta

 

gfjardim ... was that reply for me?

 

If so i do not understand what you are trying to tell me. I do not have a preclear disk beta tab within settings. FYI i downloaded the 'amended' version by 'search'ing for 'preclear' on the CA apps tab. It downloaded the new version (dated today) and the install script confirmed this but the plugin it installed still shows old timestamps and version number against it and hasn't got 'beta' anywhere to be seen. You know much better than me how these things work. I'm just happy to confirm that it does!!

That's it!

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

I hope you realize that for CA to display the beta version, you're going to have to fork the plugin repo (sent you a PM a while back) https://github.com/Squidly271/gfjardims-plugin-repository/  (should probably do it anyways as you can then take advantage of what CA offers)

Good to hear ... but can you explain what is 'preclear disk beta' in settings??

 

I run my preclear disk from the plugins tab and although its timestamp doesnt match it runs the new script without issues.

 

There are currently two versions of preclear: one that is the current stable version and has a version number of 2015.11.18 and a testing beta version released today.

 

 

The first one use Joe L. script to preclear; the new one uses a new script I wrote, but still on beta.

I hope you realize that for CA to display the beta version, you're going to have to fork the plugin repo (sent you a PM a while back) https://github.com/Squidly271/gfjardims-plugin-repository/  (should probably do it anyways as you can then take advantage of what CA offers)

 

I honestly missed that. I need to fork everything or just add a new xml file on it?