Trying to upgrade to dual-parity, but WebGUI is failing me

[tillkrueger]

I'm in the process of trying to upgrade to dual-parity and doing a bunch of changes to my data-disk configuration (size upgrades and re-shuffling)...it'll be a lengthy process, but the dual parity is what I am trying to do first.

But just as I am getting ready to do that, the WebGUI is starting to get all wonky on me. 
Right now, even after rebooting 3 times, I can't even generate a Diagnostic report...just like the New Config command from Tools, the WebGUI just sits there, acting like it will start downloading the Diagnostics, but never will. I have never seen this behaviour in all of the 10+ years of using unRAID and came up empty when searching for a terminal-based method of at least generating the Diag report...is there, so that there is even a basis to discuss here?

All I can tell you right now is that I am on the latest stable version (it doesn't show in the WebGUI, but I think it is 6.6.1?) and that until yesterday, everything was zipping along just fine...nothing changed since yesterday, other than that I did a New Config, removed one disk from the configuration and let unRAID re-calculate parity overnight.

Is there a way to get diagnostics via Terminal?

[tillkrueger]

And just now, when I returned to my WebGUI and refreshed the Main tab, the diags downloaded...like 15mins after I issued the command.

here they are.

unraid-diagnostics-20181019-1729.zip

[JorgeB]

Try rebooting in safe mode.

[tillkrueger]

I shall do that, but let me ask you first:

after the diags downloaded, the WebGUI is now behaving much better than it did before this last reboot...I was able to upgrade my plugins (which it didn't let me do before the last reboot).

I don't wanna get this issue too convoluted, but I have to ask:

since I want to upgrade from single to dual parity, and since I have the IT guy at the location in Berlin (I am in Loas Angeles right now) standing by, waiting for me to tell him the next step to physically execute, what is the way to do this next step? is it to do a New Config, then he removes the old parity drive, puts in the two new parity drives, then I re-assign all the drives and start the array or what?

or is it foolish to do *anything* right now while I can't be sure that the New Config command will even execute via the WebGUI? should I try again, at least? is that even the right thing to do?

[tillkrueger]

I just upgraded to 6.6.2 and am rebooting, just so that I'm on the newest OS...haven't been following closely enough, but maybe some bugs were squashed that affected me just now...who knows.

[itimpi]

Is there a reason you want to upgrade both parity disks simultaneously?    That adds some risk as you are unprotected until parity is rebuilt.     You could simply stop the array, assign the new parity 2 and restart the array to get it start building parity on the drive.    When parity 2 is built you could then go through the steps to swap the parity1 drive.

[tillkrueger]

I guess the best answer to that is: it's an OCD thing

I have parity in a slot I want to occupy with a data disk and move parity to the last slot in my server (slot 15) while also upgrading to dual parity (in slot 14), so that all my data drives can go from slot 01 to slot 13...yeah, I know, it's probably my German sense of order that's "making" me do this, but it also has the benefit of allowing me to ask people at the location to swap out a disk for me, when necessary and allowing them to just count the slots and get the right disk.

please don't judge me for being an OCD German 😕

[tillkrueger]

with that said, would it be possible to just stop the array, have him insert the two new 8TB parity drives in slots 14 and 15, do a new config and assign everything as it were, plus the two new parity drives, then calculate parity? I am aware of the risk during calculation (about 20hrs).

[itimpi]

1 minute ago, tillkrueger said:

with that said, would it be possible to just stop the array, have him insert the two new 8TB parity drives in slots 14 and 15, do a new config and assign everything as it were, plus the two new parity drives, then calculate parity? I am aware of the risk during calculation (about 20hrs).

No reason that should not work.

 

from what you are saying I think you intend the old parity disk to become an additional data drive?     If so when you first start the array the disk will be unmountable, and the option to format it will be offered.   When it is formatted it will be mounted (however before telling Unraid to format it check it is the correct drive to avoid an accident).    The format can be done even while parity is being built if that is what you want (although the parity build will slow down while the format is in progress).

[tillkrueger]

you are correct, Sir, but it gets a lot more complicated than that...I have drawn up a strategy that involves moving all of the drives in the array to new slots, and *also* upgrading some of them to larger drives, while *also* removing that pesky 6TB drive which I purchased used and which has been throwing errors every once in a while...so it's be a process that, according to my calculations, might take a week or two to complete, as parity will need to be re-calculated a few times over the course of this plan's execution.

But I'll feel a lot better once parity has been upgraded to dual parity...so I'll do that first and then take it one step at a time after that...I'll probably have more questions that I will post here if/when the need arises, but thanks, so far, for confirming that my theory for upgrading to dual parity isn't totally off!

[tillkrueger]

While things with the WebGUI are working much better now, they are still not working perfectly, as they used to be.

Like trying to invoke the Mover...when I press the MOVE NOW button, it ignores the click...the button will highlight on roll-over, but it won't actually invoke the operation...why could this be happening? I haven't even tried New Config yet, but chances are that it, also, will refuse to actually execute the command, like what was happening earlier today...and without New Config, I can't move on with what I'm planning to do.

What could cause the WebGUI to only partially function...why do some functions work, while others do not?
I mean, I can live without the Mover, for now, but I can't without the New Config.
Is there a terminal command to do a New Config, and if so, what is it?

[JorgeB]

17 minutes ago, tillkrueger said:

.when I press the MOVE NOW button, it ignores the click...

Is anything logged when you do this?

[tillkrueger]

I wish I could tell...when I press the "Log" link in the WebGUI, an empty window pops up...something is totally wonky with the WebGUI...can't even stop the array right now, and my IT guy there is waiting to put in some disks and go home...ugh.

[tillkrueger]

with that said, is there a process and commands via terminal to stop the array (clean), reboot (clean), and do a New Config?

[tillkrueger]

and again, 15mins (why?!) later that empty Log window populated with the following:

 

ErrorWarningSystemArrayLogin

Oct 19 19:23:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:24:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:25:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:26:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:27:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:28:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:29:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:30:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:30:36 unRAID kernel: scsi 5:0:11:0: Direct-Access ATA WDC WD8003FFBX-6 0A83 PQ: 0 ANSI: 6
Oct 19 19:30:36 unRAID kernel: scsi 5:0:11:0: SATA: handle(0x001c), sas_addr(0x443322110e000000), phy(14), device_name(0x0000000000000000)
Oct 19 19:30:36 unRAID kernel: scsi 5:0:11:0: enclosure logical id (0x500605b002c8b369), slot(13) 
Oct 19 19:30:36 unRAID kernel: scsi 5:0:11:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: Attached scsi generic sg12 type 0
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: Power-on or device reset occurred
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] 15628053168 512-byte logical blocks: (8.00 TB/7.28 TiB)
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] 4096-byte physical blocks
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] Write Protect is off
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] Mode Sense: 7f 00 10 08
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] Write cache: enabled, read cache: enabled, supports DPO and FUA
Oct 19 19:30:36 unRAID kernel: sd 5:0:11:0: [sdm] Attached SCSI disk
Oct 19 19:30:38 unRAID kernel: scsi 5:0:12:0: Direct-Access ATA WDC WD8003FFBX-6 0A83 PQ: 0 ANSI: 6
Oct 19 19:30:38 unRAID kernel: scsi 5:0:12:0: SATA: handle(0x001d), sas_addr(0x443322110b000000), phy(11), device_name(0x0000000000000000)
Oct 19 19:30:38 unRAID kernel: scsi 5:0:12:0: enclosure logical id (0x500605b002c8b369), slot(8) 
Oct 19 19:30:38 unRAID kernel: scsi 5:0:12:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: Power-on or device reset occurred
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: Attached scsi generic sg13 type 0
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] 15628053168 512-byte logical blocks: (8.00 TB/7.28 TiB)
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] 4096-byte physical blocks
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] Write Protect is off
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] Mode Sense: 7f 00 10 08
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] Write cache: enabled, read cache: enabled, supports DPO and FUA
Oct 19 19:30:38 unRAID kernel: sd 5:0:12:0: [sdn] Attached SCSI disk
Oct 19 19:31:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:32:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:33:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:34:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:35:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:36:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:37:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:38:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:39:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:40:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:41:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:42:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null
Oct 19 19:43:01 unRAID crond[1535]: exit status 127 from user root /usr/lib/sa/sa1 1 1 &> /dev/null

[JorgeB]

I still think you should boot in safe mode, assuming you're on v6.6 you can select that by clicking on the flash drive on the main page, switch to advanced view, select safe mode and then reboot.

[tillkrueger]

ugh...*nothing* about the WebGUI is working as it should right now...what the heck is going on?!

I don't see the Advanced view, if I'm in the right place.

[tillkrueger]

but I need to ask again:

what is the sequence of commands to do a clean stop array, reboot, and New Config? then, at least, I could get around the wonky WebGUI until I make some progress with the array upgrades/changes...my IT guy there wants to go home for the weekend.

[JorgeB]

3 minutes ago, tillkrueger said:

what is the sequence of commands to do a clean stop array, reboot,

reboot

Then use the GUI for the new config, and before typing reboot set boot to safe mode.

[tillkrueger]

thanks

reboot appears to be initiated...let's see what the system comes back with, and whether I can then initiate a clean reboot in safe mode.

how do I select safe mode? is that the same as selecting the array not to mount after reboot, or is there more involved?

as I showed in the post above, I do not see the Advanced toggle when I look at the Flash disk tab...am I looking in the wrong place?

and is there a sequence of commands or a command to do a New Config from Terminal?

[tillkrueger]

and again, after this reboot, I cannot stop the array via the WebGUI...is there a terminal command or command sequence for that?

what is wrong with the WebGUI???!!!

[tillkrueger]

ok, not sure why, but the GUI finally accepted the STOP ARRAY request...it won't take my New Config request, though...can I do it (and stop the array) via Terminal? (sorry for sounding like a broken record)

[tillkrueger]

when I refreshed the Main page just now, all slots showed as empty...can I assume that unRAID actually did execute the New Config, and proceed with assigning dual parity and all other slots as they were, then start the array again to re-calculate parity, *without* the danger of something terrible happening when I do that? there is no other reason why all slots would show as empty, other than the New Config having worked in the background, right?

[JorgeB]

10 minutes ago, tillkrueger said:

when I refreshed the Main page just now, all slots showed as empty...can I assume that unRAID actually did execute the New Config

Yes, it's the most likely explanation.

 

10 minutes ago, tillkrueger said:

and proceed with assigning dual parity and all other slots as they were, then start the array again to re-calculate parity, *without* the danger of something terrible happening when I do that?

There should be no danger to your data as long as you don't assign a data disk to a parity slot by mistake, but if the GUI isn't working correctly it might become irresponsive during the sync, though you can still follow the sync progress using the command line

 

Edit: Going out for a few hours, back before/around midnight my time (GMT+1)

Edited October 19, 2018 by johnnie.black

[tillkrueger]

Hope you've had a good time out there!

I also thought there would be no danger, but with the WebGUI being so wonky that that pretty much *every* operation doesn't work the first (or second or third or fourth...) time around, I never know what instruction just got executed on the backend...was it the New Config that just refreshed the page, or was it the disk I just re-assigned?

Long story short, when I just spent almost an hour on the phone with my IT guy there who physically swapped disks for me, Slot 11 suddenly stopped working altogether (would see any disk that was inserted into it), and when I assigned disks that were moved from slot 04 and 05 to slots 12 and 13, those slots suddenly showed as blue, when all other assigned slots showed as green, making me worry that had I started the array in that state, it might have thought that those were new disks and start formatting/clearing them.

long story short: the WebGUI has become totally unreliable and mostly unresponsive...I never know whether a command I issued via it was actually executed...so that's no viable way of working with unRAID.

How can such a wonky WebGUI be trouble-shooted?

And just to ask the same question for the 4th or 5th time:

what would be the terminal methods for doing a clean stop array, and a New Config...if I could do those via terminal for now, then I could at least make progress on the new disk-configuration and upgrades while trying to figure out the WebGUI issues with the help of the esteemed unRAID community.

If I don't talk to you today anymore, have a good night jb.
you are one of the true blessings of this community.