OS Update issue to 6.10.3 or 6.11

[Casperen]

I have been in the process of updating the OS from 6.9.2 to 6.11 and I have had nothing but issues. After updating to either 6.10.3(stable) or 6.11(stable) the system freezes during loading.

 

Started the process of updating the OS by selecting the update of from the popup I received in the GUI. It updated the OS on the flashdrive and restarted. The restart stopped shortly after selecting boot within a GUI screen.

Did some preliminary research and found some other folks were having similar issues: 

Rebuild my flash drive from my backup, made sure it loaded and all my VMs were functioning, then manually updated the USB stick and copied the config files over. Didn't work... decided to capture an image of where it stopped: Image1

 

Reloaded my working config of 6.9.2 and went thru the fixing common issues plugin and made sure I did have any issues. Mostly ensuring that it displayed errors warnings, but did find one of my VMs still had the VM HD passthrough through syslinux. Fixed that and now have no isses displayed running the normal and extended error checker. Reran the update from the tools menu and manually using both 6.10.3 and 6.11 versions. Still not working.

Thought it might be a hardware issue, so went thru and made sure the BIOS and firmware were updated on the motherboard(they were). So started to unplug pieces of hardware one at a time and seeing if that was the issue, no luck.

 

Thought the issue might be the config, so I loaded 6.10.3 from scratch on a new USB drive and tried to load that(I knew it went error out from registration, but was hoping it would atleast load). Here is where it stopped during the loading process: Image2.

 

Since the OS will not load, I have no way to pull logs to upload, just capturing the images of where it stops.

 

Any recommendation for something else to try?

Server spec:

Motherboard: ASUS Z10PE-WS

CPU: Dual XEON E5-2650

Memory: 64gig DDR 2133

5 array drives of different spec

2 SSD cache drives

1 Backup drive

[SimonF]

2 minutes ago, Casperen said:

I have been in the process of updating the OS from 6.9.2 to 6.11 and I have had nothing but issues. After updating to either 6.10.3(stable) or 6.11(stable) the system freezes during loading.

 

Started the process of updating the OS by selecting the update of from the popup I received in the GUI. It updated the OS on the flashdrive and restarted. The restart stopped shortly after selecting boot within a GUI screen.

Did some preliminary research and found some other folks were having similar issues: 

Rebuild my flash drive from my backup, made sure it loaded and all my VMs were functioning, then manually updated the USB stick and copied the config files over. Didn't work... decided to capture an image of where it stopped: Image1

 

Reloaded my working config of 6.9.2 and went thru the fixing common issues plugin and made sure I did have any issues. Mostly ensuring that it displayed errors warnings, but did find one of my VMs still had the VM HD passthrough through syslinux. Fixed that and now have no isses displayed running the normal and extended error checker. Reran the update from the tools menu and manually using both 6.10.3 and 6.11 versions. Still not working.

Thought it might be a hardware issue, so went thru and made sure the BIOS and firmware were updated on the motherboard(they were). So started to unplug pieces of hardware one at a time and seeing if that was the issue, no luck.

 

Thought the issue might be the config, so I loaded 6.10.3 from scratch on a new USB drive and tried to load that(I knew it went error out from registration, but was hoping it would atleast load). Here is where it stopped during the loading process: Image2.

 

Since the OS will not load, I have no way to pull logs to upload, just capturing the images of where it stops.

 

Any recommendation for something else to try?

Server spec:

Motherboard: ASUS Z10PE-WS

CPU: Dual XEON E5-2650

Memory: 64gig DDR 2133

5 array drives of different spec

2 SSD cache drives

1 Backup drive

Images are not attached.

[Casperen]

Apologies, they should be attached now.

[Casperen]

Spent a little more time tinkering with this again. Thought it might an issue with the USB disk. Used the Uuraid USB creator tool to create a new USB disk of each version. 6.11, 6.10.3 both failed to load again on the new USB drives.

 

Wanted to confirm the usb, so I loaded 6.9.2 and it everything loaded correctly.

 

So the issue is with something that changes in 6.10.3(I think?).

 

I have looked but can't find a hardware compatibility checker? I have a friend that is running the same mother board and CPU on 6.11 so it's not that, and I have unplugged and tried to load with out the rest.

[JonathanM]

8 hours ago, Casperen said:

I have a friend that is running the same mother board and CPU on 6.11

Have you tried your key in their rig as a test? Have you compared BIOS settings?

[Casperen]

Spoke t othe friend and apparently he is using a slightly different motherboard than mine.

[Casperen]

Is there a way to log the loading of the program so I can determine what piece is failing? I have looked thru the flash drive to find it myself(with no luck), but maybe I am just missing something. 

 

It makes no sense for me to have to replace my entire server if only a piece needs to be updated. Currently I have no method to diagnose what is going on.

[ChatNoir]

4 hours ago, Casperen said:

Is there a way to log the loading of the program so I can determine what piece is failing? I have looked thru the flash drive to find it myself(with no luck), but maybe I am just missing something.

The log is stored in RAM and thus lost on a reboot.

You should set up a syslog server and analyse or share the file here.

[Casperen]

This is exactly what I was looking for, apologies for my ignorance on this.

 

Implemented the syslog info to the flash drive and completed the updated to 6.11.1. The log is attached.

 

I appears that it is stopping when mounting one of the SSD cache drives. Does it mean that the cache drives are no longer supported?

thetower-diagnostics-20221012-1619.zip

[JorgeB]

Diags are from v6.9.2? Where's the permanent log?

[Casperen]

Here is the permanent log

syslog

[JorgeB]

Except for some networking config issues not seeing anything relevant, do you now the actual time it last crashed?

[Casperen]

Finally had some time to mess with this again. I didn't have the times from the first time I ran it, so I just redid the installation a few times and recorded the times.

 

First Attempt 10-16--1:
Performed the update at Oct 16 @ 07:51
Performed the server restart at Oct 16 @ 07:55
The reboot stopped at Oct 16 @ 07:59

 

Second Attempt 10-16--2:
Performed the update at Oct 16 @ 08:29
Performed the server restart at Oct 16 @ 08:32
The reboot stopped at Oct 16 @ 08:44

 

The two logs show something is not working with the network config, going to try and fix that. Also noticed an NTP error on the second log, going to look at that also.

 

Is there anything else I am missing?

syslog-10-16--2 syslog 10-16--1

[JorgeB]

Server booted after the upgrade, do you meant you could not access the GUI?

[Casperen]

I am able to see the first gui that asks if I want to boot to command line or GUI, but it never loads enough for me to actually see the GUI screen. It locks up before the second GUI loads.

[Squid]

Can you access it from another system?  Are you seeing just a flashing cursor?

[Casperen]

Nope, it completely freezes. No cursor or ability to access it. I have let it sit that way overnight thinking maybe it was processing something and it doesn't change. Just sits in the say location on the loading screen.

[JorgeB]

9 hours ago, Casperen said:

It locks up before the second GUI loads.

Did you try booting without the GUI mode and then accessing the webGUI?

[Casperen]

Got some time to work on this again today. I tried updating from 6.9.3 to 6.11.1 and loading into all of the options: GUI, No GUI, Safemode, and GUI with no plugins. All of the options fail to load in the same location.

 

I am uploading the log from the no GUI attempt. Seems to look similar to the previous attempts.

Times of the attempt:
10-22 @ 10:17 Update to 6.11.1

10-22 @ 10:20 reboot the server

10-22 @ 10:23 boot freezes.

 

Again, everything seems to load but just stops.

syslog -10-22

[Casperen]

I did see that there was a new version of fix common problems. Updated it to that version:

 

Again, no issues here. 

 

Reran the update assistant and still no issues:

[JorgeB]

Try booting with a new flash drive, with a new stock Unraid install, no key needed, just to see if it boots.

[Casperen]

I have done that to all versions. Nothing boots except for 6.9.2

[JorgeB]

That suggests some compatibility issue with your hardware, most likely the board, other users had success after updating (or downgrading if using latest) the BIOS.