Hi there,

 

I've been running Unraid flawlessly on an HP EliteDesk Mini for a couple of years. I'm now trying to migrate to new hardware, but for some reason the new server only boots in safe mode. When I try to boot into full-fat Unraid, it hangs at 'Loading /bzroot... OK' and never progresses further.

 

I've tried booting it from a newly-created Unraid USB and the system boots fine. I suspect there's some config that causes the new hardware to hang when it's loaded, but which isn't executed in safe mode. 

 

Is there any way to persist the boot logs to the flash drive so I can review them for any possible issues? Would there be any value in sharing diagnostic files from the system after booting in safe mode? Thanks in advance!

 

Note: I'm booting without any of the pool/array discs connected. Could that cause any issues? I would have assumed Unraid only looks for those discs after booting successfully.

 

Edit 1: Both the new and old systems are booting in UEFI mode.

 

Edit 2: It also intermittently hangs at the same stage, even in safe mode. I thought this might have been related to booting with a display attached to the discrete GPU, but that may have been a red herring.

 

Edit 3: I've exported the diagnostics after booting in safe mode - they're attached to this post now.

steeju-diagnostics-20221219-1750.zip

Edited December 19, 20223 yr by stevehopkinson

8 minutes ago, stevehopkinson said:

sharing diagnostic files from the system after booting in safe mode?

yes

13 minutes ago, stevehopkinson said:

or some reason the new server only boots in safe mode. When I try to boot into full-fat Unraid, it hangs at 'Loading /bzroot... OK'

@trurl may respond with something else after looking at your diagnostics but I got that same behavior once after a hardware swap because new board wanted to boot in UEFI and my unRAID USB was set to boot legacy.  Changing to UEFI (remove the "-" character on the EFI- folder on the unRAID USB flash) allowed it to boot properly after also specifying UEFI:{name of USB flash drive} as the first boot item in the BIOS.

@trurl - I've booted the system in safe mode and downloaded the diagnostics. I've attached them to this post.

@Hoopster - both the old and new hardware are in UEFI mode, so I don't believe that should be an issue. But thanks - I've had that problem before so I know what a pain it can be!

 

I've also noticed that even when I boot in safe mode, the system will hang if I have a display connected to the discrete GPU. I usually keep a dummy plug in the GPU for running it headlessly, but if I tried to boot with the dummy plug connected the system hanged on 'Loading /bzroot… OK' again. On the old hardware I had to do some pretty funky tweaks to get the iGPU passed through to a VM, so it's possible there's still some hardware-specific config left over that's causing an issue with the new hardware…

 

Edit: The GPU display may be a red herring. It seems to intermittently hang/not hang even in safe mode, regardless of whether or not the dummy plug is entered.

steeju-diagnostics-20221219-1750.zip

Edited December 19, 20223 yr by stevehopkinson

On mobile now so can't look at Diagnostics yet. 

 

Are you booting GUI mode or not?

@trurl - not booting in GUI mode (I only ever use the web interface).

I booted into safe mode and set Unraid to bind my GPU's IOMMU group to Unraid at boot. When it prompted me to restart so that changes to system devices could take effect, I did, and then it restarted into full-fat Unraid - successfully! I have no idea whether the VFIO binding actually changed anything, but this was my first successful boot without safe mode.

 

I tried to do a clean boot into full-fat Unraid with the new config, but it hung at the same place again. After another restart, it managed to boot successfully. There seems to be a variable I can't pin down, which is causing it to fail some of the time but boot successfully the rest of the time.

 

I've attached the diagnostics from a successful boot into full-fat Unraid, in case they shed any light on things.

steeju-diagnostics-20221219-1928.zip

I've continued debugging for a couple of days but I still can't get Unraid to boot reliably. Half the time it hangs; half the time it boots successfully. I can't see any obvious variable that leads it to go one way or the other.
 

@trurl - is there anything in the diagnostics that I'm missing? Until I can get it booting successfully 100% of the time on the new hardware I won't be migrating – tearing my hair out trying to understand what's going on!