Reboot from GUI don't find USB to boot from

[archedraft]

And then there were 3 and the plot thickens! I'll be able to post those 4 results later tonight. Hopefully peter can get his test as well and it reveals something useful.

[peter_sm]

Don't try an older beta yet.  Try the steps I've outlined and share the info.  thanks!!

 

@Jonp

I have now booted in to safe mode, did a reboot ,and same issue as before. Power down, and power on it's boot's is fine

 

Below are a link to a movie and some images and my syslinux.cfg

 

https://www.dropbox.com/s/mfeon0coba2pvg1/rebooting.zip?dl=0

 

 

//Peter

[jonp]

Don't try an older beta yet.  Try the steps I've outlined and share the info.  thanks!!

 

@Jonp

I have now booted in to safe mode, did a reboot ,and same issue as before. Power down, and power on it's boot's is fine

 

Below are a link to a movie and some images and my syslinux.cfg

 

https://www.dropbox.com/s/mfeon0coba2pvg1/rebooting.zip?dl=0

 

 

//Peter

Thanks Peter. Did you get pic of the bios too?  I will review these later today.

[peter_sm]

Yes. its included in the zip file, enjoy the movie  Not the best quality, but is good enough.

zip is about 200 Meg

//Peter

[archedraft]

OK my tests:

 

1)  Please boot up unRAID 6 beta 10a into safe mode (non-Xen) and attempt a reboot, see if it work.

Even in safe mode the computer will not reboot properly

 

2)  Post a copy of your current syslinux.cfg so I can see what additional kernel parameters are being used.

label KVM/unRAID OS

  menu default

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot

 

3)  Please record a short video of the looping reboots so we can witness the actual experience.

Peter's video is almost exactly the same as what mine looks like. The only difference is I only see the ASROCK screen and it reboots

 

4)  Please take pictures of your motherboard BIOS settings (as much as you can).

Will post these tonight

[saarg]

Do you have an option in the uefi setup that is called Full HD Uefi? It is on the main page on my motherboard (H97 Pro4).

When this was set as auto it made problems for me like the boot loops you're having. I have it disabled.

 

[archedraft]

OK my tests UPDATED:

 

1)  Please boot up unRAID 6 beta 10a into safe mode (non-Xen) and attempt a reboot, see if it work.

Even in safe mode the computer will not reboot properly

 

2)  Post a copy of your current syslinux.cfg so I can see what additional kernel parameters are being used.

label KVM/unRAID OS

  menu default

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot

 

3)  Please record a short video of the looping reboots so we can witness the actual experience.

https://www.dropbox.com/sh/12ude0593dpoowk/AACBLq_PzN8oG9Hvc8onvi-Qa?dl=0

 

4)  Please take pictures of your motherboard BIOS settings (as much as you can).

Please see link above (sorry about the large file size... apparently my new phone takes 1080p videos by default...

 

Do you have an option in the uefi setup that is called Full HD Uefi? It is on the main page on my motherboard (H97 Pro4).

When this was set as auto it made problems for me like the boot loops you're having. I have it disabled.

 

 

I could not find that option in my bios. I also decided to update my bios to the latest version just for kicks but it did not solve the endless reboot issue.

[jonp]

Just an FYI, have this on my task list for tonight, but the list is backing up quick.  Might not be till tomorrow for me to look.  Just keeping you guys apprised.

 

[archedraft]

Thanks, I appreciate the update.

[bungee91]

Here's what I'd like to see from each of you having this issue:

 

1)  Please boot up unRAID 6 beta 10a into safe mode (non-Xen) and attempt a reboot, see if it work.

2)  Post a copy of your current syslinux.cfg so I can see what additional kernel parameters are being used.

3)  Please record a short video of the looping reboots so we can witness the actual experience.

4)  Please take pictures of your motherboard BIOS settings (as much as you can).

 

This is the best way to start diagnosing.  Start with #1 from above.

 

- Jon

 

1) Didn't attempt, sorry but didn't work for the other 2 people, certainly will if still needed.

 

2)

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS with KVM

  menu default

  kernel /bzimage

  append pcie_acs_override=downstream initrd=/bzroot

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

label Xen/unRAID OS

  kernel /syslinux/mboot.c32

  append /xen --- /bzimage --- /bzroot

label Xen/unRAID OS Safe Mode (no plugins)

  kernel /syslinux/mboot.c32

  append /xen --- /bzimage --- /bzroot unraidsafemode

 

3)

Looping (started from reboot sequence, it's a bit long)

https://www.youtube.com/watch?v=LNkcxr4Y9Gk

 

Normal Boot (for comparison, also long and a complete boot)

https://www.youtube.com/watch?v=L3vBf-cPZbE

 

4) Non UEFI is default, no special options, most stuff unneeded turned off (blurry, but still informative, resized/zipped, download button at the top)

https://drive.google.com/file/d/0B0_CMCHyY8iWNVgxU0tNMGYtams/view?usp=sharing

 

[archedraft]

So i noticed that all three of us are using "USB: FLASH Drive" to boot with but there is also an option "UEFI: FLASH Drive". I am not 100% sure what the difference is but it is suppose to be newer technology. Maybe the newer linux kernels like UEFI?? peter_sm or bungee91 do you guys have any time this morning to try and change that setting and see what happens?

[bungee91]

I unfortunately don't until late this evening.

I wouldn't think this would be the issue considering older MB's don't have this option, and there has to be some running 6b10a, however who knows!?.

 

[peter_sm]

So i noticed that all three of us are using "USB: FLASH Drive" to boot with but there is also an option "UEFI: FLASH Drive". I am not 100% sure what the difference is but it is suppose to be newer technology. Maybe the newer linux kernels like UEFI?? peter_sm or bungee91 do you guys have any time this morning to try and change that setting and see what happens?

Did a test with "UEFI: FLASH Drive" , with only this settings it's jumps directly in to BIOS settings when trying to boot, so NO, UEFI don't work, need USB: but would http://bit.ly/11nOIuD be something for LT to look at ?

 

//Peter

[bungee91]

So i noticed that all three of us are using "USB: FLASH Drive" to boot with but there is also an option "UEFI: FLASH Drive". I am not 100% sure what the difference is but it is suppose to be newer technology. Maybe the newer linux kernels like UEFI?? peter_sm or bungee91 do you guys have any time this morning to try and change that setting and see what happens?

Did a test with "UEFI: FLASH Drive" , with only this settings it's jumps directly in to BIOS settings when trying to boot, so NO, UEFI don't work, need USB: but would http://bit.ly/11nOIuD be something for LT to look at ?

 

//Peter

 

Which also confirms the difference that you get when not having an available boot device, and the reboot loop that we're seeing. Meaning it tries to boot from USB, fails, and then continues to loop trying again, again....

[archedraft]

I figured it was a long shot. Such a strange issue for sure, normally I have a few ideas to try and fix the issues but on this one I am at a loss?

[jonp]

Still researching this one guys... 

[jonp]

Spoke with Tom and Eric on this today and we're investigating now.  This is a bit of a tricky one, but we'll figure it out!

[archedraft]

Sweet! Thanks for the help.

[jonp]

OK this is a tough one because we aren't able to replicate the issue and it occurs before unRAID even loads.  There is nothing in syslog either.  We are prepping for another release so we will need to check in after the update.

[garycase]

I've been following this with interest, as it's a REALLY strange issue. 

 

As Jonp noted, it occurs before the OS even loads, so it's really strange that it's in any way OS-related.  But clearly the difference in behavior between restart and power-on indicates SOMETHING is likely being flagged in either the CMOS or the system memory that's impacting the boot process (the memory is, of course, cleared by a power-off, which explains the different behavior). 

 

In addition, the problem does NOT occur on the exact same hardware with v5.0.5 !!

 

The latter, together with the fact there are now 3 folks with the same issue, is a pretty good indication that v6 is doing something really weird here ... but just what that might be is certainly perplexing.

 

 

[jonp]

Ok guys, I finally was able to get this to happen on one of my systems as well.  It's not consistent, I have no clues yet as to what is exactly causing it, but it is occurring.  The motherboard it is affecting of my test systems is the ASRock Fatal1ty Z87 Pro.  I have latest BIOS on it and my BIOS settings are solid.

 

So the good news is that we at LT have been able to recreate the issue on a test system in our lab.  The bad news is we still don't have a fix for it yet.

 

Just keeping everyone informed...

[archedraft]

Whoooo we are not crazy! It's really interesting that it only seems to affect ASRock boards. I really do like puzzles, unfortunately this one is way beyond what I could figure out. Thanks for the update and the continual effort into solving this.

[garycase]

Definitely good that Limetech can replicate this VERY strange issue.    With a single instance, it seemed very unlikely it was OS-related, since the problem is occurring pre-boot.    But when two of you ...and then three of you ... all had the same issue -- and the test I suggested of trying it with v5 showed that v5 doesn't have the issue -- it was clear this was some kind of very weird problem that WAS related to the OS.

 

Now that LT can replicate it, it will be much easier for them to isolate just what's happening.    It's indeed strange that it only happens with AsRock boards -- although it's not tied to a specific chipset,  since it's now occurring on Z77, H77, Z87, and Z97 boards !!

 

Question:  Do you all have your boards set to boot in UEFI mode?    If so, it might be useful to confirm whether or not the issue still occurs in legacy mode.

 

 

[archedraft]

I'm not 100% sure what UEFI mode is so forgive my ignorance if my response isn't what you were asking. I know I can select to boot my usb stick from USB or UEFI mode. I have always chosen USB mode because I wasn't sure what UEFI mode meant.

[garycase]

UEFI is the "Universal Extensible Firmware Interface" ... which is a replacement for the BIOS (Basic Input/output System) that has been since the early days of CP/M in the late 70's (although clearly it evolved a good bit).

 

Although the intent of UEFI was to REPLACE the BIOS, most vendors still call the code the provides the layer between the OS and hardware the "BIOS", and simply refer to the normal operating mode UEFI, with a legacy BIOS interface included as part of the UEFI.

 

In most implementations, you can switch between UEFI and "legacy" mode for the boot devices.    I don't have an AsRock board, so I can't look in their implementation and see what they call the two modes -- the "USB mode" you're using may in fact be the legacy mode.    In any event, my question is whether the behavior is the same regardless of which mode you select.

 

I'd think it would be, but clearly SOMETHING is happening here that's persistent throughout the next boot attempt.    And the UEFI DOES communicate slightly differently with the OS than a traditional BIOS ... so this may somehow be involved.

 

Without an AsRock system, I can't help isolate this ... but if I did [and if one of you has a couple hours to help with the diagnosis you may want to], I would start trying all of the v6 Betas to isolate exactly WHICH version first exhibited this problem.  I'm sure that knowledge would make it FAR easier for LT to isolate just what's causing this.