Too many USB devices leads to failure to boot

[harperhendee]

First things first--I have a lot of USB devices.  Take a look at this giant list.  In particular, have a look at what the Vive adds to the mix:

 

USB Devices

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:800a Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 8087:8002 Intel Corp.
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 002: ID 047d:2043 Kensington
Bus 003 Device 003: ID 8087:0a2b Intel Corp.
Bus 003 Device 004: ID 1e71:170e NZXT
Bus 003 Device 005: ID 045b:0209 Hitachi, Ltd
Bus 003 Device 006: ID 045b:0209 Hitachi, Ltd
Bus 003 Device 007: ID 1b1c:1c08 Corsair
Bus 003 Device 008: ID 0930:6544 Toshiba Corp. TransMemory-Mini / Kingston DataTraveler 2.0 Stick (2GB)
Bus 003 Device 010: ID 045e:00cb Microsoft Corp. Basic Optical Mouse v2.0
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 002: ID 045b:0210 Hitachi, Ltd
Bus 004 Device 003: ID 045b:0210 Hitachi, Ltd
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 002: ID 2833:0211  
Bus 006 Device 003: ID 2833:0211  
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 002: ID 045e:02e6 Microsoft Corp.
Bus 007 Device 003: ID 1a40:0101 Terminus Technology Inc. Hub
Bus 007 Device 004: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 007 Device 005: ID 2833:0211  
Bus 007 Device 006: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 007 Device 007: ID 2833:2031  
Bus 007 Device 008: ID 2833:0031  
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 002: ID 2109:0812 VIA Labs, Inc. VL812 Hub
Bus 008 Device 003: ID 2833:3031  
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 009 Device 002: ID 28de:1142  
Bus 009 Device 003: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 009 Device 004: ID 046d:c52b Logitech, Inc. Unifying ReceiverBus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 009 Device 002: ID 28de:1142  
Bus 009 Device 003: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 009 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 009 Device 005: ID 0bb4:2744 HTC (High Tech Computer Corp.)
Bus 009 Device 006: ID 0bb4:2134 HTC (High Tech Computer Corp.)
Bus 009 Device 007: ID 0bb4:0306 HTC (High Tech Computer Corp.)
Bus 009 Device 008: ID 0424:274d Standard Microsystems Corp.
Bus 009 Device 009: ID 0d8c:0012 C-Media Electronics, Inc.
Bus 009 Device 010: ID 0bb4:2c87 HTC (High Tech Computer Corp.)
Bus 009 Device 011: ID 28de:2101  
Bus 009 Device 012: ID 28de:2101  
Bus 009 Device 013: ID 28de:2000  
Bus 009 Device 014: ID 0bb4:2c87 HTC (High Tech Computer Corp.)
Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 010 Device 002: ID 2109:0812 VIA Labs, Inc. VL812 Hub

Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 010 Device 002: ID 2109:0812 VIA Labs, Inc. VL812 Hub

Notice that there are 10 US buses, and 36 devices!  If I plug the Vive into a motherboard USB slot, I get the following:

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:800a Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 8087:8002 Intel Corp.
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 002: ID 047d:2043 Kensington
Bus 003 Device 003: ID 8087:0a2b Intel Corp.
Bus 003 Device 004: ID 1e71:170e NZXT
Bus 003 Device 005: ID 045b:0209 Hitachi, Ltd
Bus 003 Device 006: ID 045b:0209 Hitachi, Ltd
Bus 003 Device 007: ID 1b1c:1c08 Corsair
Bus 003 Device 008: ID 0930:6544 Toshiba Corp. TransMemory-Mini / Kingston DataTraveler 2.0 Stick (2GB)
Bus 003 Device 009: ID 0bb4:2744 HTC (High Tech Computer Corp.)
Bus 003 Device 010: ID 045e:00cb Microsoft Corp. Basic Optical Mouse v2.0
Bus 003 Device 011: ID 0bb4:2134 HTC (High Tech Computer Corp.)
Bus 003 Device 012: ID 0bb4:0306 HTC (High Tech Computer Corp.)
Bus 003 Device 013: ID 0424:274d Standard Microsystems Corp.
Bus 003 Device 014: ID 28de:2000  
Bus 003 Device 015: ID 0bb4:2c87 HTC (High Tech Computer Corp.)
Bus 003 Device 016: ID 0d8c:0012 C-Media Electronics, Inc.
Bus 003 Device 017: ID 0bb4:2c87 HTC (High Tech Computer Corp.)
Bus 003 Device 018: ID 28de:2101  
Bus 003 Device 019: ID 28de:2101  
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 002: ID 045b:0210 Hitachi, Ltd
Bus 004 Device 003: ID 045b:0210 Hitachi, Ltd
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 002: ID 2833:0211  
Bus 006 Device 003: ID 2833:0211  
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 002: ID 045e:02e6 Microsoft Corp.
Bus 007 Device 003: ID 1a40:0101 Terminus Technology Inc. Hub
Bus 007 Device 004: ID 2109:2812 VIA Labs, Inc. VL812 Hub
Bus 007 Device 005: ID 2833:0211  
Bus 007 Device 006: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 007 Device 007: ID 2833:2031  
Bus 007 Device 008: ID 2833:0031  
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 002: ID 2109:0812 VIA Labs, Inc. VL812 Hub
Bus 008 Device 003: ID 2833:3031  
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 

Plugging into the PCIE card causes the Vive USB devices to show up earlier in the enumeration sequence.  I'm not sure why this affects things, but it appears to work.  If I want to boot from a USB drive, I need to unplug the VIVE from the PCIE card.  This is kind of pain in the ass.  But I don't know that my BIOS can cope with this mysterious enumeration sequence.

 

 

[BRiT]

No, most likely when you plugin the VIVE PCIE Card you simply need to go into BIOS and re-adjust the boot order so that your FLASH drive is the device selected to boot.

 

Computer systems are notorious for being a pain in the ass whenever you add or remove devices when it comes to boot order. Make sure the boot order is still the same after you plugin the new device.

[harperhendee]

I just did a bunch of experiments with this last night.  The difference between the passing case and the failing case is that in the failing case, I cannot boot to the USB device, regardless of what I do in BIOS.  BIOS simply doesn't see it as an option, even over multiple power cycles.  

 

You'll notice that in both cases, the Kingston Datatraveller has the same device credentials.  I purposely connected it to the MB ports that went directly to the chipset so that it would be enumerated as simply as possible.  All the "basic" systems are this way--hard wired mouse+keyboard, controls to power supply and cooler, the unraid thumb drive.  I have a few thoughts on what is going on...

 

1)  The enumeration sequence in unraid is different than the enumeration sequence in early boot, or in BIOS.  

In early boot, there is only one USB 2.0 port that is monitored so that FW patches can be applied.  Early boot is controlled by the PCH (the motherboard chipset) and runs on an embedded CPU that cannot be controlled by the user.  

During BIOS execution, all the USB devices are discovered and enumerated.  This is how BIOS sees the USB world.

During OS execution, all the USB devices are rediscovered and enumerated.  This is how Unraid sees the USB world

 

2)  I only have visibility into the mapping during OS execution.  Perhaps BIOS execution phases experiences discovery failures that go unreported.  

 

3)  BIOS seems to always prefer the "last" USB device.

When I have three USB keyboards plugged in:  

a)  Kensington direct to Motherboard

b)  Logitech A on PCIE USB card at 4e

c)  Logitech B on PCIE USB card at 54

 

BIOS will only recognize Logictech B in this case.  I'm not sure what I can infer from that, other than the USB stack that BIOS uses is more primitive than what an OS provides.  Remember, USB enumeration is an OS level task.  The code that discovers USB during BIOS is entirely different than the code that does it again using OS runtime.  It is entirely possible that some topologies will confound the BIOS code.  BIOS code is written by a handful of SW programmers using "generic" lab builds that have few tested configurations.   There isn't a lot of feedback from end uses to those BIOS writers.  So I can easily believe there are "fatal" USB topologies.

 

In any case, I hope this info is useful to the folks at Limetech.  I think there is a real danger for relying on USB devices for boot.  The USB topology is fussy and dynamic.  It is hard to guarantee a deterministic and reliable boot recipe including USB.

 

I think the solution is to boot from a PCIE mapped device and use the USB thumb drive as a license dongle.  The USB stick as OS works brilliantly when it works, but it is a huge problem if something goes wrong here.  The problems found in USB enumeration are hard to understand and harder to solve. This is not a good place for Limetech to provide technical support, but it prevents users from using Unraid, so it is a problem for the company.

[Squid]

If the BIOS also recognizes the unRaid stick as a Hard Drive, then set the BIOS to boot from hard drive first, and in the BBS priorities set the flash to be first (actually the only one is nice and safe).