Unraid in headless operation with 3 GPUs

[harperhendee]

I have a server with 3 NVIDIA GTX 1070 cards in it.  Currently, I am booting up unraid on one of the cards, then launching gaming VMs on the other two.  However, I'd really like to run a third gaming VM with the third card, which unraid/bios is displaying to by default.  When I try to pass this GPU to my third VM, I lose the monitor for unraid, and get nothing out of the GPU.  When launching that VM, I see CPUs doing something that looks like boot, but then no display--just a blank screen.  

 

The solution that most folks seem to do is to use integrated graphics or a cheap PCIe card.  However, I am completely out of usuable PCIe slots.  Currently, I am using 5:

Slot 1 : GFX 1

-Gap-

Slot 2 : USB 1

Slot 3: GFX 2

Slot 4: Covered by GFX 2

Slot 5: USB 2

Slot 6: GFX 3

 

I've had a bunch of harebrained ideas on how to get my third GFX card back:

 

1.  PCIe 1x extension cable to cheap-o GFX card

Use a PCIE 1x extension cable to run to the back of the case and connect to a jerry-rigged cheapo PCIE 1x card

Cost: ~$50

Pros:  Should be easy to set up and fairly cheap.

Cons:  It is super tight in there, and already seeing some signs of signal integrity problems with the PCIE bus.  The PCIE 1x extension cable I have is not long enough.  

 

2.  U.2 to PCIe riser to cheap-o GFX card

Same as solution #1, but use adapters to repurpose the unused U.2 connector

Cost: ~$75

Pros:  Easy cable routing

Cons:  Many risks and unknowns doing this kind of PCIE to U.2 to PCIE to GFX connection.  I'm not sure if the MOBO would recognize it.   I'm not sure if the PCIE routers would deal with a GFX card where normally an SSD would exist.

 

3.  Use a USB to VGA converter for unraid

I tried this approach with a USB3 hub with integrated DVI out.  The problem was that as soon as I put this hub in, I could no longer boot from my USB flash drive.  My BIOS couldn't map it, due to the changes in the USB hub topology.  

Cost: ~$20-50 (or $0 if I can find one laying around)

Pros:  Super simple physical install, inexpensive HW

Cons:  It may not solve my problem at all, since the BIOS probably won't detect this as a display adapter

 

4.  Convince unraid to run headless, free up all GFX cards

Cost: $0

Pros:  No changes to HW at all.

Cons:  No monitor for debugging.  Might not allow the MOBO to boot

 

5.  Discover the magic incantation that lets the VM steal the card from unraid

Cost: $0

Pros:  No changes to HW

Cons:  I can't get it to work...

 

Any feedback on which is the best path to pursue?

 

--Harper

[thomas]

You might try to dump the BIOS and add it to the VM xml. see post