Ryzen and dual GPUs, 1 passthrough works great, the other does not.

[xanvincent]

AMD Ryzen 1700x build, per some advice on here, I went ahead and bought another video card. Since the Ryzen CPU doesn't have onboard graphics, I'm having a bit of trouble getting them to passthrough correctly.

 

First PCIe 16x slot: AMD Vega 56

Second: Nvidia GTX 1070Ti 

 

The second card passthrough works just fine, and is a huge success; however, if possible, I'd like to make use of the first slot as well. I've added: 

vfio-pci.ids=1002:687f,1002:aaf8

to my syslinux.cfg, but it doesn't seem to make a difference. Have tried SeaBIOS and OVMF. Is it worth trying another nvidia card in the first slot? I thought AMD cards were supposed to "just work"? I've also tried another AMD card, the RX 580 in this slot, couldn't get that to work either. Ideas greatly appreciated!

 

Log Output: https://pastebin.com/13Uh2iCv

XML: https://pastebin.com/rxKstPEe

[PROJECTBLUE]

You could try passing the ROM to the graphics card.

I had this issue too when I only had 2 GPU's, the first GPU in the system just would not allow me to pass it through no matter what I did.
To get both of my 290X's passed through to separate VM's I had to add a 3rd GPU in the slot above both 290X's for unraid to use to let me pass them through.

I even tried following a guide to run unraid headless but that didn't help at all.

Edited April 19, 2018 by PROJECTBLUE

[xanvincent]

Appreciate the response. I think this is the next step. I'm gonna try to dump the 1070 ROM and use that as the primary. 

 

I don't really have room to add a third card, so I really need this to work, as everyone says it should...

[PROJECTBLUE]

36 minutes ago, xanvincent said:

Appreciate the response. I think this is the next step. I'm gonna try to dump the 1070 ROM and use that as the primary. 

 

I don't really have room to add a third card, so I really need this to work, as everyone says it should...

Let me know how it goes!

Do you not have a spare PCIE slot to use for a low powered GPU?

[xanvincent]

27 minutes ago, PROJECTBLUE said:

Let me know how it goes!

Do you not have a spare PCIE slot to use for a low powered GPU?

I do, but it'd be under the two video cards, I'm not sure I can tell unRAID to pick the third card right? I thought it will just pick the first card it sees...

[JonathanM]

43 minutes ago, xanvincent said:

I thought it will just pick the first card it sees...

It will, but most decent BIOS implementations will allow you to choose the primary display. It's controlled by the motherboard.

[dispake]

8 hours ago, xanvincent said:

I don't really have room to add a third card, so I really need this to work, as everyone says it should...

 

So I was about to try and go down the same road / build as you.  Curious which threads you're referring to that says passover of multiple GPUs should work and how to do it?

 

10 hours ago, PROJECTBLUE said:

You could try passing the ROM to the graphics card.

I had this issue too when I only had 2 GPU's, the first GPU in the system just would not allow me to pass it through no matter what I did.
To get both of my 290X's passed through to separate VM's I had to add a 3rd GPU in the slot above both 290X's for unraid to use to let me pass them through.

I even tried following a guide to run unraid headless but that didn't help at all.

 

Question:  What ROM are you guys referring to?  Are there specific ROMs that need to be utilized for unRAID to be able to pass them through?

 

Also, based on your comment, does that mean one GPU always needs to be dedicated to the unRAID server?  but sounds like there's a way to run unRAID headless but not 100% feasible?

 

 

[PROJECTBLUE]

13 hours ago, dispake said:

 

So I was about to try and go down the same road / build as you.  Curious which threads you're referring to that says passover of multiple GPUs should work and how to do it?

 

 

Question:  What ROM are you guys referring to?  Are there specific ROMs that need to be utilized for unRAID to be able to pass them through?

 

Also, based on your comment, does that mean one GPU always needs to be dedicated to the unRAID server?  but sounds like there's a way to run unRAID headless but not 100% feasible?

 

 

I'm referring to the VBIOS of the graphics card. It's not always necessary to use them but with my system it would get stuck on a black screen if I didn't pass the VBIOS ROM to the GPU's.

 

For me it seems that way. I tried to run it headless on my X99 system (no integrated GPU) but every time I tried to boot the VM that was assigned the GPU in the topmost PCIE slot it would not boot the VM (black screen just like if not passing the VBIOS). I just added an old HD5450 dedicated for the unraid GUI and now I have no issues passing my 290X's to individual VM's.

Edited April 20, 2018 by PROJECTBLUE

[dispake]

7 hours ago, PROJECTBLUE said:

For me it seems that way. I tried to run it headless on my X99 system (no integrated GPU) but every time I tried to boot the VM that was assigned the GPU in the topmost PCIE slot it would not boot the VM (black screen just like if not passing the VBIOS). I just added an old HD5450 dedicated for the unraid GUI and now I have no issues passing my 290X's to individual VM's.

 

I had a somewhat similar problem where my BIOS would use PCIE as priority (over integrated) but if I didn't have a monitor hooked up to the GPU, it wouldn't boot properly.  I got a HDMI dummy plug (https://www.amazon.com/Headless-Display-Emulator-Headless-1920x1080-generation/dp/B06XT1Z9TF) and then things started working.  But, I did also change my BIOS to default to the integrated GPU so not sure if that helped too.

[fr05ty]

i too was having a similar problem i tried both of SI1's video's and i can't get either to work, with the easy way (tech powerup rom) i get a code 43 and 800x600 res when i try to pass through the zotac 1060 6gb by it self, i have an old gt250 that i had to blow the cobwebs off and put in the first slot, got the win10 vm to boot with the 1060 passed through with no rom and it worked so i then proceeded to follow the how to dump a rom video all worked as i went along then i pulled the gt250 out started the vm and all i got was a black screen. 

after a lot of reading on the forum and reddit i found this little gem (change unraid boot usb from uefi to legacy mode) rebooted started win10 vm reinstalled the nvidia drivers and i was away its all working now

Edited April 21, 2018 by fr05ty
missed text on what i changed