Will adding a second graphic card eliminate Error 43 with my GTX 1080?

[MacDingo]

Greetings dear UnRAIDers,

I converted my former gaming PC into an UnRAID server and overall things are working quite well.

 

The ONE thing I still cannot get to work is the passthrough of my single "Zotac GeForce GTX 1080 AMP Extreme 8GB" into Windows VMs.
My motherboard is a Gigabyte GA-Z170X-Gaming 7.
I do not game much but I would like to still be able to use my rather powerful PC as a gaming device from time to time (no dual boot).

This PC still has it's own screen, keyboard and mouse (which I understand should help with having something plugged into the graphics card)

 

I have watched tons of videos and tried all possible things I could find in the forums:

1)Ensure the 1080 has its own IOMMU group:

IOMMU group 15:    [10de:1b80] 01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1)
IOMMU group 16:    [10de:10f0] 01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) 

2) Editing the XML to ensure the virtualised graphic and sound carda are using the "same slot"

3) Dumped my 1080 bios and added it to the VM template

 

Yet I have still been unable to get around that Error43 code.

 

Now I read that many people are using a cheap second graphic card to make THAT ONE the main one for UnRAID and free up the more powerful graphics cards for the VM.

 

If it's what it takes to finally solve my problem I'd happily buy another card but:

1) How good are the odds that this really solves the issue

2) Is there are card that people here have tried that has a "proven" compatibility with UnRAID

3) Should I go for another Nvidia card or something different.

 

PS: Incidentally, if someone happens to have that same motherboard and graphics card and managed to get things to work, I'd love to hear from you!!

 

Any thoughts and advice greatly welcome!

[testdasi]

What's your CPU?

If your CPU has iGPU then there's no need to buy another low-end GPU for Unraid, especially with a Gigabyte mobo.

In BIOS, look for "Initial Display Output" and pick the iGPU. Unraid will then boot with the iGPU (make sure to connect the display to the mobo HDMI / DP output to check that it is indeed booting on the iGPU.

 

If your CPU doesn't have iGPU then any cheap one will do (e.g. GT710 is a popular choice on here). With Gigabyte mobo, keep your 1080 on the 1st PCIe slot and plug the lowend to either of the slower slot and then pick that slot for Initial Display Output in the BIOS.

 

In terms of effectiveness in resolving error code 43, iGPU and low-end GPU are equivalent.

It's a matter of whether your CPU has a iGPU or not for you to decide on which option.

 

 

Now you said you have dumped your own vbios for the 1080.

• Did you dump it when Unraid is NOT booting with the 1080?
• Did you dump it with the 1080 in the 1st PCIe slot?
• Did you perform the hex edit on the header (i.e. following SpaceInvader One tutorial on Youtube)?

If you answer no to any of the above question, redump your vbios.

 

 

As in the "odds" of it resolving the error, you will understand it better if you know the background.

 

So Nvidia wants you to buy the expensive Quadro to use in virtualised environment so the Nvidia driver will refuse to load if it detects a consumer card (e.g. GTX) being used in a VM. The error code 43 is actually a generic code that says "it doesn't work" (i.e. a wonky GPU / incomplete pass-through / not enough power etc. will all have the exact same error so you have to be 100% sure that "it works" before even considering any resolution).

 

One of the detection methods is to see if the GPU has been initialised before being used in the current machine. Therefore,

• A vbios is critical because it makes the GPU act as if it has been freshly turned on (as well as helping the GPU initialise properly to the VM - hence it's better to NOT use a vbios rather than using the wrong one!).
• Having a GPU for Unraid to boot with (either iGPU or low-end dedicated) ensures the to-be-passed-through GPU is not initialised prior to being passed-through (because Unraid doesn't boot with it initially). So it generally helps with resolving error code 43.
• Another thing you can try is to boot Unraid in legacy mode (i.e. NOT UEFI). Some cards for whatever reason, don't like booting in UEFI before being passed to a VM.

So as you can see here, booting Unraid with a non-1080 will help your case but it is no guarantee (e.g. a wrong vbios won't work regardless of any other resolution, hence my comment above to redump the vbios).

 

Another detection method is via Hyper-V. The old advice (circ. 2018) was to disable Hyper-V in the VM template.

The new advice is to keep it on and make these xml edits:

Add dummy vendor ID tag in the Hyper-V section.

    <hyperv>
...
      <vendor_id state='on' value='0123456789ab'/>
...
    </hyperv>

and add this kvm tag:

    <kvm>
      <hidden state='on'/>
    </kvm>

 

 

From my personal experience, what I did (actually just a few months ago due to wanting to factory reset my workstation VM) was:

1. Boot Unraid in legacy mode with my GT710 on 3rd PCIe slot and my 1070 on the 1st PCIe slot.
   Remember that Initial Display Output settings! that's the whole point of getting a Gigabyte mobo for Unraid in my opinion.
2. Dump my 1070 vbios properly (with edit etc. i.e. SpaceInvader One tutorial)
3. Start a VM template (Q35 + OVMF is my preferred choice) with Hyper-V + VNC display.
4. Install Windows + turn on RDP (so I can remote access to check for issues if the GPU doesn't work, although I actually didn't need it)
5. Edit my VM template to remove VNC and pick the 1070 GPU with vbios and HDMI Audio + the aforementioned hyperv / kvm xml edits.
   I didn't make the GPU + Audio on the same bus with multifunction but I would still recommend everyone to do it regardless.
6. Boot my VM (which booted successfully with display from the 1070). Install Nvidia drivers, reboot a few times (including a full shutdown + start) to be 100% sure no error.
7. Reboot Unraid to BIOS. Change Initial Display Output to 1st slot (the 1070), reboot to Unraid (which now uses the 1070 to boot!) and turn on the VM to re-verify that it would work without the need for the GT710.
8. Shut down Unraid, remove the GT710 and run my Unraid with only 1 GPU.

In other words, the GT710 was mainly to dump the vbios.

 

 

 

PS: next time you have a problem, it's generally a good idea to attach diagnostics zip (Tools -> Diagnostics).

Edited March 12, 2020 by testdasi

[MacDingo]

@testdasi Many thanks for that detailed answer!

Well great, that gives me some new things to try out!

 

My CPU is an Intel i7-6700K Processor (6th gen Skylake). 

According to spec sheet I should indeed have an Intel HD Graphics 530 on board, so back to the BIOS to check that out.

 

For the BIOS dumping I had followed SpaceInvader's video, but did indeed that with my 1080 being the only active card in the system.

 

I will follow your instructions and see where that takes me.

 

Thanks for the quick and extensive answer, you just gave me new hope

 

I will post a follow-up as soon as I have new and relevant information!

[MacDingo]

@testdasi 

I would like to report progress, and it is GOOD progress (yeah!).

After activating the iGPU in the BIOS I simply rebooted my VM and... the error code 43 was gone!

The 1080 card is properly listed and I do get the output on the monitor directly attached to the PC.

We're making progress!

 

So I now have one item in my device manager that is no longer recognized (not sure which one it is) and my audio doesn't seem to work (except through my headset). So a few things I still need to look into, but I am quite happy already so far!

 

On a side note, with regards to using the right Nvidia drivers:

1) Should I stay on the default Windows ones? Use the Geforce Experience ones or some other regular Nvidia drivers?

2) I assume that once I opt for a version I can safely update them within the VM going forward (through direct download.update)?

[testdasi]

Silly question but have you passed through the HDMI Audio? Is your Monitor using HDMI input with audio with volume up? Check your Device Manager to see if you have NVIDIA High Definition Audio?

I once spent 30 minutes trying to fix audio issue and then realised my monitor had volume at 0.

 

Then see the SpaceInvader One tutorial below and get the msi_util (msi interupt fix) from the vid description and tick your GPU and Audio device.

 

In terms of driver, google "nvidia patch windows" to find the github site and use the latest supported driver from there ("Driver Link" are direct links to Nvidia website so they are genuine). I usually pick the latest Studio Driver but you may pick anything latest.

Then read the github page to understand what the patch does and apply it if you want to.

It's like Fight Club. I'm not supposed to talk about Fight Club. 😉

 

 

 

[MacDingo]

Hi @testdasi

I had passed the Nvidia audio portion through and the sound card appears in the device manager just fine. I don't have any errors in the device manager.

 

So, I get audio through my screen (display port connection), my headset and even through remote VNC, but not through my speakers (!).

I tried connecting other speakers (in case) but no sound there either. And yes I did volume checks as well :)

 

I downloaded the MSI file but when I launch it, strangely enough the window stays empty (nothing to select as per SpaceInvader's video).

Is it maybe no longer compatible with the recent UnRAID releases? (I'm on 6.8.3)

 

Btw, I haven't yet re-disabled the built-in eGPU as per your first reply. After so many months of not getting the Nvidia card to work, I am hesitant to screw up my VM and not being able to get it back working again (but I did copies of the vdisk.img).

 

Well I guess some more tinkering ahead :D