[Solved] KVM pass-through on X399 platform issues


Recommended Posts

I have issues trying to pass-through two Windows 10 KVMs, each with it's own dedicated GPU on X399 platform. I've been successfully using a similar configuration with Z97 platform & i7 4970K for three years, now I've upgraded to X399. Notable difference between the platforms is that X399 does not have the on-board graphics.

 

I've set all BIOS settings that make sense to me (AMD-v/SVM, IOMMU, C-state...) and I can't seem to get both VMs to use GPUs simultaneously - only the KVM with GPU, that's not used by motherboard's default output seems to work. The other VM starts, the unraid console on it's display turns black and nothing happens. I can VNC to the VM, but the GPU is not present in the OS. If I change the initial display output in BIOS they the issue moves to the other GPU/VM.

 

Currently I am using ACS override = downstream + multi-function without any specific vfio instructions.

label unRAID OS
  menu default
  kernel /bzimage 
  append iommu=pt pcie_acs_override=downstream,multifunction initrd=/bzroot

System:

  • Motherboard: X399 Gigabyte Designare EX (BIOS ver F11)
  • CPU: AMD Ryzen Threadripper 1950x
  • RAM: 4x 8GB Corsair Vengeance RGB Pro 3200 MHz
  • GPU 1: Nvidia GTX 970
  • GPU 2: Nvidia GTX 750 Ti
  • SSD cache: 1x 120GB SATA
  • SSD unassigned: 2x 250GB NVME, 1x 120GB SATA
  • HDD array: 6x 4TB, 2x 2TB
  • PCIe cards: SATA controller

 

PCIe slot configuration:

  • Slot 1: Nvidia GTX 970
  • Slot 2: /
  • Slot 3: SATA controller
  • Slot 4: Nvidia GTX 750 Ti
  • Slot 5: /

 

 

I've attached diagnostics. 

Does anybody have some idea what I have to do to make it work? vifo instead of ACS override? Disable Unraid console display output at boot?

 

triglav-diagnostics-20190705-0228.zip

Edited by Nihil
Marking solved
Link to comment

The card in the first slot passing through a VM for me needs a bios rom file. Dump it yourself or download the exact one for your card from Techpowerup. Check SpaceInvaderOnes youtube channel and you will find a couple tutorials how to do it.

Link to comment

I have the same issue and I tried it with 4 different GPUs each with two GPU BIOS dump (terminal and GPUz w/o NVheader) but on the X99 Platform maybe that is only a special problem with my Mainboard. Try to dump the vbios of your graphics card you have to listen carefully because every step and clue is important!

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.