Hi Guys,

 

First i had an issue with my i5 not support IOMMU so i bought an i7.

Getting closer i can select my graphics card from the list but it throws an error when i start.

I have tried to follow the guide and i get this error.

can anyone please help?

 

root@Vault:/sys/bus/pci/devices/0000:02:00.0# cat rom > /mnt/user/Downloads/msi1070.dump
cat: rom: Input/output error
 

02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Micro-Star International Co., Ltd. [MSI] Device 330c
        Flags: fast devsel
        Memory at f4000000 (32-bit, non-prefetchable) [disabled]
        Memory at e0000000 (64-bit, prefetchable) [disabled]
        Memory at f0000000 (64-bit, prefetchable) [disabled]
        I/O ports at d000 [disabled]
        Expansion ROM at f5000000 [disabled by cmd]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [250] Latency Tolerance Reporting
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [420] Advanced Error Reporting
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Capabilities: [900] #19
 

 

When i start the VM i get this error.

Execution error

internal error: qemu unexpectedly closed the monitor: 2018-01-09T04:29:08.439513Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2018-01-09T04:29:08.439531Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 1
2018-01-09T04:29:08.439540Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.0,addr=0x5: Device initialization failed

 

 

 

Here is my VM XML file is there an error here, have i missed something?

i have tried to run it in Slot 1 and just moved it to Slot 2 but that has made no difference.

 

<domain type='kvm'>
  <name>Miner</name>
  <uuid>96ea6ac6-0537-c284-a455-d1b5d6d15a27</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='6'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/96ea6ac6-0537-c284-a455-d1b5d6d15a27_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/Vm/Miner/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows10.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:2c:17:3a'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
</domain>
 

 

ok so i turned on pcie ASC override and it looks like that split out the 1070 from the IOMMU group 1.

Which means i can now start my VM with the graphics card..

some progress finally...

 

But in my kernel logs i now see the below but all my drives are there and no errors.

 

Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 16:11:50 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 16:11:50 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
 

 

actually one of my drives i can't see the temperature for it seems to be the SATA controller its plugged into that is throwing the above errors.

Anyway i can keep the IOMMU groups divided so i can use my 1070 and exclude 04:00.1 from this division. 

it was suggested to use append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

in the syslinix config file over pcie_acs_override=downstream 

 

will that work or break things more..

If this is the only GPU in the system, you need to get a vbios and add it in the XML. I think @gridrunner has a collection of vbioses if you don't manage to dump it yourself. You can also use on from techpowerup and strip it per gridrunners guide. 

Second thing is that you should not use vnc together with GPU pass through. That will mostly lead to a black screen. 

i dumped my vbios and put the rom in the VM.

but i can only do that with PCIE ACS on, otherwise i cannot access the device.

 

passthrough is working fine at the moment but i am seeing a heap of errors in my log files.

 

Jan  9 19:08:26 Vault kernel: dmar_fault: 15 callbacks suppressed
Jan  9 19:08:26 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 19:08:26 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 19:08:33 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 19:08:33 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 19:08:33 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 19:08:33 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
Jan  9 19:08:33 Vault kernel: DMAR: DRHD: handling fault status reg 3
Jan  9 19:08:33 Vault kernel: DMAR: [DMA Write] Request device [04:00.1] fault addr 39d9c0000 [fault reason 02] Present bit in context entry is clear
 

So trying to find a way to keep the passthrough working but get rid of the errors, there must be another way other than turning on PCIE ACS.