Hey,

Ive replaced my old GTX 1070 GPU with a new 2080 RTX TI

after installing the new gpu my pcie_acs_override were missing, and i couldnt pass the gpu through(cuz it was in the same iommu group as other nvidia hardware ( serial controller etc), so ive set pcie_acs_override to "both" and rebooted and started the vm

Quote

Model: Custom

M/B: MSI - X370 SLI PLUS (MS-7A33)

CPU: AMD Ryzen 7 1700 Eight-Core @ 3700

HVM: Enabled

IOMMU: Enabled

Cache: 768 kB, 4096 kB, 16384 kB

Memory: 32 GB (max. installable capacity 256 GB)

Network: bond0: fault-tolerance (active-backup), mtu 1500 
 eth0: 1000 Mb/s, full duplex, mtu 1500

Kernel: Linux 4.18.15-unRAID x86_64

now the weird problem is

when i Read from the file share its limited to 5mb/s

while writing to it its 100mb/s

 

ive tried copying over the unraid terminal and there i get the usual 100mb/s

VM XML:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='1'>
  <name>Windows 10 neue gpu</name>
  <uuid>7a3fe785-1b82-f535-dbea-2bf70637982c</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>25165824</memory>
  <currentMemory unit='KiB'>25165824</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>14</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='7'/>
    <vcpupin vcpu='6' cpuset='8'/>
    <vcpupin vcpu='7' cpuset='9'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <vcpupin vcpu='10' cpuset='12'/>
    <vcpupin vcpu='11' cpuset='13'/>
    <vcpupin vcpu='12' cpuset='14'/>
    <vcpupin vcpu='13' cpuset='15'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/7a3fe785-1b82-f535-dbea-2bf70637982c_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='14' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:ba:02:ed'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 10 neue gpu/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/boot/NVIDIA.RTX2080Ti.11264.180829.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x26' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x28' slot='0x00' function='0x3'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x045e'/>
        <product id='0x0719'/>
        <address bus='7' device='6'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc32b'/>
        <address bus='3' device='2'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x04d9'/>
        <product id='0xa116'/>
        <address bus='7' device='2'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0d8c'/>
        <product id='0x0014'/>
        <address bus='1' device='3'/>
      </source>
      <alias name='hostdev6'/>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1852'/>
        <product id='0x7022'/>
        <address bus='7' device='3'/>
      </source>
      <alias name='hostdev7'/>
      <address type='usb' bus='0' port='5'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1a7c'/>
        <product id='0x0195'/>
        <address bus='7' device='5'/>
      </source>
      <alias name='hostdev8'/>
      <address type='usb' bus='0' port='6'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

Attached the Diagnostics

 

tower-diagnostics-20181022-1906.zip

Update:, without Gpu assigned and using qxl I get full read speed from the samba share, with the 2080rtx ti I get 5 mb/s max,

 

Talked with the vfio guys on discord and they got no clue, they might suggest it's because of the new 2080rtx ti not beeing supported yet,

 

Anyone else uses the 2080rtx ti successfully in unraid? Or another rtx Gpu? 

Hi Mister!

 

I actually have a 2080 Ti myself and have not seen this behavior in my VM.  Are you passing through the GPU, the GPU's audio device, and the two other devices (USB and Serial) that are all apart of the GPU?  Also, is there a reason you have to pass the ROM file for the GPU?

 

You also have a ton of plugins installed, I'd be curious if you experience the same issues booting in safe mode.  Worth a try!

Hey Jonp,

Thanks for the Help, 

safe mode same experience,

ive tried passing all nvidia pci devies too

 

i pass the rom file because i got a  ryzen cpu and only having one gpu , i needed that with gtx 1070 i had before because it never got initalized correctly without it

is it not needed anymore?

 

Update:

Tested file transfer over SFTP:

it starts at 100kb/s slows to 40kb/s  then this comes:

just tested that network model: <model type='e1000-82545em'/> instead of the virtio one

i get now 4 times the read speeds on the samba server, but still only 20MB/S instead of 100Mb/S+

 

 

while without the 2080rtx ti Assigned and using vnc to get on windows i get full samba read speeds 100MB/s

 

Also same Performance problems inside Linux VM's once the 2080rtx ti is assigned

 

local disk speeds are normal, writing to samba too,

just not reading:

 

Edited October 22, 20187 yr by MisterOutofTime