ASUS ROG Zenith Extreme Alpha X399


Recommended Posts

Hey all. I have an issue. However, I have just read the first post. I will get back to you once I have made the changes to the BIOS. 


 

Execution error

internal error: qemu unexpectedly closed the monitor: 2019-06-14T16:42:54.367135Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio 0000:0b:00.0: failed to setup container for group 20: failed to set iommu for container: Operation not permitted

 

This was 1 VM I setup with 2 video card, GTX 1080 and a GTX 650TI.

 

I will keep you updated. 

 

Thank you

Link to comment
4 hours ago, authorleon said:

Hey all. I have an issue. However, I have just read the first post. I will get back to you once I have made the changes to the BIOS. 


 


Execution error

internal error: qemu unexpectedly closed the monitor: 2019-06-14T16:42:54.367135Z qemu-system-x86_64: -device vfio-pci,host=0b:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio 0000:0b:00.0: failed to setup container for group 20: failed to set iommu for container: Operation not permitted

 

This was 1 VM I setup with 2 video card, GTX 1080 and a GTX 650TI.

 

I will keep you updated. 

 

Thank you

Hi All. Okay, I did what John said on the first post. Thank you John. 

 

So, I have disabled ACS. 

 

The VM boots but I get not output from the GTX 650TI. 

 

Am I missing something ?

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='1'>
  <name>Windows 10 - TEST2</name>
  <uuid>f9c5482f-3fc5-e50d-d81d-87ad181944af</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='32'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/f9c5482f-3fc5-e50d-d81d-87ad181944af_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='2' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 - TEST2/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/W10X64.MULTi6.OCT2018.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:0c:e3:8e'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 10 - TEST2/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x2003'/>
        <address bus='9' device='2'/>
      </source>
      <alias name='hostdev2'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

Thank You 

Link to comment

Did some numa node & infinity fabric benchmarking CPU's and a graphics card (GTX 980ti Classified). I ran 3D Mark Timespy with the CPU's on the same numa node as the card and off and what I found was interesting. The GPU scores were in the same range but the CPU scores dropped off node . 2.2% slower on Timespy Extreme 4K and 9.5% slower on the regular Timespy.

 

A Passmark CPU benchmark with no graphical aspects other than Windows itself scored 0.6% faster on node.

 

Each benchmark was repeated 5 times and the results averaged.

Edited by jbartlett
  • Like 1
  • Upvote 1
Link to comment
17 hours ago, authorleon said:

Hi All. Okay, I did what John said on the first post. Thank you John. 

 

So, I have disabled ACS. 

 

The VM boots but I get not output from the GTX 650TI. 

 

Am I missing something ?

 


<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='1'>
  <name>Windows 10 - TEST2</name>
  <uuid>f9c5482f-3fc5-e50d-d81d-87ad181944af</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='32'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/f9c5482f-3fc5-e50d-d81d-87ad181944af_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='2' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10 - TEST2/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/W10X64.MULTi6.OCT2018.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:0c:e3:8e'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 10 - TEST2/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x2003'/>
        <address bus='9' device='2'/>
      </source>
      <alias name='hostdev2'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

Thank You 

Can anyone help me wit this please or point me in the right direction. 

 

Thank you

Link to comment
1 hour ago, jbartlett said:

Sorry, I haven't done anything with passing multiple GPU's to a VM before.

 Hello John, I'm not passing multiple GPUs.

 

PCI1 - 1080 GTX is for UNraid GUI at present

 

PCI4 - GTX 650 TI Boost

 

1VM using the GTX650 Ti Boost.  Your box standard set up .  When turning on the VM. Nothing is outputted from the GTX 650

Link to comment

I benchmarked a Win10 VM with PassMark's PerformanceTest assigned to the different Numa nodes on the 2990WX, separately and different combinations. The last column was truncated, it's all four numa nodes except CPU 15/47.

 

Each test was repeated five times and the results averaged.

 

641152592_2990WXNumaNodePassmarkCPUBenchmark.png.2a7384b28b91039b6c9d2ca970e5be6b.png

 

1169678436_2990WXNumaNodePassmarkMemoryBenchmark.png.349e454562f104248ac1915645fc690c.png

 

495364568_2990WXNumaNodePassmarkMemoryBenchmarkJitter.png.8a831da21f9d4a9f4712cfb43ca1d6e9.png

  • Like 1
Link to comment
3 hours ago, jbartlett said:

I benchmarked a Win10 VM with PassMark's PerformanceTest assigned to the different Numa nodes on the 2990WX, separately and different combinations. The last column was truncated, it's all four numa nodes except CPU 15/47.

 

Each test was repeated five times and the results averaged.

When running the benchmark, did you also set numatune to force memory load to the right NUMA node too?

 

Your Node 0 vs Node 1-3 looks suspiciously like your VM is fully allocated to node 0 only.

 

For VM that spread across multiple nodes, you might even want to force it so that memory is loaded evenly across different nodes. In my case, I have had to go as far as running 2 dummy VMs to load up 2 nodes to the right amount so that my main VM splits almost 50-50 across 2 nodes (and then shutdown the dummy VMs - thank goodness for CA User Scripts).

Link to comment
53 minutes ago, jbartlett said:

I didn't do any memory placement testing, just loaded up one VM with 8 GB of RAM assigned to it. This strikes me as how the vast majority of people will launch their VM's letting KVM do the memory assignments.

Indeed your tests make it clear that these WX Threadripper need some intervention for best performance. Unraid KVM is not smart enough to deal with it.

Link to comment
1 hour ago, testdasi said:

Unraid KVM is not smart enough

Even Windows isn't smart enough to handle the multi-Numa-node design. I remember watching Wendell in one of his videos showing that W10 in a VM on TR4 with extra Numa configuration runs way faster in multiple software and close to bare metal linux performance, than bare metal windows on the same system. I hope for the Ryzen 3000 series Microsoft is doin their homework.

Link to comment
  • 4 weeks later...

Hi guys,
First of all, very good documentation work you have done here. Very appreciated!


I am also planning to build an unraid system, where as one build option is the Threadripper 2920 or 50 with the discussed mainboard. This option should serve as an Unraid server but also contain a VM used as my normal PC. This VM should not only be capable for gaming but also not be too high maintenance due to whatever reason (e.g. Windows updates). The alternative to this option would be two separate builds which would be more expensive.

 

Now I have some questions relating to the Threadripper build if you would be so kind to answer:

1.      From your experience, would I, as a newbie to Unraid and advanced bios settings, be able to set up such a machine and keep it working? (as mentioned e.g. due to software updates or pass through stability issues)

2.      Threadripper has its handicaps with gaming in comparison to other modern CPU‘s. Do you think, adding another difficulty level (running games on the Windows VM instead on the base platform) will result in material performance losses?

3.      As far as I understood you were not able to pass through a GPU in PCIe 1 to a VM. Thus PCIe 1 is just usable for an Unraid GPU or another PCIe card other than a passed through GPU. Is this still correct or would it change with an AMD GPU?

4.      In connection to question 3. Is it correct that it is not possible to put the GPU dedicated to Unraid somewhere else than in the very valuable PCIe 1 slot (x16 + space)? Ideal would be the PCIe 4 (x4) but that does not work I guess.

 

Thank you very much in advance!

Link to comment

@Georg

1. OS updates are outside the scope of unraid as you would still face them if you built a bare metal system. In the case of Windows 10, you can "defer" updates for up to three months with Windows Pro.

2. Intel has handicaps too. You can expect to lose a few FPS running a gaming rig as a VM but not anywhere near you'd be able to notice assuming your video card is sufficiently powerful. I have not noticed any video performance drops if the video card & pinned CPU's aren't on the same NUMA node but I did see a CPU hit.

3. I haven't check for new BIOS updates in the past month or so but there was no option to select which PCIe slot the BIOS used for it's primary video card. I would believe that if you could change it, you would still experience the same thing but with whatever slot is given. Note: This is a known issue with nVidia cards, I have not tried any AMD video cards. The problem is that the nVidia driver can detect that the card was in an active state when it took over and then is coded to not load (Error 43)

4. You still need a video card in PCIe #1 for the BIOS to bind to. Omitting a video card in that slot would just cause the BIOS to bind to the next available slot with a video card.

 

 

Link to comment
On 7/15/2019 at 10:52 PM, jbartlett said:

@Georg

1. OS updates are outside the scope of unraid as you would still face them if you built a bare metal system. In the case of Windows 10, you can "defer" updates for up to three months with Windows Pro.

2. Intel has handicaps too. You can expect to lose a few FPS running a gaming rig as a VM but not anywhere near you'd be able to notice assuming your video card is sufficiently powerful. I have not noticed any video performance drops if the video card & pinned CPU's aren't on the same NUMA node but I did see a CPU hit.

3. I haven't check for new BIOS updates in the past month or so but there was no option to select which PCIe slot the BIOS used for it's primary video card. I would believe that if you could change it, you would still experience the same thing but with whatever slot is given. Note: This is a known issue with nVidia cards, I have not tried any AMD video cards. The problem is that the nVidia driver can detect that the card was in an active state when it took over and then is coded to not load (Error 43)

4. You still need a video card in PCIe #1 for the BIOS to bind to. Omitting a video card in that slot would just cause the BIOS to bind to the next available slot with a video card.

 

 

Thank you very much John for taking the time to answer!

Link to comment
  • 2 weeks later...

EUREKA! Now able to pass primary GPU to VM.

 

I got the idea from Spaceinvader One's latest videos about the AMD GPU Bug work-around.

 

 

So, Here's what I did....

 

Booted unraid (watching it boot on primary display)

All VMs were stopped

Ran User Script (edited with my GPU info) image.png.4821f1fd14ecb21be4af41b88819d55a.png

 

(this script disconnects the GPU from Unraid. Puts Unraid to sleep [requiring you to press the power button to quickly turn Unraid /system back on]. The primary monitor turns off or reports "No signal". Once the system is back on, the script has Unraid re-scan for GPUs. The primary monitor remains off/no signal)  Once you press the power button to wake the system, wait for the Script to report "Done" (takes 45 to 60 seconds for me).

 

Start VM linked to Primary GPU = WORKS!  (Note: I pre-installed the GPU drivers and it WOULD NOT work if I linked the "Graphics ROM BIOS". Leave it blank)

 

Alright Community, I need you to help me figure out how to get this to work automatically, on boot up, without having to turn it back on from sleep. Also, this causes an issue when rebooting/shutting down Unraid. Unraid will not successfully reboot or shut down after running the script (on the primary GPU). It does seem to stop the Array correctly whey you select Reboot or Power Off. It just doesn't seem to shut all the way down. Pressing the "Restart" button on the front of the PC successfully restarts Unraid and Unraid reports no issue with the array or parity upon boot. 

 

*UPDATE: I have discovered that this has the unintended consequence of also disconnecting my PCIe TV Tuner from Plex. I have not yet come up with a fix.

 

Again, Shout out to Spaceinvader One! I would be nowhere with out his work.

 

CPU:          1950x 
Mobo:        rog zenith extreme
RAM:         8x4GB ECC
GPU:         GIGABYTE GeForce GT 1030 (in Primary Slot....for now), and ZOTAC GeForce GTX 1050 Ti (in secondary slot, passed thru to VMs [unfortunately covering a PCIe1x slot])
Storage:    500GB m.2 (Cache); 1TB m.2 (Dockers & VMs);  500GB m.2 (Games); 12x HDDs (Storage)

Edited by joshS
  • Upvote 1
Link to comment
3 hours ago, joshS said:

help me figure out how to get this to work automatically, on boot up, without having to turn it back on from sleep.

You'll need another computer to send out a WOL to it. So your script will need to communicate to some process on another PC to send a WOL in x seconds. For example, call a php page on another PC on your LAN that sleeps for a few seconds and then sends out a WOL.

Link to comment
  • 3 weeks later...
On 4/16/2019 at 2:12 PM, jbartlett said:

 

nVidia drivers do not detect KVM and report a Code 43 if Slot 1 has a graphics card dedicated to unraid and the nVidia card is in a different slot passed through to a VM.

 

 

Can you clarify this? The way it's worded makes it sound like I can't use slot one for GPU for unraid and also pass a different card thru to a VM, but it seems that's what you're doing. 

Link to comment
3 hours ago, teh1percent said:

Can you clarify this? The way it's worded makes it sound like I can't use slot one for GPU for unraid and also pass a different card thru to a VM, but it seems that's what you're doing. 

The nVidia drivers can tell if a video card was already in use when it takes control. The drivers assume that it's running in a virtualized environment and refuse to initialize. So to pass a graphics card in a different slot than #1, you have to have a card in the #1 slot for the OS to grab - so it leaves the card you want to pass through alone.

Link to comment
  • 2 weeks later...

FYI - These post came from an 6.8.0 release thread but replying here on this thread as it's more on-topic. I've been having issues getting VM's to run with a video passthrough if I OC the MB by any amount.

 

On 11/9/2019 at 4:53 PM, bastl said:

@jbartlett Maybe you have set some cores which are not directly attached to the memory? Did you manualy changed the topology for the VM?


<topology sockets='1' cores='8' threads='2'/>

I always do this for all my VMs to match the actual core/thread count. Default is always all selected cores and 1 thread.

 

I haven't seen any difference between halving the cores and setting the threads to 2. Based off of Microsoft's "coreinfo" utility, the VM still sees single CPU's, none being hyper-threaded. I have not been successful in finding an XML config that passed the cpu pairing to the OS where the VM saw a hyper-threaded CPU. I'm doing another round of benchmarks with 10 iterations, I'll comment again when done.

 

On 11/10/2019 at 12:59 PM, bastl said:

@jbartlett Just an idea: Switch the slot for your card. Maybe the one you are using is wired via the chipset to the cpu and limits the card. Other devices like usb or network cards often share the same x4 connection to the CPU. Maybe thats your bottleneck

I still wasn't able to OC with the P2000 in the 3rd PCIe slot after moving it from the 2nd PCIe slot. In addition, I wasn't able to run any SeaBIOS VM's at all passing the card in slot #3. The most I'd see is the monitor activate but never even a POST. Graphics scores on the Quadro P2000 were in the same ballpark running in the x8 slot and the x16 slot. The 2nd & 3rd slots attach to different numa nodes but taking that into consideration with CPU pinning made no difference.

 

My current guess is that it's due to having a different GPU (GeForce GT 1030) in the 1st PCIe slot for unraid to bind to. Unfortunately, I don't have two identical GPU's that I could put in it to test that theory.

Link to comment
  • 3 weeks later...

Hello all, 

 

The ROG ZA has 2 NICs.

 

1. 1GBit

2. 10Gbit

 

If I use the 1 Gb Everything starts perfectly and I am assigned an IP address. The moment I plug in a cable to the 10 Gb LAN port. The system panics. 

 

I am not sure what is going on, I have the latest version of UNRAID And the latest BIOS. 

 

Thank you very much

 

cb6abee0-228c-4e15-b7eb-3cc5f95dc60a.jpg

Link to comment
On 12/1/2019 at 3:28 AM, authorleon said:

If I use the 1 Gb Everything starts perfectly and I am assigned an IP address. The moment I plug in a cable to the 10 Gb LAN port. The system panics.

I don't get a panic but it's not working for me currently either. When I plug in the 10G port after it was already booted, I get a solid orange light but no activity. Booted with it plugged in and the port doesn't get any lights at all. Confused by it. When it was working previously, the port would briefly go dead if I jiggled the cable - might be a defective port on my board.

 

The built-in driver seems to be buggy, I get the occasional network drop.

Link to comment

Hello John,

 

I should give you an update what's happened with my install. Unfortunately I had some water damage so I had to go through the insurance company and buy a new motherboard and memory.

 

I have a few questions for you if I may:

 

  1. What version by us are you running. I have installed the latest one. Version 2001
  2. when installing a VM, it is installing incredibly slow. What are your settings regarding the BIOS and any other special settings. Please note I am using only a SSD at the moment. And I would like to get it working first before I try using the MVMe
  3. I am not using a second video card just yet. But when installing Windows 10 it is incredibly slow,  so I'm not quite sure what I'm doing wrong.

Thank you very much.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.