February 3, 20179 yr Today I finally had the chance to swap out the motherboard and processor that I've been using with Unraid. This gives me the option to run VM's with hardware passthrough which I wasn't able to do before. My plan is to consolidate my Windows 10 pc and my Unraid server. First I converted the Windows 10 install to an image following the wiki. I started by using Seabios, but that had the "Booting from Hard Disk" error so I switched to OVMF. That worked, except that I couldn't get it to passthrough my video card. This may be because of IOMMU so I enabled the PCIe Override. I still wasn't getting any output from the video card. So next I created a Windows 10 VM from scratch and used Seabios. The video card does work with this one. However, the performance is pretty awful. Everything is moving very slowly. It feels very laggy. I'm not sure what to do from here. I guess I have 2 questions, what can I do to improve the performance and is there some way I can get the Windows 10 install that's currently using OVMF to work with the video card passthrough? Not sure what all info is needed so if I miss something lt me know. (It should be noted that I do plan to upgrade the processor to one with Hyperthreading and I also plan to upgrade the video card as well. however, that won't be for awhile yet and I wanted to do some testing with my current hardware first.) Model: Custom M/B: Gigabyte Technology Co., Ltd. - Z170X-Gaming 7 CPU: Intel® Core™ i5-6500 CPU @ 3.20GHz HVM: Enabled IOMMU: Enabled Cache: 256 kB, 1024 kB, 6144 kB Memory: 16 GB (max. installable capacity 64 GB) Network: eth0: 1000 Mb/s, full duplex, mtu 1500 eth1: not connected eth2: not connected eth3: not connected Video card is a Nvidia GeForce GTX 660 PCIe Devices: 00:00.0 Host bridge [0600]: Intel Corporation Skylake Host Bridge/DRAM Registers [8086:191f] (rev 07) 00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16) [8086:1901] (rev 07) 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:1912] (rev 06) 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31) 00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31) 00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] [8086:a102] (rev 31) 00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Root Port #17 [8086:a167] (rev f1) 00:1b.2 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Root Port #19 [8086:a169] (rev f1) 00:1b.3 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Root Port #20 [8086:a16a] (rev f1) 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #1 [8086:a110] (rev f1) 00:1c.1 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #2 [8086:a111] (rev f1) 00:1c.2 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #3 [8086:a112] (rev f1) 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #5 [8086:a114] (rev f1) 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #9 [8086:a118] (rev f1) 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC Controller [8086:a145] (rev 31) 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H PMC [8086:a121] (rev 31) 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD Audio [8086:a170] (rev 31) 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus [8086:a123] (rev 31) 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8] (rev 31) 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 [GeForce GTX 660] [10de:11c0] (rev a1) 01:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio Controller [10de:0e0b] (rev a1) 03:00.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2304] (rev 05) 04:01.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2304] (rev 05) 04:02.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2304] (rev 05) 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c) 06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c) 07:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02) 0a:00.0 Ethernet controller [0200]: Qualcomm Atheros Killer E2400 Gigabit Ethernet Controller [1969:e0a1] (rev 10) 0b:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578] 0c:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578] 0c:01.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578] 0c:02.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578] 0c:04.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578] 0f:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 3.1 Controller [Alpine Ridge] [8086:15b6] IOMMU Groups: /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:14.0 /sys/kernel/iommu_groups/4/devices/0000:00:16.0 /sys/kernel/iommu_groups/5/devices/0000:00:17.0 /sys/kernel/iommu_groups/6/devices/0000:00:1b.0 /sys/kernel/iommu_groups/6/devices/0000:00:1b.2 /sys/kernel/iommu_groups/6/devices/0000:00:1b.3 /sys/kernel/iommu_groups/6/devices/0000:03:00.0 /sys/kernel/iommu_groups/6/devices/0000:04:01.0 /sys/kernel/iommu_groups/6/devices/0000:04:02.0 /sys/kernel/iommu_groups/6/devices/0000:05:00.0 /sys/kernel/iommu_groups/6/devices/0000:06:00.0 /sys/kernel/iommu_groups/6/devices/0000:07:00.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.1 /sys/kernel/iommu_groups/7/devices/0000:00:1c.2 /sys/kernel/iommu_groups/7/devices/0000:00:1c.4 /sys/kernel/iommu_groups/7/devices/0000:0a:00.0 /sys/kernel/iommu_groups/7/devices/0000:0b:00.0 /sys/kernel/iommu_groups/7/devices/0000:0c:00.0 /sys/kernel/iommu_groups/7/devices/0000:0c:01.0 /sys/kernel/iommu_groups/7/devices/0000:0c:02.0 /sys/kernel/iommu_groups/7/devices/0000:0c:04.0 /sys/kernel/iommu_groups/7/devices/0000:0f:00.0 /sys/kernel/iommu_groups/8/devices/0000:00:1d.0 /sys/kernel/iommu_groups/9/devices/0000:00:1f.0 /sys/kernel/iommu_groups/9/devices/0000:00:1f.2 /sys/kernel/iommu_groups/9/devices/0000:00:1f.3 /sys/kernel/iommu_groups/9/devices/0000:00:1f.4 /sys/kernel/iommu_groups/10/devices/0000:00:1f.6 XML from the fresh Windows 10 install: <domain type='kvm' id='5'> <name>Windows 10 Test</name> <uuid>ad76896b-af8d-17f4-9007-b07b6d5d573c</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>3</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='3'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='3' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/appdata/Windows 10 Test/vdisk1.img'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Data/ISO/Windows 10/Windows.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <alias name='sata0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Data/ISO/virtio-win-0.1.130.iso'/> <backingStore/> <target dev='hdb' bus='sata'/> <readonly/> <alias name='sata0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:cd:65:97'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target port='0'/> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-Windows 10 Test/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc313'/> <address bus='1' device='9'/> </source> <alias name='hostdev2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x04d9'/> <product id='0xfa50'/> <address bus='1' device='7'/> </source> <alias name='hostdev3'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1130'/> <product id='0x1620'/> <address bus='1' device='5'/> </source> <alias name='hostdev4'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </memballoon> </devices> </domain>
February 3, 20179 yr is the disk image on a cache drive/pool? in the array? What version of unRaid are you on? a little weird that it is in the app data folder (/mnt/user/appdata/Windows 10 Test/vdisk1.img) but that alone shouldn't cause the issue I think.
February 3, 20179 yr Author is the disk image on a cache drive/pool? in the array? What version of unRaid are you on? a little weird that it is in the app data folder (/mnt/user/appdata/Windows 10 Test/vdisk1.img) but that alone shouldn't cause the issue I think. Heh... that's the default location and I apparently forgot to change it to the appropriate location. But it's on my SSD cache drive, no cache pool. Unraid version is 6.2.4
February 3, 20179 yr Author I switched USB Mode to XHCI and that helped. It is no longer painfully slow to browse the web. There is still noticeable lag, though. (I also moved the vm to the correct /mnt/cache/domains/Windows 10 Test/vdisk1.img location, but of course that didn't affect anything.)
February 3, 20179 yr my next suggestion was to try and rule out if it was a slow usb response issue or slow vm/video issue, but it appears you've done that. what driver is the video card using? are you running anything in docker/plugin? what does your cpu utilization look like with the vm running?
February 3, 20179 yr Author my next suggestion was to try and rule out if it was a slow usb response issue or slow vm/video issue, but it appears you've done that. what driver is the video card using? are you running anything in docker/plugin? what does your cpu utilization look like with the vm running? When I first installed this VM the video card was using the Nvidia driver that Windows 10 installed. That had a lot of issues with not being able to start the VM everytime and crashing and some weird visual corruption. Managed to get the most current driver from Nvidia, 378.49. No more crashing or or anything. Plex, Emby, TVheadend, DelugeVPN for dockers. Then some of the basic plugins like CA, UD, Preclear, Nerd Tools, User Scripts, Dynamix Local Master, Dynamix Cache Directories (installed, but not setup), Dynamix System Temp (which disappeared on me at one point yesterday and I had to reinstall it). CPU utilization is currently sitting around 15-25% on the 3 cores assigned to the VM. About 5-9% on the leftover core. There were issues yesterday where 1 assigned core would peg 100%, but I can't recall whether that was on both VM's or just one of them. It does not seem to be happening now. Sometimes I try to move the mouse and it doesn't want to move right. Extremely slow and not steady and not in the exact direction I try to move it. At the same time, trying to open a program will take forever, even just opening the start menu.
February 3, 20179 yr try changing your cpus to 1,3 only. Many find that keeping windows on paired thread on a core helps with lag, and the addition of having 1 thread from one core (cpu 2) with 2 on another may be causing issues (I'm not 100% convinced of this, but windows isn't my main working environment and haven't done extensive experimentation with it.) And at this point it's worth a shot. Also keep an eye on your core utilization while booting/using the vm. If you haven't isolated cores from unraid/dockers for the vm exclusively to ue, it may be that they are bogging it down. But if utilization doesn't max out while trying to do normal things (web, etc...) then that probably isn't the issue but it' not helping the situation either (it also wouldn't hurt to consider letting the vm have it's own core(s).)
February 3, 20179 yr Author I wasn't sure if there were thread pairs without hyperthreading. I'll try that. What do you mean about isolating the cores? Is there something to do besides selecting which cores to use in the VM settings? I've only had 1, 2, and 3 selected leaving 0 for Unraid.
February 3, 20179 yr I wasn't sure if there were thread pairs without hyperthreading. I'll try that. What do you mean about isolating the cores? Is there something to do besides selecting which cores to use in the VM settings? I've only had 1, 2, and 3 selected leaving 0 for Unraid. wait, you're right, just looked up the specs on your cpu, no hyperthreading. if you don't isolate cores in the syslinux.cfg file, then unRaid and the docker still have access to all 4 cores, regardless of what you do in the vm manager.
February 3, 20179 yr Author if you don't isolate cores in the syslinux.cfg file, then unRaid and the docker still have access to all 4 cores, regardless of what you do in the vm manager. Ok, where do I find out how to do that? I must not be searching for the right thing.
February 3, 20179 yr Search for isolate cpu in the forums My use case does not dictate to isolate cpu cores from unRaid. What I do do however is limit the access that certain docker apps have to certain cores. You can find that info in the docker FAQ
February 3, 20179 yr Author Thanks Squid. Ok so here is what I did. Is this correct? I have the VM set to use just cores 2 and 3, so those are the 2 cores I wanted to isolate. label unRAID OS menu default kernel /bzimage append pcie_acs_override=downstream isolcpus=2,3 initrd=/bzroot Because I have been booting into the GUI mode, do I have to add that to the unRAID OS GUI mode as well?
February 3, 20179 yr Thanks Squid. Ok so here is what I did. Is this correct? I have the VM set to use just cores 2 and 3, so those are the 2 cores I wanted to isolate. label unRAID OS menu default kernel /bzimage append pcie_acs_override=downstream isolcpus=2,3 initrd=/bzroot Because I have been booting into the GUI mode, do I have to add that to the unRAID OS GUI mode as well? Yes.
February 3, 20179 yr Author Was that yes to both? Initially I did it just like in my previous post and rebooted. Before starting the VM, there was still some activity on cores 2 and 3. Started up the VM and it seemed much better and more like it should be. Assuming that saarg was saying yes to both, I added isolcpus bit to unRAID OS GUI mode as well. Rebooted and cores 2 and 3 were at 0 before the VM was started. So if it did indeed need to be added to both, that would mean that the problem was something else and for some reason rebooting the first time fixed it. I don't know what the could be because I didn't make any other changes. (There was an error about irq16 in the logs that seems to be gone after the first reboot. Something about no one cared?) I am so confused. I guess I'll use this for a little bit and see if it stays like this.
February 4, 20179 yr If you're booing into GUI mode, then yes, the isolcpus should be under both as you have put it. Maybe placebo effect for why it seemed "better" the first reboot. But either way, better is better!
February 4, 20179 yr Author It was a big enough difference that I doubt it was a placebo. I'll do some testing tomorrow just to double check and I'll report back. Thank you for the help.
February 4, 20179 yr Author I took out the isolcpus lines and rebooted and the VM to see what would happen and the VM is still ok. Programs open up like they should and the mouse is behaving normally. If it's working now, leave out the isolcpus lines? I guess if this starts happening again I should post the full diagnostics.
February 4, 20179 yr I took out the isolcpus lines and rebooted and the VM to see what would happen and the VM is still ok. Programs open up like they should and the mouse is behaving normally. If it's working now, leave out the isolcpus lines? I guess if this starts happening again I should post the full diagnostics. It's up to you if you want to share the resources of the core or not. You don't have much available with only 4 cores....
Archived
This topic is now archived and is closed to further replies.