bilbo6209 Posted May 25, 2015 Share Posted May 25, 2015 I followed the directions on this post http://lime-technology.com/forum/index.php?topic=39499.0 reply # 10... When I entered "lspci" I got the following I then run "lspci -n" and I get this I edited my syslinux.cfg to the following rebooted created a new VM edited the VM XML file and added the following and now when I attempt to start the VM I get the following error I'm not sure what I did wrong. Bill Quote Link to comment
CHBMB Posted May 25, 2015 Share Posted May 25, 2015 Have you enabled Vt-d virtualisation in your BIOS? What motherboard and CPU do you have and have you checked you can do pass through mate? Quote Link to comment
bilbo6209 Posted May 25, 2015 Author Share Posted May 25, 2015 All specs are listed in my signature, I have another vm running that has my GPU passed through so I'm sure everything is set up correctly... But to answer you questions The mobo is am Asus m5a97 v2.0 Cpu amd fx8350 24gb ram SVM (secure virtual machine) enabled in the bios, this is either Asus's term for vt-d or another term for AMDs implementation. And gpu virtual pass through (sorry can't remember the exact acronym right now) is enabled. Quote Link to comment
CHBMB Posted May 26, 2015 Share Posted May 26, 2015 All specs are listed in my signature, I have another vm running that has my GPU passed through so I'm sure everything is set up correctly... But to answer you questions The mobo is am Asus m5a97 v2.0 Cpu amd fx8350 24gb ram SVM (secure virtual machine) enabled in the bios, this is either Asus's term for vt-d or another term for AMDs implementation. And gpu virtual pass through (sorry can't remember the exact acronym right now) is enabled. Ok, so if you're passing though a Gfx it must be passthrough compatible. Can't see the syslinux.cfg pic, not showing up here, but the rest looks OK. I'm assuming that the line starting <address type='pci' was added automatically for you? Post your syslinux.cfg for us The only other thing I can think of to try assuming you haven't made a typo in the syslinux.cfg is turning on PCIe ACS Override, may or may not work and is recommended to leave it off, but I had it on for a fair while and didn't cause me any trouble. YMMV. Quote Link to comment
jonp Posted May 26, 2015 Share Posted May 26, 2015 All specs are listed in my signature, I have another vm running that has my GPU passed through so I'm sure everything is set up correctly... But to answer you questions The mobo is am Asus m5a97 v2.0 Cpu amd fx8350 24gb ram SVM (secure virtual machine) enabled in the bios, this is either Asus's term for vt-d or another term for AMDs implementation. And gpu virtual pass through (sorry can't remember the exact acronym right now) is enabled. Ok, so if you're passing though a Gfx it must be passthrough compatible. Can't see the syslinux.cfg pic, not showing up here, but the rest looks OK. I'm assuming that the line starting <address type='pci' was added automatically for you? Post your syslinux.cfg for us The only other thing I can think of to try assuming you haven't made a typo in the syslinux.cfg is turning on PCIe ACS Override, may or may not work and is recommended to leave it off, but I had it on for a fair while and didn't cause me any trouble. YMMV. CHBMB has it right. You need this setting on because your secondary NIC shares a root ACS port with another device on the system. There is a note in the unRAID 6 Wiki Manual (see link in my signature) about this and a link to a full article about the risks this flag includes and that while it could be fine, there still could be issues. Quote Link to comment
bilbo6209 Posted May 26, 2015 Author Share Posted May 26, 2015 CHBMB thank you for the reply! Im going to try to address your comments/questions in order. 1. Yes <Address type='pci' was auto addedd to the XML 2. I corrected the typo that was preventing the syslinux.cfg from showing, so it is showing now.... but I also noticed one thing, I had added the info "pci-stup.ids=10ec:8169" under the SafeMode section so I changed it to teh UnRaid OS section PCIe ACS OverRide was flagged "yes" I changed it to NO I created a new VM and here is the XML and here is the error I am receiving, please note if I dont add in the lines highlighted in the above picture the VM boots perfectly. I just tried changing the PCIe ACS OverRide setting to YES (rebooted etc) and tried to launch the VM again and got the same error again. So the PCIe ACS Override isn't it. Quote Link to comment
jonp Posted May 26, 2015 Share Posted May 26, 2015 CHBMB thank you for the reply! Im going to try to address your comments/questions in order. 1. Yes <Address type='pci' was auto addedd to the XML 2. I corrected the typo that was preventing the syslinux.cfg from showing, so it is showing now.... but I also noticed one thing, I had added the info "pci-stup.ids=10ec:8169" under the SafeMode section so I changed it to teh UnRaid OS section PCIe ACS OverRide was flagged "yes" I changed it to NO I created a new VM and here is the XML and here is the error I am receiving, please note if I dont add in the lines highlighted in the above picture the VM boots perfectly. I just tried changing the PCIe ACS OverRide setting to YES (rebooted etc) and tried to launch the VM again and got the same error again. So the PCIe ACS Override isn't it. Are both of these NICs "on-board" meaning did they both ship with the motherboard itself and are not installed into a discrete PCI slot? Quote Link to comment
jonp Posted May 26, 2015 Share Posted May 26, 2015 CHBMB thank you for the reply! Im going to try to address your comments/questions in order. 1. Yes <Address type='pci' was auto addedd to the XML 2. I corrected the typo that was preventing the syslinux.cfg from showing, so it is showing now.... but I also noticed one thing, I had added the info "pci-stup.ids=10ec:8169" under the SafeMode section so I changed it to teh UnRaid OS section PCIe ACS OverRide was flagged "yes" I changed it to NO I created a new VM and here is the XML and here is the error I am receiving, please note if I dont add in the lines highlighted in the above picture the VM boots perfectly. I just tried changing the PCIe ACS OverRide setting to YES (rebooted etc) and tried to launch the VM again and got the same error again. So the PCIe ACS Override isn't it. Are both of these NICs "on-board" meaning did they both ship with the motherboard itself and are not installed into a discrete PCI slot? You could try changing the driver from vfio to kvm in the XML to see if that works. It's the legacy method of assigning PCI devices. Quote Link to comment
CHBMB Posted May 26, 2015 Share Posted May 26, 2015 I haven't really got anything else I can offer on this I'm afraid, other than my rudimentary suggestions to begin with. Quote Link to comment
bilbo6209 Posted May 26, 2015 Author Share Posted May 26, 2015 You could try changing the driver from vfio to kvm in the XML to see if that works. It's the legacy method of assigning PCI devices. I changed line 104 in the XML from 'vfio' to 'kvm' and I am no longer getting the same error message on launch. and we are golden!!! Thank you to all for your help and suggestions! Quote Link to comment
bilbo6209 Posted May 26, 2015 Author Share Posted May 26, 2015 hmmm Did this actually fix anything? I am building Widows now, and when I started loading drivers I had to point to the RedHat Linux drivers disk for a driver for the NIC... I would have thought it would have shown up as a PCI REALTec NIC in a PCI Slot... Or did we tell the KVM front end to tell Windows it was the Linux KVM RedHat NIC, but the backend is actually using the 2nd hardware NIC? I am a 100% Linux Novice so if these are stupid questions Im sorry. Quote Link to comment
jonp Posted May 27, 2015 Share Posted May 27, 2015 hmmm Did this actually fix anything? I am building Widows now, and when I started loading drivers I had to point to the RedHat Linux drivers disk for a driver for the NIC... I would have thought it would have shown up as a PCI REALTec NIC in a PCI Slot... Or did we tell the KVM front end to tell Windows it was the Linux KVM RedHat NIC, but the backend is actually using the 2nd hardware NIC? I am a 100% Linux Novice so if these are stupid questions Im sorry. Open device manager in windows. Expand the networking section. How many network devices do you see? Quote Link to comment
bilbo6209 Posted May 30, 2015 Author Share Posted May 30, 2015 Open device manager in windows. Expand the networking section. How many network devices do you see? Sorry for the delay in responding, I didn't see you had replied. In device manager there is only RedHat VirtIO ethernet adaptor. I wasn't sure if this was correct because of changing the driver in the xml to KVM, but I had expected to see the hardware Nic passed through. When I built the vm there were 3 unknown devices, the red hat Nic, the memory balloon driver, and the red hat serial port driver. All unknown devices in device manager are accounted for. Quote Link to comment
jonp Posted May 30, 2015 Share Posted May 30, 2015 Open device manager in windows. Expand the networking section. How many network devices do you see? Sorry for the delay in responding, I didn't see you had replied. In device manager there is only RedHat VirtIO ethernet adaptor. I wasn't sure if this was correct because of changing the driver in the xml to KVM, but I had expected to see the hardware Nic passed through. When I built the vm there were 3 unknown devices, the red hat Nic, the memory balloon driver, and the red hat serial port driver. All unknown devices in device manager are accounted for. Can you share your XML for the VM? Quote Link to comment
bilbo6209 Posted May 31, 2015 Author Share Posted May 31, 2015 Yep here you go <domain type='kvm' id='19'> <name>Harry</name> <uuid>8de9a698-557c-7c89-7458-ac66e9310b1d</uuid> <description>The Seeker</description> <metadata> <vmtemplate name="Custom" icon="windows7.png" os="windows7"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>3</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='3'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='3' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/cache/harry/vdisk1.img'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Installs/OS/Windows 7 Professional X64 Dell OEM/Windows 7 Professional X64 Dell OEM.iso'/> <backingStore/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <alias name='ide0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Installs/OS/unraid drivers/virtio-win-0.1.96.iso'/> <backingStore/> <target dev='hdb' bus='ide'/> <readonly/> <alias name='ide0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <alias name='usb0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:37:73:ce'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target port='0'/> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Harry.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='vmvga' vram='16384' heads='1'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x413c'/> <product id='0x2003'/> <address bus='8' device='3'/> </source> <alias name='hostdev0'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> Quote Link to comment
jonp Posted May 31, 2015 Share Posted May 31, 2015 I don't see anywhere in the xml that you are passing through a NIC. Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 Wow sorry, I have rebuilt VMs on this box several times trying to resolve this issue (and to get the GPU Pass trough working ) and some how I missed adding the NIC to this XML I added the NIC now <domain type='kvm'> <name>Harry</name> <uuid>8de9a698-557c-7c89-7458-ac66e9310b1d</uuid> <description>The Seeker</description> <metadata> <vmtemplate name="Custom" icon="windows7.png" os="windows7"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>3</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='3'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='3' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/cache/harry/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Installs/OS/Windows 7 Professional X64 Dell OEM/Windows 7 Professional X64 Dell OEM.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Installs/OS/unraid drivers/virtio-win-0.1.96.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:37:73:ce'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Harry.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='vmvga' vram='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x413c'/> <product id='0x2003'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x05' slot='0x06' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> and I am now getting the error again I changed the line <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> to <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='kvm'/> and I am now getting this error I have seen a similar error before when my RAM wasn't a multiple of 4gb, but this VM was running minutes ago, I stopped it, modified the XML, and now it will not start.... so it kind of would surprise me if there was a config issue other than attempting to pass through the NIC. Bill Quote Link to comment
jonp Posted June 1, 2015 Share Posted June 1, 2015 Ok, first, did you follow the steps in this guide? Specifically did you add the pci-stub entry in your syslinux configuration file to hide the NIC from unRAID OS at boot? Second, try this: edit your XML again, for the NIC pass through, try this: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x05' slot='0x06' function='0x0'/> </source> </hostdev> Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 I believe all of that is set correctly, please verify the images in the first post. The code you supplied is exactly what I entered in the first xml, the system appears to add in the "address type " line... This gives the first error I supplied. Yes I followed your guide, I even included the link to it in the opening post Bill Quote Link to comment
jonp Posted June 1, 2015 Share Posted June 1, 2015 I believe all of that is set correctly, please verify the images in the first post. The code you supplied is exactly what I entered in the first xml, the system appears to add in the "address type " line... This gives the first error I supplied. Yes I followed your guide, I even included the link to it in the opening post Bill Ok, you may need to reboot the system, edit the XML again, remove the second <address> line from the <hostdev> for the pass through, and change the driver from vfio to kvm. I believe this could be because once the system has been started with vfio, it doesn't want to hand it back to kvm for management independently. Not sure, but a reboot / retest will confirm. Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 Grrrr that's kinda what I was afraid you were going to say.... That is now a long process due to the other bug I have in my server, I will loose access to the shares and will need to drop the server from my AD domain, reboot the server, then rejoin the server, reset perms on the shares, reboot the clients... Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 Hmmm weird, after rebooting the lines I added for the NIC are gone from the xml. Did I edit the syslinux file correctly? Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 Ok with leaving the driver type as vfio and rebooting the info stays in the xml, but the vm throws the same error, and the address line was added. I modified the driver again to KVM, and deleted the address line and rebooted again. Let's see what happens Quote Link to comment
bilbo6209 Posted June 1, 2015 Author Share Posted June 1, 2015 after rebooting several times, the XML showed the expected settings (driver of kvm), but the VM would not boot, with the same error shown above (Not a multiple of 4k). I changed the number of processors from 3 to 4 and attempted to boot the system again, and it booted (required a reboot), on reboot there was still only 1 NIC listed in device manager. I looked at the XML again and the lines pertaining to the NIC were missing again. So for whatever reason it doesn't seem like the settings are sticking in the xml. Quote Link to comment
jonp Posted June 1, 2015 Share Posted June 1, 2015 after rebooting several times, the XML showed the expected settings (driver of kvm), but the VM would not boot, with the same error shown above (Not a multiple of 4k). I changed the number of processors from 3 to 4 and attempted to boot the system again, and it booted (required a reboot), on reboot there was still only 1 NIC listed in device manager. I looked at the XML again and the lines pertaining to the NIC were missing again. So for whatever reason it doesn't seem like the settings are sticking in the xml. Hmm, well, the only other thing you could try is physically moving the NIC to another PCI slot and see if that changes anything for you. NIC assignment is something we are still classifying as experimental as we don't understand fully all the types of NIC devices/scenarios that will work with assignment just yet. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.