March 4, 20179 yr I've just had a very frustrating afternoon trying to passthrough a network card to a FreeBSD/pfSense VM. Initially it worked first time with the card showing in two seperate IOMMU groups with the same device ID, so I just added it to my syslinux config file. But, then the VM crashed or something weird and said device unavailable. I eventually spotted that the same dual LAN card was now showing in the same IOMMU group and the device ID had changed...???, even though I hadn't moved it or anything. IOMMU group 42 [8086:10aa] 0d:00.0 Ethernet controller: Intel Corporation Device 10aa (rev 02) [8086:10aa] 0d:00.1 Ethernet controller: Intel Corporation Device 10aa (rev 02) After a few reboots I still can't get it to go back to separate IOMMU groups and I'm stumped on how to passthrough now. I tried following this post: but it didn't work either. help please. PS, for a FreeBSD VM should I be using i440fx-2.7 or Q35-2.7? Both seem to work and the template defaults to Q35, but the help tips say try i440fx highlander-diagnostics-20170304-1827.zip Edited March 16, 20179 yr by DZMM
March 4, 20179 yr Author Hmm, it might be a driver error. I moved the card to another slot to see if that helped: IOMMU group 35 [8086:10aa] 07:00.0 Ethernet controller: Intel Corporation Device 10aa (rev 02) [8086:10aa] 07:00.1 Ethernet controller: Intel Corporation Device 10aa (rev 02) And then added to my XML: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/> </source> </hostdev> I think it worked, but when pfsense boots I see the following: pci2: <network, ethernet> at device 5.0 (no driver attached) pci2: <network, ethernet> at device 6.0 (no driver attached) Which makes me think there's a problem with my drivers???
March 5, 20179 yr Author I don't think that's the problem as it loaded on to start with when the card was in two groups, but then it went to one group and the device ID changed. I also tried passing through to a Windows VM to see what happened, and I got exclamation marks in device manager.
March 5, 20179 yr Author ditching the card and sending back - wasting too much time trying to get working and found a new AOC-SGP-I2 which is recommended by pfSense https://store.pfsense.org/AOC-SGP-I2/ for a decent price. Decided to value my time a bit more and buy decent kit
March 9, 20179 yr Author I've bought another card Supermicro AOC-SG-I2 which also isn't working and I think I need to add the drivers somehow to freebsd/pfSense - can anyone help please? https://www.freebsd.org/cgi/man.cgi?query=igb&apropos=0&sektion=0&manpath=FreeBSD+8.1-RELEASE&arch=default&format=html Quote To compile this driver into the kernel, place the following line in your kernel configuration file: device igb Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5): if_igb_load="YES"
March 10, 20179 yr There should be no need to compile the driver yourself. It's even sold by the pfsense guys, so the drivers are in there. I would start with making a Linux VM to see if it gets recognized there and then move to the pfsense VM when you managed to get them working there. Have you stubbed the card?
March 10, 20179 yr Author Thx saarg - that's what's confusing me. I've actually tried two cards now both although I didn't realise they were actually the same chipset and I've got the newer SGP card coming in the post. The first card worked the first time, with the two adapters showing in separate iommu groups but then crashed and they've been in the same groups ever since. I've stubbed and even passed through to a Windows VM to see what happens there and the adapters show up with exclamation marks in device manager and can't be used. Edited March 10, 20179 yr by DZMM
March 10, 20179 yr Author IOMMU group 39 [8086:0001] 0b:00.0 Ethernet controller: Intel Corporation Device 0001 (rev 02) [8086:0001] 0b:00.1 Ethernet controller: Intel Corporation Device 0001 (rev 02) sysconfig default /syslinux/menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 80 label unRAID OS (stubbed USB,TV,Network) menu default kernel /bzimage append vfio-pci.ids=1b21:1242,14f1:8852,8086:0001 initrd=/bzroot label unRAID OS GUI Mode (stubbed USB,TV,Network) kernel /bzimage append vfio-pci.ids=1b21:1242,14f1:8852,8086:0001 initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest VM XML <domain type='kvm'> <name>Baymax</name> <uuid>c9681910-445a-838e-efd9-a3a147d6dc51</uuid> <metadata> <vmtemplate xmlns="unraid" name="FreeBSD" icon="freebsd.png" os="freebsd"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.7'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='1' threads='2'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/disks/sm961/domains/Baymax/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/disks/sm961/isos/pfSense-CE-2.3.3-RELEASE-amd64.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <model name='i82801b11-bridge'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <model name='pci-bridge'/> <target chassisNr='2'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:36:be:13'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-gb'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x05' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/> </memballoon> </devices> </domain>
March 10, 20179 yr Which motherboard do you have? Have you tried changing the machine type to i440 just to check if that works? Diagnostics would also be nice to have.
March 10, 20179 yr Author X99-A-II. Yep, I tried i440 as well (tried to passthrough to a W10 VM as well as a i440 FreeBSD VM) - no change Updated diagnostics attached for 2nd card - diagnostics for first card in first post highlander-diagnostics-20170310-1154.zip
March 10, 20179 yr What happens if you enable ACS Override? I don't see anything wrong in the logs. Might be worth a try to update to the latest BIOS.
March 11, 20179 yr if you've been using ovmf for the vm, create a new one with seabios and see how that goes.
March 11, 20179 yr Author 6 hours ago, 1812 said: if you've been using ovmf for the vm, create a new one with seabios and see how that goes. Ovmf doesn't work with freebase so I've been using seabios
March 11, 20179 yr It's been a few month since I played around with a pfsense vm, so it appears i've forgotten that part! I had some issues getting cards passed to it too, but later found out I was using an unsupported 10gbe card, but I don't think that is the issue here. Have you tried passing the new (new to you) card to a windows vm? trying to figure out if yours is hardware related, unRaid related, or pfsense related. are you still getting an error with pfsense? if so, what is the exact error?
March 11, 20179 yr 1 minute ago, 1812 said: It's been a few month since I played around with a pfsense vm, so it appears i've forgotten that part! I had some issues getting cards passed to it too, but later found out I was using an unsupported 10gbe card, but I don't think that is the issue here. Have you tried passing the new (new to you) card to a windows vm? trying to figure out if yours is hardware related, unRaid related, or pfsense related. are you still getting an error with pfsense? if so, what is the exact error? In some posts above he mentions that he has problems in a windows VM also. So this might be a hardware problem. He is not on the latest bios, but never answered if he was going to update it.
March 16, 20179 yr Author On 3/11/2017 at 10:51 AM, saarg said: In some posts above he mentions that he has problems in a windows VM also. So this might be a hardware problem. He is not on the latest bios, but never answered if he was going to update it. I am on the latest bios - think you've got me messed up with someone else! Update: installed the AOC-SGP-i2 with the i350 chipset and it worked first time. The giveaway that the AOC SG i2 card with the older 82575EB chipset doesn't work with unraid and/or my motherboard was that it didn't show up settings/network settings as eth1 and eth2, so unRAID never installed it properly
March 16, 20179 yr 1 hour ago, DZMM said: I am on the latest bios - think you've got me messed up with someone else! Mar 9 23:38:51 Highlander kernel: DMI: ASUS All Series/X99-A II, BIOS 1401 11/08/2016 It says in the syslog you are not on the latest BIOS. Latest is 1504. Quote Update: installed the AOC-SGP-i2 with the i350 chipset and it worked first time. The giveaway that the AOC SG i2 card with the older 82575EB chipset doesn't work with unraid and/or my motherboard was that it didn't show up settings/network settings as eth1 and eth2, so unRAID never installed it properly As it's not really working to either pass through the card or work in unraid, it's most likely a hardware problem. When you stub a card, drivers doesn't get loaded for the device so it's available for pass through. Edited March 16, 20179 yr by saarg
March 16, 20179 yr Author 1 hour ago, saarg said: Mar 9 23:38:51 Highlander kernel: DMI: ASUS All Series/X99-A II, BIOS 1401 11/08/2016 It says in the syslog you are not on the latest BIOS. Latest is 1504. As it's not really working to either pass through the card or work in unraid, it's most likely a hardware problem. When you stub a card, drivers doesn't get loaded for the device so it's available for pass through. hmm weird as I was on 1401 - 1504 only came out last Friday which I updated straight away as it fixed my sm961 problems. Re the card not working - I meant it wasn't showing in the network settings for unRAID to use when passthrough wasn't on, whereas the i350 card was which let me know it would work definitely when passed through
Archived
This topic is now archived and is closed to further replies.