Armed Ferret Posted May 12, 2020 Posted May 12, 2020 I am trying to pass my 4 port GB NIC to pass through to my pfSense VM. IOMMU shows enabled on Unraid but I do not have a IOMMU setting in my bios. My NIC is in IOMMU Group 15 with this ID 8086:10bc [1022:43d5] 02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller (rev 01) [1022:43c8] 02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller (rev 01) [1022:43c6] 02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge (rev 01) [1022:43c7] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) [111d:8018] 06:00.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [111d:8018] 07:02.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [111d:8018] 07:04.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [8086:10bc] 08:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 08:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 09:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 09:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) I tried adding this under flash vfio-pci.ids=8086:10bc And the NIC still wont show under other PCI devices I changed it to the following: vfio-pci.ids=8086:10bc,1022:43d5,1022:43c8,8086:1539 When I do this I cannot ping my unraid server nor access the web gui remotely. I tried enabling ACS override and even VFIO allow unsafe interrupts I cannot get it to work and hoping someone could help. Quote
T0a Posted May 12, 2020 Posted May 12, 2020 (edited) What Unraid version are you using? There is a new vfio-bind method in Unraid 6.7 and upwards. Also check out the VFIO-PCI Config Plugin in this context. Please use the code tags for configuration snippets next. Edited May 12, 2020 by T0a Quote
Armed Ferret Posted May 12, 2020 Author Posted May 12, 2020 I do not seem to have the Vfio-pci.cfg file on my flash drive Is there a way to get it there. Im on version 6.8.3 Quote
meep Posted May 12, 2020 Posted May 12, 2020 5 minutes ago, Armed Ferret said: I do not seem to have the Vfio-pci.cfg file on my flash drive Is there a way to get it there. Im on version 6.8.3 The easiest way is to install the VFIO-Config plugin from Community Applications, mentioned by @T0a, above This provides a visual listing of devices, allows you click checkboxes to add them an generates the .cfg file for you. Quote
Armed Ferret Posted May 12, 2020 Author Posted May 12, 2020 So I added the plugin. Then I checked the devices I wanted to use for passthrough, rebooted and now I cannot access the Unraid Web GUI. I tried turning on ACS override to break up the IOMMU group and still cannot access the Web GUI or Ping the server from my PC. I unchecked the devices from the plug in and turned off ACS Override and rebooted again. Still cant ping or access. Went into my flash drive and deleted the config file. Still cant ping or access. What do I need to do next? Quote
T0a Posted May 12, 2020 Posted May 12, 2020 (edited) 21 minutes ago, Armed Ferret said: So I added the plugin. Then I checked the devices I wanted to use for passthrough, rebooted and now I cannot access the Unraid Web GUI. I tried turning on ACS override to break up the IOMMU group and still cannot access the Web GUI or Ping the server from my PC. I unchecked the devices from the plug in and turned off ACS Override and rebooted again. Still cant ping or access. Went into my flash drive and deleted the config file. Still cant ping or access. What do I need to do next? Which device did you exactly checked (8086:1539 or 8086:10bc)? Note, when ticking the wrong one, Unraid has no access to this device such as the eth controller, which could be the cause for your ping issues. The "ACS Override" option shouldn't be required as the device you want to pass through resides in its own IOMMU group, right? In case you haven't changed something else, you should be fine with only removing the pcie-vfio config file from the flash drive as you did. Can you attach a monitor to your server? Then pls look for any printed issues. Also try to boot in "save mode" (with disabled plugins and a virgin sys config file). Does this work? I'm so thankful that limetech put their OS on a flash drive with easy access in case of such a problem. Edited May 12, 2020 by T0a Quote
Armed Ferret Posted May 12, 2020 Author Posted May 12, 2020 (edited) I figured it out. But to answer your question, I specifically checked the 4 port nic 8086:10bc. Somehow it seems my interface rules got changed when doing the vfio config. I saw one of the 4 port nics was now assigned to eth0 and my on board mobo nic was assigned to eth2. I switched them back and rebooted and bam...I have internet. So let me try this tool again. And no...The device I am passing through does not reside in its own IOMMU group. So I will need to enable ACS override, correct? IOMMU group 15: [1022:43d5] 02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller (rev 01) [1022:43c8] 02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller (rev 01) [1022:43c6] 02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge (rev 01) [1022:43c7] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) [111d:8018] 06:00.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [111d:8018] 07:02.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [111d:8018] 07:04.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e) [8086:10bc] 08:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 08:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 09:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 09:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) 8086:10bc is my 4 port nic im trying to passthrough Should I enable ACS override then? Edited May 12, 2020 by Armed Ferret more info Quote
Armed Ferret Posted May 12, 2020 Author Posted May 12, 2020 (edited) Ok.. I have them all in their IOMMU Groups and enabled ACS override to separate them into their IOMMU groups However, only two of my 4 ports are showing. I looked in the vfio-cfg file and I see all 4 ids added BIND=08:00.0 08:00.1 09:00.0 09:00.1 And here is my IOMMU groups IOMMU group 25: [8086:10bc] 08:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 26: [8086:10bc] 08:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 27: [8086:10bc] 09:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 28: [8086:10bc] 09:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06) In my VM the two that are showing: In my network settings these two are not being passed: Is there a reason I cannot get all 4 to pass through? Edited May 12, 2020 by Armed Ferret screenshots Quote
Armed Ferret Posted May 12, 2020 Author Posted May 12, 2020 I do see that my interface eth1 shows also that I can assign an IP to it under network settings but eth2 does not. SO maybe its something in the unraid holding onto this for some reason? Quote
Armed Ferret Posted May 16, 2020 Author Posted May 16, 2020 On 5/13/2020 at 4:47 AM, T0a said: Does maybe this thread help: Symptoms are the same So it helps some. However, to me, its not as clear. This is what is in my XML for the two ports that are currently being passed <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </hostdev> When I look up the other two I want to pass here are their info: 08:00.0 0200: 8086:10bc (rev 06) 08:00.1 0200: 8086:10bc (rev 06) 09:00.0 0200: 8086:10bc (rev 06) 09:00.1 0200: 8086:10bc (rev 06) Now I get I should be changing the bus to 0x09 for the other two. But I am not sure what else I need to change? Im guessing I would need to set one to function 0x01. But on the second source like there is bus 0x03 and 0x04. Do I need to change those? Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.