sooperdoodie Posted September 3, 2016 Share Posted September 3, 2016 I have a 4 port Intel NIC with 2 ports passed through to pfsense. Is it possible to pass through another one of those ports to another VM? I'm not having any luck. Quote Link to comment
JorgeB Posted September 24, 2016 Share Posted September 24, 2016 Before knowing that they are not optimal for VMs I got a Skylake board and Xeon to upgrade one of my servers, I need to pass trough at least one NIC and I need to enable PCIe ACS override to be able to do it, I tested and it works, my question is: Is this usually stable? With any issues be easily detected? I just don't want to replace the hardware and find out that the system is not stable after a few days or weeks of use. Quote Link to comment
JorgeB Posted September 24, 2016 Share Posted September 24, 2016 Before knowing that they are not optimal for VMs I got a Skylake board and Xeon to upgrade one of my servers, I need to pass trough at least one NIC and I need to enable PCIe ACS override to be able to do it, I tested and it works, my question is: Is this usually stable? With any issues be easily detected? I just don't want to replace the hardware and find out that the system is not stable after a few days or weeks of use. I guess this question doesn't have an easy answer, so I'll try another, is anyone here using a Supermicro X11SSM-F with ACS override? Any issues? Quote Link to comment
white13wolf Posted October 18, 2016 Share Posted October 18, 2016 Hey there i need a bit of help. I'm a noob so have a bit of patience please. I have a unraid box that has 2 nic's on the motherboard and a pci express x4 card with 4 1gb ports. I want to install pfsense to be a firewall and put the ISP PPPoE connection in one NIC that is NOT available to unraid ( want the internet to go in pfsense and then be routed in my home switch and give internet to all devices (pc, xbox, tv, unraid etc.) and not have my unraid exposed) can you help ? PfSense is not absolutely necessary (can use different os) i'm just familiar with this one. Thank you Quote Link to comment
SpaceInvaderOne Posted December 23, 2016 Share Posted December 23, 2016 Hi Guys i have put together a video showing how to isolate then passthrough a Network Controller whilst checking that it can be reset. Hope you find it useful How to pass through a network controller in unRAID 1 Quote Link to comment
manorfan Posted February 17, 2017 Share Posted February 17, 2017 (edited) Hi, Thanks so much for the video!! Edited February 21, 2017 by manorfan Quote Link to comment
DZMM Posted March 4, 2017 Share Posted March 4, 2017 I'm struggling to passthrough my cards to a pfSense VM: At first the cards were shown in separate groups, now they are in the same group and the device ID changed (??). I'm not sure if this is the problem though as when pfSense boots it says: pci2: <network, ethernet> at device 5.0 (no driver attached) pci2: <network, ethernet> at device 6.0 (no driver attached) any ideas? Thanks Quote Link to comment
johner Posted March 29, 2017 Share Posted March 29, 2017 So after fighting and losing to using two bridges with pfsense (Physical machines could access the Internet but VMs on br0 couldn't - appeared to be an issue with inbound packets to the VMs), I passed through two of my Intel NICs (same vendor IDs). I was fortunate that they were in different IOMMU groups, and all I did was add the hostdev rows to the pfsense VM XML. No syslinux.cfg stubs and no ACS override. Really simple in the end. Unraid no longer sees the cards in the network settings. Did something change in 6.3.2 to make this so easy? Should I expect the world to end as I'm missing something? My Windows VM can now use the Internet, I can RDP to it from the internet (although this did work from the LAN even though internet didn't). I'm curious why twin bridges didn't work, I might search harder for an answer one day out of curiosity. Thanks all, couldn't have done it without this thread. ESXi night night! John Quote Link to comment
saarg Posted March 30, 2017 Share Posted March 30, 2017 6 hours ago, johner said: So after fighting and losing to using two bridges with pfsense (Physical machines could access the Internet but VMs on br0 couldn't - appeared to be an issue with inbound packets to the VMs), I passed through two of my Intel NICs (same vendor IDs). I was fortunate that they were in different IOMMU groups, and all I did was add the hostdev rows to the pfsense VM XML. No syslinux.cfg stubs and no ACS override. Really simple in the end. Unraid no longer sees the cards in the network settings. Did something change in 6.3.2 to make this so easy? Should I expect the world to end as I'm missing something? My Windows VM can now use the Internet, I can RDP to it from the internet (although this did work from the LAN even though internet didn't). I'm curious why twin bridges didn't work, I might search harder for an answer one day out of curiosity. Thanks all, couldn't have done it without this thread. ESXi night night! John I would suggest you stub them. The reason is that then the drivers aren't loaded in unraid and less chances of something suddenly not working. The other reason is that you can choose the cards in the other PCI device list in the vm template. When adding stuff to the XML, it gets wiped if you do any edits in the vm template. Quote Link to comment
hahaa24 Posted June 3, 2017 Share Posted June 3, 2017 On 1.5.2015 at 5:36 PM, jonp said: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/> </source> </hostdev> Modify the address line entering in the two digit bus, slot, and function from your ID. So 00:19.0 translates to what I have above. My question: How can I do this for four NIC´s? I have a quad Port NIC and want to use it in pfSense. Quote Link to comment
jonp Posted June 6, 2017 Author Share Posted June 6, 2017 Quote My question: How can I do this for four NIC´s? I have a quad Port NIC and want to use it in pfSense. The pass through method you quoted me on is for passing through an entire controller, not just an individual interface. For passing through an interface, see this in the libvirt documentation. Quote Link to comment
Boomer42 Posted August 16, 2017 Share Posted August 16, 2017 (edited) I installed a TP-Link TG-3468 (Realtek) network card into my system, it sees it but its grouped with my motherboard Ethernet (Intel) IOMMU group 14. My ACS Override is ON. and I tried swapping the card to another PCIe Slot. Any other way to force it to another seperate IOMMU group? EDIT: Solution: Have ACS patch ON, in Main>Flash>Syslinux configuration: I changed pcie_acs_override=downstream to pcie_acs_override=multifunction This separated the IOMMU groups even more. Ill leave this here for future People Quote IOMMU group 0 [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 1 [1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 2 [1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 3 [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 4 [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 5 [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 6 [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 7 [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 8 [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 9 [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 10 [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 11 [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 12 [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467 IOMMU group 13 [1987:5007] 01:00.0 Non-Volatile memory controller: Device 1987:5007 (rev 01) IOMMU group 14 [1022:43b9] 03:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02) [1022:43b5] 03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02) [1022:43b0] 03:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02) [1022:43b4] 1d:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 1d:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 1d:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 1d:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 1d:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 1d:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [10ec:8168] 24:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) [1b21:1343] 25:00.0 USB controller: ASMedia Technology Inc. Device 1343 [8086:1539] 26:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 15 [10de:1187] 28:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 760] (rev a1) [10de:0e0a] 28:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) IOMMU group 16 [1022:145a] 29:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a [1022:1456] 29:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456 [1022:145c] 29:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c IOMMU group 17 [1022:1455] 2a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 [1022:7901] 2a:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) [1022:1457] 2a:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457 Edited August 16, 2017 by Boomer42 Quote Link to comment
Azyx Posted September 27, 2017 Share Posted September 27, 2017 On 13/04/2016 at 11:35 AM, saarg said: You should try this from the unraid wiki The extra line added is supposed to be added. It is the address inside the specific VM. So nothing to worry about. <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/> OMG! Thank you so much Saarg, I've been trying the whole day to passthrough my dual interface Intel NIC. Tried with and without the "pci-stub" and "vfio-pci" in the syslinux config. Also tried to pass just one of the interfaces.... Tried so many things until i stumbled upon your post with the link for the "vfio_iommu_type1.allow_unsafe_interrupts=1" command. This should be added to the OP's post, I think. 1 Quote Link to comment
JWMutant Posted September 28, 2017 Share Posted September 28, 2017 On 19/10/2016 at 6:32 AM, white13wolf said: Hey there i need a bit of help. I'm a noob so have a bit of patience please. I have a unraid box that has 2 nic's on the motherboard and a pci express x4 card with 4 1gb ports. I want to install pfsense to be a firewall and put the ISP PPPoE connection in one NIC that is NOT available to unraid ( want the internet to go in pfsense and then be routed in my home switch and give internet to all devices (pc, xbox, tv, unraid etc.) and not have my unraid exposed) can you help ? PfSense is not absolutely necessary (can use different os) i'm just familiar with this one. Thank you If you still need help with this let me know as I have spent some time getting the perfect pfsense vm running on unRAID with a fallback if unRAID craps itself for some reason. 1 Quote Link to comment
JorgeB Posted November 13, 2017 Share Posted November 13, 2017 On 25/09/2016 at 12:58 AM, johnnie.black said: On 24/09/2016 at 5:27 PM, johnnie.black said: Before knowing that they are not optimal for VMs I got a Skylake board and Xeon to upgrade one of my servers, I need to pass trough at least one NIC and I need to enable PCIe ACS override to be able to do it, I tested and it works, my question is: Is this usually stable? With any issues be easily detected? I just don't want to replace the hardware and find out that the system is not stable after a few days or weeks of use. I guess this question doesn't have an easy answer, so I'll try another, is anyone here using a Supermicro X11SSM-F with ACS override? Any issues? Update on this, as it may be useful info for other users, without ACS override the X11SSM-F groups together the two top PCIe slots, so I've been using ACS override for more than year because I wanted to pass-trough an addon NIC on the 2nd slot, server has been pretty much 100% stable, until recently when I added a SATA controller to the other PCIe slot, since then my VMs started crashing regularly, in hindsight, mostly when both the NIC and SATA controller were being used at the same time, so I disabled ACS override, swapped the NIC to a now free bottom slot that is on its own group and no more issues, so ACS override may appear to work correctly but not be completely stable in some cases. Quote Link to comment
marceloliv3 Posted December 18, 2017 Share Posted December 18, 2017 On 28/09/2017 at 6:17 AM, JWMutant said: If you still need help with this let me know as I have spent some time getting the perfect pfsense vm running on unRAID with a fallback if unRAID craps itself for some reason. I dont know about him, but id love to know how you did that. A setup like that would suit perfectly for my use case Quote Link to comment
JWMutant Posted December 18, 2017 Share Posted December 18, 2017 I dont know about him, but id love to know how you did that. A setup like that would suit perfectly for my use caseI’ll write up a quick guide when I finish work.Sent from my iPhone using Tapatalk 2 Quote Link to comment
L0rdRaiden Posted February 16, 2018 Share Posted February 16, 2018 On 18/12/2017 at 9:05 PM, JWMutant said: I’ll write up a quick guide when I finish work. Sent from my iPhone using Tapatalk Have you prepared that guide? It's published somewhere? The only thing stopping me from getting unraid is that I am not sure I will be able to setup pfsense with all the network setup right so all my traffic goes through pfsense first and then unraid or the rest of my network. I have a hyperv config right now and the Linux word is a bit unkown for me. Quote Link to comment
DZMM Posted February 17, 2018 Share Posted February 17, 2018 4 hours ago, L0rdRaiden said: Have you prepared that guide? It's published somewhere? The only thing stopping me from getting unraid is that I am not sure I will be able to setup pfsense with all the network setup right so all my traffic goes through pfsense first and then unraid or the rest of my network. I have a hyperv config right now and the Linux word is a bit unkown for me. setting up a pfsense VM and passing through NICs is very easy. I've just done a screenshot of the VM setup page for a new pfsense VM. After start, you just setup pfsense as normal and treat it like a normal machine Quote Link to comment
L0rdRaiden Posted February 17, 2018 Share Posted February 17, 2018 10 hours ago, DZMM said: setting up a pfsense VM and passing through NICs is very easy. I've just done a screenshot of the VM setup page for a new pfsense VM. After start, you just setup pfsense as normal and treat it like a normal machine Thanks With the network bridge you are giving exclusive access to pfsense to one of the nic s? How do you configure, and where the virtual switches? Is there an user interface or you have to go CLI? Quote Link to comment
Osiris Posted March 9, 2018 Share Posted March 9, 2018 (edited) Is the video on NIC isolation still viable in Unraid 6.4.1 ? Also, I've got 2 NIC with the same 'ids' IOMMU group 11: [8086:1533] 05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) IOMMU group 12: [8086:1533] 06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) different groups though. I fear to isolate them both if I do this. Edited March 9, 2018 by Osiris Quote Link to comment
saarg Posted March 9, 2018 Share Posted March 9, 2018 3 hours ago, Osiris said: Is the video on NIC isolation still viable in Unraid 6.4.1 ? Also, I've got 2 NIC with the same 'ids' IOMMU group 11: [8086:1533] 05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) IOMMU group 12: [8086:1533] 06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) different groups though. I fear to isolate them both if I do this. Use the one below instead. Change it to your id's of course. xen-pciback.hide=(04:00.3) Quote Link to comment
Osiris Posted March 10, 2018 Share Posted March 10, 2018 (edited) 12 hours ago, saarg said: Use the one below instead. Change it to your id's of course. xen-pciback.hide=(04:00.3) Just to clarify I want to isolate only the second NIC of those two with the same id. IOMMU group 11: [8086:1533] 05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) IOMMU group 12: [8086:1533] 06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) as lspci gives me 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers (rev 07) 00:13.0 Non-VGA unclassified device: Intel Corporation Sunrise Point-H Integrated Sensor Hub (rev 31) 00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31) 00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H Thermal subsystem (rev 31) 00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #1 (rev 31) 00:16.1 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #2 (rev 31) 00:17.0 SATA controller: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] (rev 31) 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #1 (rev f1) 00:1c.1 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #2 (rev f1) 00:1c.4 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #5 (rev f1) 00:1c.6 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #7 (rev f1) 00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller (rev 31) 00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31) 00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31) 01:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) 02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) 03:00.0 Non-Volatile memory controller: Device 1987:5007 (rev 01) 04:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 03) 05:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) should I use xen-pciback.hide=(02:00.0) ? Secondly, I was wondering, isn't there a way to offer isolation via the network settings in the GUI? Something that's only configurable when there's more than one Ethernet controller in the lspci or system devices output? Edited March 10, 2018 by Osiris Quote Link to comment
saarg Posted March 10, 2018 Share Posted March 10, 2018 A little bit confused here about what you want to do. You want to pass through one of the NICs to a VM or separate the network using vlans? This thread is about passing stuff through to a VM. So if you want network separation, you should move along to another thread (search Google). If you want to pass through only one of the NICs, then play along here Either you have changed PCI slot for your NICs or it wasn't from you the first time as the PCI ID changed. Your xen-pciback.hide looks good. Quote Link to comment
Osiris Posted March 12, 2018 Share Posted March 12, 2018 (edited) I want to pass it through to a VM. I've reviewed my own post (don't laugh) and don't really see where I might have been confusing about this. I spoke about network card isolation as to be possibly offered via the unraid GUI (in order to pass it through to a VM). I was just asking if the tutorial video was indeed not really applicable to my situation, as both my NICs are part of the same 'id', namely 8086:1533 So thanks for confirming that. Edited March 12, 2018 by Osiris Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.