Alphahelix Posted July 26, 2018 Share Posted July 26, 2018 (edited) I don't know if this issue has been asked before, but I did not seem to find anything on either unraid forum nor google. Here goes: I have 4 NICs in 4 IOMMU groups, BUT they all have the same ID [8086:1521]??? IOMMU group 17: [8086:1521] 06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 06:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 06:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 06:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) How do I reserve 2 for VMs? I normally use the vfio-pci.ids in the syslinux config to reserve resources for VMs. Any help is much appreciated. /Alphahelix *********************************************** The summary of this thread: In order to passthrough PCIe devices that have the same PCIe ID, but are located in different IOMMU groups, Saarg was kind to provide us with a solution. I have tried to make a step by step guide here: Find the device(s) you need passed through to a VM. (Tools -> System Devices) Note the device(s) PCI-number you want to be passed through. (in my case 06:00.2 & 06:00.3) In the syslinux config file between "append" and "initrd=/bzroot" add "xen-pciback.hide=(PCI-number)" (in my case 06:00.2 & 06:00.3) append xen-pciback.hide=(06:00.2)(06:00.3)initrd=/bzroot In the VM you want to use your selected PCIe device(s), edit the XML file and add: (in this example I have added the PCI device with PCI-number: 06:00.2) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/> </source> </hostdev> A restart is needed to activate the changes made in syslinux config file. /Alphahelix Edited July 29, 2018 by Alphahelix 1 Quote Link to comment
1812 Posted July 26, 2018 Share Posted July 26, 2018 you use vfio in syslinux one time (the single id) and it covers them all. then pass through which one you want by address (the 06 number) Quote Link to comment
Alphahelix Posted July 26, 2018 Author Share Posted July 26, 2018 27 minutes ago, 1812 said: you use vfio in syslinux one time (the single id) and it covers them all. then pass through which one you want by address (the 06 number) But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix Quote Link to comment
saarg Posted July 26, 2018 Share Posted July 26, 2018 1 hour ago, Alphahelix said: But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix You use xen-pciback.hide and instead of the ID use the PCI number. The downside is that you have to add the devices to the xml manually. So should look like this: xen-pciback.hide=(06:00.2)(06:00.3) Quote Link to comment
1812 Posted July 26, 2018 Share Posted July 26, 2018 2 hours ago, Alphahelix said: But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix Didn't realize it was your only ones. Try the advice from saarg above. I've never done it, but if he says it should work, then it should work. Quote Link to comment
Alphahelix Posted July 27, 2018 Author Share Posted July 27, 2018 14 hours ago, saarg said: You use xen-pciback.hide and instead of the ID use the PCI number. The downside is that you have to add the devices to the xml manually. So should look like this: xen-pciback.hide=(06:00.2)(06:00.3) First thank you Saarg! So to get it straight, I: * do not use: vfio-pci.ids=8086:1521 i syslinux config file * do use: xen-pciback.hide=(06:00.2)(06:00.3) still in the syslinux config file * in the XML file I add this for each port: (tried my best to google a solution) <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> </source> </hostdev> But where is: domain / bus / slot / function is resolved from? I am no Linux expert, but I am always ready to learn. :) /Alphahelix Quote Link to comment
saarg Posted July 27, 2018 Share Posted July 27, 2018 59 minutes ago, Alphahelix said: First thank you Saarg! So to get it straight, I: * do not use: vfio-pci.ids=8086:1521 i syslinux config file * do use: xen-pciback.hide=(06:00.2)(06:00.3) still in the syslinux config file * in the XML file I add this for each port: (tried my best to google a solution) <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> </source> </hostdev> But where is: domain / bus / slot / function is resolved from? I am no Linux expert, but I am always ready to learn. :) /Alphahelix You are correct about the syslinux.cfg. For the xml, the below is the correct form. Bus is the first two numbers (06), slot is the numbers after : (00) and function is the last number (2). Below is the example for 06:00.2. Add this just above the <memballoon model='virtio'> line. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/> </source> </hostdev> Quote Link to comment
WingsGB Posted August 10, 2018 Share Posted August 10, 2018 I am having the same issue, I have tried the above suggested method but upon reboot after adding xen-pciback.hide=(01:00.0)(01:00.1)(01:00.2)(01:00.3), my network ports are still showing in unraids network settings? the command doesn't seem to do anything. Quote Link to comment
saarg Posted August 11, 2018 Share Posted August 11, 2018 (edited) 17 hours ago, WingsGB said: I am having the same issue, I have tried the above suggested method but upon reboot after adding xen-pciback.hide=(01:00.0)(01:00.1)(01:00.2)(01:00.3), my network ports are still showing in unraids network settings? the command doesn't seem to do anything. Then you have done something wrong. Post your syslinux.cfg here. Edit: Looks like you didn't read the first number correctly. It's 81, not 01. Edited August 11, 2018 by saarg Quote Link to comment
WingsGB Posted August 11, 2018 Share Posted August 11, 2018 (edited) 4 hours ago, saarg said: Then you have done something wrong. Post your syslinux.cfg here. Edit: Looks like you didn't read the first number correctly. It's 81, not 01. IOMMU IOMMU group 17: [8086:1521] 03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 03:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 03:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 03:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 21: [8086:1521] 07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 22: [8086:1521] 07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) syslinux.cfg default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3)initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I have since tried the card in different slots. Edited August 11, 2018 by WingsGB Quote Link to comment
saarg Posted August 11, 2018 Share Posted August 11, 2018 2 hours ago, WingsGB said: IOMMU IOMMU group 17: [8086:1521] 03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 03:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 03:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 03:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 21: [8086:1521] 07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 22: [8086:1521] 07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) syslinux.cfg default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3)initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I have since tried the card in different slots. You need a space after the last bracket. Quote Link to comment
WingsGB Posted August 12, 2018 Share Posted August 12, 2018 Quote Label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3) initrd=/bzroot,/bzroot-gui I have tried with and without a space at the end, it makes no difference. Any other suggestions. Quote Link to comment
saarg Posted August 13, 2018 Share Posted August 13, 2018 9 hours ago, WingsGB said: I have tried with and without a space at the end, it makes no difference. Any other suggestions. If everything is on one line and there is a space after the last bracket, it should not show up in unraids network settings. It will show up in the list of all devices on unraid though. You need to post your diagnostics. Quote Link to comment
WingsGB Posted August 14, 2018 Share Posted August 14, 2018 21 hours ago, saarg said: If everything is on one line and there is a space after the last bracket, it should not show up in unraids network settings. It will show up in the list of all devices on unraid though. You need to post your diagnostics. I have attached my log file and below are some picture. appreciate the help. htpc-diagnostics-20180814-1653.zip Quote Link to comment
saarg Posted August 14, 2018 Share Posted August 14, 2018 Looks like pciback is not grabbing the card. Can you try to add it to the non gui mode and see if it works there? It's working here, but I'm on 6.5.2 still. Quote Link to comment
WingsGB Posted August 14, 2018 Share Posted August 14, 2018 34 minutes ago, saarg said: Looks like pciback is not grabbing the card. Can you try to add it to the non gui mode and see if it works there? It's working here, but I'm on 6.5.2 still. I moved the "pciback" into "Label unRAID OS" and booted up in none gui. Made no change? Quote Link to comment
saarg Posted August 14, 2018 Share Posted August 14, 2018 I tested on 6.5.3 and it works as expected here. I have no idea what is going on on your side, but there is something strange going on. You shouldn't get anything in the other devices list using pciback. Or was that screenshot taken after trying to pass it through to a VM? Quote Link to comment
WingsGB Posted August 15, 2018 Share Posted August 15, 2018 17 hours ago, saarg said: I tested on 6.5.3 and it works as expected here. I have no idea what is going on on your side, but there is something strange going on. You shouldn't get anything in the other devices list using pciback. Or was that screenshot taken after trying to pass it through to a VM? Yes the screenshot was taken after i added the <hostdev> to the xml, i didnt do this untill after editing syslinux and rebooting. I have not tried running the VM yet as the ports are showing in the "network settings" so decided not to go any further Quote Link to comment
saarg Posted August 15, 2018 Share Posted August 15, 2018 2 hours ago, WingsGB said: Yes the screenshot was taken after i added the <hostdev> to the xml, i didnt do this untill after editing syslinux and rebooting. I have not tried running the VM yet as the ports are showing in the "network settings" so decided not to go any further How do you edit the syslinux.cfg? Quote Link to comment
WingsGB Posted August 15, 2018 Share Posted August 15, 2018 3 hours ago, saarg said: How do you edit the syslinux.cfg? I edit it via the webui but after you saying that i thought i would open up the actual file on the flash drive, /boot/syslinux/syslinux.cfg the changes i had made was in the file. Quote Link to comment
saarg Posted August 15, 2018 Share Posted August 15, 2018 36 minutes ago, WingsGB said: I edit it via the webui but after you saying that i thought i would open up the actual file on the flash drive, /boot/syslinux/syslinux.cfg the changes i had made was in the file. I was thinking that there might have been some weird characters in the file. I have seen others had that problem lately. Maybe try to edit the file in a linux file editor (or notepad++ on windows) and write it again. Do not copy/paste it. It looks like the xen-pciback.hide= is not recognized from the boot command by the kernel, as it loads the igb module instead of pciback. Quote Link to comment
WingsGB Posted August 16, 2018 Share Posted August 16, 2018 17 hours ago, saarg said: I was thinking that there might have been some weird characters in the file. I have seen others had that problem lately. Maybe try to edit the file in a linux file editor (or notepad++ on windows) and write it again. Do not copy/paste it. It looks like the xen-pciback.hide= is not recognized from the boot command by the kernel, as it loads the igb module instead of pciback. ? I opened up the file in different editors first to compare, i noticed some margin alignment being out but thats it. To be safe though i copied the default cfg- and started again without copy/ paste. rebooted and same issue? is there anything else i could try disabling with the command to test it? Quote Link to comment
saarg Posted August 16, 2018 Share Posted August 16, 2018 I'm out of ideas, except to clear the network settings and then reboot. There are two files you need to delete on the USB for this, but I don't remember which ones. @bonienl posted this some time ago in some thread, so you could search for it. But I don't think that is the issue. Quote Link to comment
scubieman Posted February 15, 2019 Share Posted February 15, 2019 What was the solution then? 1 Quote Link to comment
s.Oliver Posted March 12, 2019 Share Posted March 12, 2019 On 2/15/2019 at 11:52 PM, scubieman said: What was the solution then? +1 it might help others in the future when doing the same thang! thx. 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.