July 26, 20187 yr I don't know if this issue has been asked before, but I did not seem to find anything on either unraid forum nor google. Here goes: I have 4 NICs in 4 IOMMU groups, BUT they all have the same ID [8086:1521]??? IOMMU group 17: [8086:1521] 06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 06:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 06:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 06:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) How do I reserve 2 for VMs? I normally use the vfio-pci.ids in the syslinux config to reserve resources for VMs. Any help is much appreciated. /Alphahelix *********************************************** The summary of this thread: In order to passthrough PCIe devices that have the same PCIe ID, but are located in different IOMMU groups, Saarg was kind to provide us with a solution. I have tried to make a step by step guide here: Find the device(s) you need passed through to a VM. (Tools -> System Devices) Note the device(s) PCI-number you want to be passed through. (in my case 06:00.2 & 06:00.3) In the syslinux config file between "append" and "initrd=/bzroot" add "xen-pciback.hide=(PCI-number)" (in my case 06:00.2 & 06:00.3) append xen-pciback.hide=(06:00.2)(06:00.3)initrd=/bzroot In the VM you want to use your selected PCIe device(s), edit the XML file and add: (in this example I have added the PCI device with PCI-number: 06:00.2) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/> </source> </hostdev> A restart is needed to activate the changes made in syslinux config file. /Alphahelix Edited July 29, 20187 yr by Alphahelix
July 26, 20187 yr you use vfio in syslinux one time (the single id) and it covers them all. then pass through which one you want by address (the 06 number)
July 26, 20187 yr Author 27 minutes ago, 1812 said: you use vfio in syslinux one time (the single id) and it covers them all. then pass through which one you want by address (the 06 number) But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix
July 26, 20187 yr 1 hour ago, Alphahelix said: But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix You use xen-pciback.hide and instead of the ID use the PCI number. The downside is that you have to add the devices to the xml manually. So should look like this: xen-pciback.hide=(06:00.2)(06:00.3)
July 26, 20187 yr 2 hours ago, Alphahelix said: But if i pass all through, what will unraid use? (They are onboard) My goal is to pass 2 through to VMs and 2 for unraid. Sorry I should have been more clear. My bad. /Alphahelix Didn't realize it was your only ones. Try the advice from saarg above. I've never done it, but if he says it should work, then it should work.
July 27, 20187 yr Author 14 hours ago, saarg said: You use xen-pciback.hide and instead of the ID use the PCI number. The downside is that you have to add the devices to the xml manually. So should look like this: xen-pciback.hide=(06:00.2)(06:00.3) First thank you Saarg! So to get it straight, I: * do not use: vfio-pci.ids=8086:1521 i syslinux config file * do use: xen-pciback.hide=(06:00.2)(06:00.3) still in the syslinux config file * in the XML file I add this for each port: (tried my best to google a solution) <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> </source> </hostdev> But where is: domain / bus / slot / function is resolved from? I am no Linux expert, but I am always ready to learn. :) /Alphahelix
July 27, 20187 yr 59 minutes ago, Alphahelix said: First thank you Saarg! So to get it straight, I: * do not use: vfio-pci.ids=8086:1521 i syslinux config file * do use: xen-pciback.hide=(06:00.2)(06:00.3) still in the syslinux config file * in the XML file I add this for each port: (tried my best to google a solution) <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x00' slot='0x00' function='0x0'/> </source> </hostdev> But where is: domain / bus / slot / function is resolved from? I am no Linux expert, but I am always ready to learn. :) /Alphahelix You are correct about the syslinux.cfg. For the xml, the below is the correct form. Bus is the first two numbers (06), slot is the numbers after : (00) and function is the last number (2). Below is the example for 06:00.2. Add this just above the <memballoon model='virtio'> line. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/> </source> </hostdev>
August 10, 20187 yr I am having the same issue, I have tried the above suggested method but upon reboot after adding xen-pciback.hide=(01:00.0)(01:00.1)(01:00.2)(01:00.3), my network ports are still showing in unraids network settings? the command doesn't seem to do anything.
August 11, 20187 yr 17 hours ago, WingsGB said: I am having the same issue, I have tried the above suggested method but upon reboot after adding xen-pciback.hide=(01:00.0)(01:00.1)(01:00.2)(01:00.3), my network ports are still showing in unraids network settings? the command doesn't seem to do anything. Then you have done something wrong. Post your syslinux.cfg here. Edit: Looks like you didn't read the first number correctly. It's 81, not 01. Edited August 11, 20187 yr by saarg
August 11, 20187 yr 4 hours ago, saarg said: Then you have done something wrong. Post your syslinux.cfg here. Edit: Looks like you didn't read the first number correctly. It's 81, not 01. IOMMU IOMMU group 17: [8086:1521] 03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 03:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 03:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 03:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 21: [8086:1521] 07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 22: [8086:1521] 07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) syslinux.cfg default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3)initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I have since tried the card in different slots. Edited August 11, 20187 yr by WingsGB
August 11, 20187 yr 2 hours ago, WingsGB said: IOMMU IOMMU group 17: [8086:1521] 03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:1521] 03:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 03:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 03:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 21: [8086:1521] 07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 22: [8086:1521] 07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) syslinux.cfg default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3)initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I have since tried the card in different slots. You need a space after the last bracket.
August 12, 20187 yr Quote Label unRAID OS GUI Mode menu default kernel /bzimage append xen-pciback.hide=(03:00.0)(03:00.1)(03:00.2)(03:00.3) initrd=/bzroot,/bzroot-gui I have tried with and without a space at the end, it makes no difference. Any other suggestions.
August 13, 20187 yr 9 hours ago, WingsGB said: I have tried with and without a space at the end, it makes no difference. Any other suggestions. If everything is on one line and there is a space after the last bracket, it should not show up in unraids network settings. It will show up in the list of all devices on unraid though. You need to post your diagnostics.
August 14, 20187 yr 21 hours ago, saarg said: If everything is on one line and there is a space after the last bracket, it should not show up in unraids network settings. It will show up in the list of all devices on unraid though. You need to post your diagnostics. I have attached my log file and below are some picture. appreciate the help. htpc-diagnostics-20180814-1653.zip
August 14, 20187 yr Looks like pciback is not grabbing the card. Can you try to add it to the non gui mode and see if it works there? It's working here, but I'm on 6.5.2 still.
August 14, 20187 yr 34 minutes ago, saarg said: Looks like pciback is not grabbing the card. Can you try to add it to the non gui mode and see if it works there? It's working here, but I'm on 6.5.2 still. I moved the "pciback" into "Label unRAID OS" and booted up in none gui. Made no change?
August 14, 20187 yr I tested on 6.5.3 and it works as expected here. I have no idea what is going on on your side, but there is something strange going on. You shouldn't get anything in the other devices list using pciback. Or was that screenshot taken after trying to pass it through to a VM?
August 15, 20187 yr 17 hours ago, saarg said: I tested on 6.5.3 and it works as expected here. I have no idea what is going on on your side, but there is something strange going on. You shouldn't get anything in the other devices list using pciback. Or was that screenshot taken after trying to pass it through to a VM? Yes the screenshot was taken after i added the <hostdev> to the xml, i didnt do this untill after editing syslinux and rebooting. I have not tried running the VM yet as the ports are showing in the "network settings" so decided not to go any further
August 15, 20187 yr 2 hours ago, WingsGB said: Yes the screenshot was taken after i added the <hostdev> to the xml, i didnt do this untill after editing syslinux and rebooting. I have not tried running the VM yet as the ports are showing in the "network settings" so decided not to go any further How do you edit the syslinux.cfg?
August 15, 20187 yr 3 hours ago, saarg said: How do you edit the syslinux.cfg? I edit it via the webui but after you saying that i thought i would open up the actual file on the flash drive, /boot/syslinux/syslinux.cfg the changes i had made was in the file.
August 15, 20187 yr 36 minutes ago, WingsGB said: I edit it via the webui but after you saying that i thought i would open up the actual file on the flash drive, /boot/syslinux/syslinux.cfg the changes i had made was in the file. I was thinking that there might have been some weird characters in the file. I have seen others had that problem lately. Maybe try to edit the file in a linux file editor (or notepad++ on windows) and write it again. Do not copy/paste it. It looks like the xen-pciback.hide= is not recognized from the boot command by the kernel, as it loads the igb module instead of pciback.
August 16, 20187 yr 17 hours ago, saarg said: I was thinking that there might have been some weird characters in the file. I have seen others had that problem lately. Maybe try to edit the file in a linux file editor (or notepad++ on windows) and write it again. Do not copy/paste it. It looks like the xen-pciback.hide= is not recognized from the boot command by the kernel, as it loads the igb module instead of pciback. ? I opened up the file in different editors first to compare, i noticed some margin alignment being out but thats it. To be safe though i copied the default cfg- and started again without copy/ paste. rebooted and same issue? is there anything else i could try disabling with the command to test it?
August 16, 20187 yr I'm out of ideas, except to clear the network settings and then reboot. There are two files you need to delete on the USB for this, but I don't remember which ones. @bonienl posted this some time ago in some thread, so you could search for it. But I don't think that is the issue.
March 12, 20197 yr On 2/15/2019 at 11:52 PM, scubieman said: What was the solution then? +1 it might help others in the future when doing the same thang! thx.
Archived
This topic is now archived and is closed to further replies.