Jump to content
vw-kombi

Cant get New vfio-bind method to work - What am I doing wrong

26 posts in this topic Last Reply

Recommended Posts

I have been working fine for ages on the old method.

Was told to convert to new method in another post due to issues upgrading to 6.8.0.

 

Old config :

append vfio-pci.ids=8086:10c9 pcie_acs_override=downstream,multifunction initrd=/bzroot

 

New config :

append pcie_acs_override=downstream,multifunction initrd=/bzroot

And added vfio-pci.cfg in flash config folder that has :

BIND="09:00.0 09:00.1 0b:00.0 0b:00.1"

 

IOMMU's i need to reserve for pfsense are :

IOMMU group 26:    [8086:10c9] 09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 27:    [8086:10c9] 09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 28:    [8086:10c9] 0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 29:    [8086:10c9] 0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

 

But the system is not reserving these on boot up after a cold restart.

 

Where am I going wrong ?

 

Thanks.

 

Share this post


Link to post

vfio-pci.ids has been working and still works with current version.

The new BIND method was for a different stubbing method.

 

Whatever happened to you, I don't think is related to the stubbing itself since both methods should achieve the same result.

Edited by testdasi

Share this post


Link to post

I tried this method a few days ago as a part of troubleshooting something else and also found that it didn't work at all. vfio-pci-ids works but I thought it was perhaps binding to late.

Share this post


Link to post

I too tried this recently, and it didn't work (6.8.0)

vfio-pci-ids do work, but not in cases where user wants to pass through, say, one of 2 on-board USB controllers which have the same ID. 

The new BIND method would be perfect for this as it uses addresses, rather than IDs. 

Share this post


Link to post
45 minutes ago, meep said:

I too tried this recently, and it didn't work (6.8.0)

vfio-pci-ids do work, but not in cases where user wants to pass through, say, one of 2 on-board USB controllers which have the same ID. 

The new BIND method would be perfect for this as it uses addresses, rather than IDs. 

Perhaps raise a bug report?

Share this post


Link to post
On 1/12/2020 at 5:43 PM, vw-kombi said:

I have been working fine for ages on the old method.

Was told to convert to new method in another post due to issues upgrading to 6.8.0.

 

Old config :

append vfio-pci.ids=8086:10c9 pcie_acs_override=downstream,multifunction initrd=/bzroot

 

New config :

append pcie_acs_override=downstream,multifunction initrd=/bzroot

And added vfio-pci.cfg in flash config folder that has :

BIND="09:00.0 09:00.1 0b:00.0 0b:00.1"

 

IOMMU's i need to reserve for pfsense are :

IOMMU group 26:    [8086:10c9] 09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 27:    [8086:10c9] 09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 28:    [8086:10c9] 0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 29:    [8086:10c9] 0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

 

But the system is not reserving these on boot up after a cold restart.

 

Where am I going wrong ?

 

Thanks.

 

Change 

BIND="09:00.0 09:00.1 0b:00.0 0b:00.1"

to

BIND=09:00.0 09:00.1 0b:00.0 0b:00.1

Better yet, to make sure it's in the right place, just type this in terminal:

echo "BIND=09:00.0 09:00.1 0b:00.0 0b:00.1" >/boot/config/vfio-pci.cfg

 

Edited by Skitals

Share this post


Link to post
2 hours ago, testdasi said:

Perhaps raise a bug report?

I haven't conducted quite enough testing to validly raise a bug. I just tried following the instructions, played about with including or excluding quotes a syyou suggested and found the device always came through to the OS.

 

I'd need to do a bit more structured testing to justify a bug report. May do so net weekend, time permitting.

 

I understand from a post from unRaid  in the 2020 wish list thread that we can expect to see this dealt with in UI this year. That would be amazing.

 

Edited by meep

Share this post


Link to post

I researched this before I did it and parole with more than one were reporting initially seeing only the first address was being reserved and limetech said to put it all in quotes if more than one address needed.

 

maybe that’s fixed now.

 

i will try with no quotes and see.

Share this post


Link to post

OK - if you @meep have already done the testing, I will save a reboot and will wait to see what you can do.

Ta.

Share this post


Link to post

I tried without the quotes.

Does not work.

Must be a bug or incorrect instructions in some way.

Share this post


Link to post
On 1/14/2020 at 8:50 AM, meep said:

I too tried this recently, and it didn't work (6.8.0)

vfio-pci-ids do work, but not in cases where user wants to pass through, say, one of 2 on-board USB controllers which have the same ID. 

The new BIND method would be perfect for this as it uses addresses, rather than IDs. 

Hmmm, I thought the BIND method did use PCI addresses not device ids. That's what I entered anyway, since that was the format that a forum post referenced. Maybe that is why it didn't work for me.

Share this post


Link to post
12 minutes ago, scorcho99 said:

Hmmm, I thought the BIND method did use PCI addresses not device ids. That's what I entered anyway, since that was the format that a forum post referenced. Maybe that is why it didn't work for me.

It is the PCI address in the new method. The post you quoted says the new method uses addresses 😉

 

You can also use PCI addresses with xen-pciback.hide in your syslinux.conf. I haven't really checked if it's still available, but it doesn't hurt to try it.

syntax is as below.

xen-pciback.hide=(08:00.0)(08:00.1)

 

 

Share this post


Link to post
13 minutes ago, saarg said:

It is the PCI address in the new method. The post you quoted says the new method uses addresses 😉

 

You can also use PCI addresses with xen-pciback.hide in your syslinux.conf. I haven't really checked if it's still available, but it doesn't hurt to try it.

syntax is as below.

xen-pciback.hide=(08:00.0)(08:00.1)

 

Ah, I see. I must be blind today.

Edited by scorcho99

Share this post


Link to post

I dont quite understand 'PCI Address'.

 

The Unraid notes state this :

 

 

BIND=<device> <device> ...

Where <device> is a Domain:Bus:Device.Function string, for example,

BIND=02:00.0

Multiple device should be separated with spaces.

 

That is what I was doing.

 

If my iommu is this below :

 

IOMMU group 26:    [8086:10c9] 09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 27:    [8086:10c9] 09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 28:    [8086:10c9] 0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
IOMMU group 29:    [8086:10c9] 0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

 

does this mean I should be doing this instead ?

 

BIND=8086:10c9

 

or should I ignore the notes and use this other method mentioned by @saarg

 

xen-pciback.hide=(09:00.0)(09:00.1)(0b:00.0)(0b:00.1)

Share this post


Link to post

You can also try the new plugin available in Apps which will do this for you

Share this post


Link to post

I dont get it - The plugin makes exactly the same changes that I have tried before in this post which I could not get to work :

 

Capture.JPG.98cb87fe0be143ca6c55eb565ce037e4.JPG

Share this post


Link to post
2 hours ago, vw-kombi said:

I dont get it - The plugin makes exactly the same changes that I have tried before in this post which I could not get to work :

 

Capture.JPG.98cb87fe0be143ca6c55eb565ce037e4.JPG

It does that since that is the way to do it. You use the PCI number.

You are confusing PCI number and the vendor:ID that is used in the syslinux.conf using vfio-pci.ids.

 

You can try the one I suggested, but I don't think it will work as none of the other 2 have. But it doesn't hurt to try it. It's just a reboot away.

 

Post the output of lspci -k and post it here. Both for your current and after doing the xen-pciback.hide.

Share this post


Link to post
4 hours ago, vw-kombi said:

But I tried this exact same line manually as shown in Above posts.

What do your IOMMU groups looks like with and without acs override?

 

On 1/14/2020 at 5:31 AM, testdasi said:

vfio-pci.ids has been working and still works with current version.

The new BIND method was for a different stubbing method.

 

Whatever happened to you, I don't think is related to the stubbing itself since both methods should achieve the same result.

According to the documentation, the vfio-pci.cfg method binds the entire IOMMU group if there are multiple devices in it even if you only list a single address. Does vfio-pci.ids work the same way? If vfio-pci.ids was working but this method does not, my guess is it's a "bug" with pcie_acs_override since you aren't seeing your "natural" IOMMU groups.

Share this post


Link to post
36 minutes ago, Skitals said:

What do your IOMMU groups looks like with and without acs override?

 

According to the documentation, the vfio-pci.cfg method binds the entire IOMMU group if there are multiple devices in it even if you only list a single address. Does vfio-pci.ids work the same way? If vfio-pci.ids was working but this method does not, my guess is it's a "bug" with pcie_acs_override since you aren't seeing your "natural" IOMMU groups.

vfio-pci.ids only binds the specified device.

Share this post


Link to post
5 hours ago, vw-kombi said:

But I tried this exact same line manually as shown in Above posts.

Did you try xen-pciback.hide with the syntax I said?

And please post the output of the command I posted.

Share this post


Link to post

I guess I will try the xen-pciback.hide option.  I dont get often outages to do this stuff.

 

I initially went down this new bind method however as I was told (see below) in another post to use it instead of my existing 'append vfio-pci.ids=8086:10c9 pcie_acs_override=downstream,multifunction initrd=/bzroot' that has been working for ages.  This was due to my unraid upgrade failing to work for my pfsense vm  :

 

 

So is the xen-pciback.hide option and enhanced version of the append vfio-pci.ids=8086:10c9 ?

 

I saw this post from Limetech :

 

'The problem we ran across was that xen-pciback.hide/pciback wasn't working any more, and rather than wait for a kernel dev to fix it, we decided to use the above alternate method.  Note that in general, kernel evolution is moving away from kernel parameters and to more flexible methods.  For example, "isolcpus" is really deprecated and there is alternate method of isolation CPU's using config files which we will adopt in a future Unraid OS release.'

 

Edited by vw-kombi

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.