Running 6.12.13. I have 2 two port serial port cards I want to passthrough to VMs, however one of the cards has apparently been bound by unraid and is greyed out in the Tools|System Devices|PCI Devices and IOMMU Groups.

 

Mousing over the device in question shows the text "in use by unraid".

 

Haven't interacted in the forums for ages and don't want to be rude asking for help, happy to provide any information needed, not sure what to provide.

You are likely to get better informed feedback if you attach your system’s diagnostics zip file to your next post in this thread. It is always a good idea when asking questions to supply your diagnostics so we can see details of your system, how you have things configured, and the current syslog.

Here is my system diagnostics zip file.

leo-diagnostics-20240907-1241.zip

On 9/7/2024 at 6:42 PM, Jimbo7136 said:

Here is my system diagnostics zip file.

leo-diagnostics-20240907-1241.zip 96.79 kB · 2 downloads

Try echo "0000:41.00.0" > /sys/bus/pci/drivers/serial/unbind or echo "0000:05.00.0" > /sys/bus/pci/drivers/serial/unbind

 

to see if that releases the cards so you can add to vfio.

 

or you may be able to add a manual entry into vfio.

 

nano /boot/config/vfio-pci.cfg
BIND=0000:05:00.0|1c00:3253

 

or

BIND=0000:05:00.0|1c00:3253 0000:41:00.01c00:3253 if you want both

echo "0000:41.00.0" > /sys/bus/pci/drivers/serial/unbind

response: bash: echo: write error: No such device

echo "0000:05.00.0" > /sys/bus/pci/drivers/serial/unbind

response: bash: echo: write error: No such device

Just now, Jimbo7136 said:

echo "0000:41.00.0" > /sys/bus/pci/drivers/serial/unbind

response: bash: echo: write error: No such device

echo "0000:05.00.0" > /sys/bus/pci/drivers/serial/unbind

response: bash: echo: write error: No such device

Sorry typo on my part 0000:41:00.0

root@Leo:~# echo "0000:41:00.0" > /sys/bus/pci/drivers/serial/unbind
bash: echo: write error: No such device

2 minutes ago, Jimbo7136 said:

root@Leo:~# echo "0000:41:00.0" > /sys/bus/pci/drivers/serial/unbind
bash: echo: write error: No such device

what does lspci -D output?

root@Leo:~# lspci -D
0000:00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex
0000:00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit
0000:00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0000:00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0000:00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)
0000:00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
0000:00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
0000:00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
0000:00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
0000:00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
0000:00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
0000:00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
0000:00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6
0000:00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
0000:00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
0000:00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
0000:00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
0000:00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
0000:00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
0000:00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
0000:00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6
0000:00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
0000:01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02)
0000:01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02)
0000:01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02)
0000:02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:04:00.0 Network controller: Intel Corporation Wireless 8265 / 8275 (rev 78)
0000:05:00.0 Serial controller: Nanjing Qinheng Microelectronics Co., Ltd. CH352/CH382 PCI/PCIe Dual Port Serial Adapter (rev 10)
0000:06:00.0 Non-Volatile memory controller: Sandisk Corp WD PC SN810 / Black SN850 NVMe SSD (rev 01)
0000:07:00.0 Ethernet controller: Intel Corporation Ethernet Controller X550 (rev 01)
0000:07:00.1 Ethernet controller: Intel Corporation Ethernet Controller X550 (rev 01)
0000:09:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
0000:09:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
0000:09:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
0000:09:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
0000:0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
0000:0b:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor (PSP) 3.0 Device
0000:0b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller
0000:0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function
0000:0c:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
0000:40:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex
0000:40:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit
0000:40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
0000:40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0000:40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0000:41:00.0 Serial controller: Nanjing Qinheng Microelectronics Co., Ltd. CH352/CH382 PCI/PCIe Dual Port Serial Adapter (rev 10)
0000:42:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 710] (rev a1)
0000:42:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)
0000:43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
0000:43:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor (PSP) 3.0 Device
0000:43:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller
0000:44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function
0000:44:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

echo "0000:05:00.0" > /sys/bus/pci/drivers/serial/unbind seemed to execute successfully, I rebooted after and still can't reserve the serial port

2 minutes ago, Jimbo7136 said:

echo "0000:05:00.0" > /sys/bus/pci/drivers/serial/unbind seemed to execute successfully, I rebooted after and still can't reserve the serial port

Dont reboot, but in sysdevs where you able to bind to vfio? Reboot will rebind if not bound to vfio. Maybe be a problem because kernel is binding.

echo "0000:05:00.0" > /sys/bus/pci/drivers/serial/unbind issued again, no reboot.

Checked system devices. Unable to reserve 0000:05:00.0

 

1 minute ago, Jimbo7136 said:

echo "0000:05:00.0" > /sys/bus/pci/drivers/serial/unbind issued again, no reboot.

Checked system devices. Unable to reserve 0000:05:00.0

 

lspci -Dks 0000:05:00.0

root@Leo:~# lspci -Dks 0000:05:00.0
0000:05:00.0 Serial controller: Nanjing Qinheng Microelectronics Co., Ltd. CH352/CH382 PCI/PCIe Dual Port Serial Adapter (rev 10)
        Subsystem: Nanjing Qinheng Microelectronics Co., Ltd. CH352/CH382 PCI/PCIe Dual Port Serial Adapter
root@Leo:~# 

Changes to configuration since original post.

 

Here is current diagnostics zip

leo-diagnostics-20240909-1333.zip

11 minutes ago, Jimbo7136 said:

Changes to configuration since original post.

 

Here is current diagnostics zip

leo-diagnostics-20240909-1333.zip 131.89 kB · 0 downloads

Looks like 41 is bound to vfio. does that show in VM manager?

 

Binding...
Unbound 0000:41:00.0 from serial

success...

Device 1c00:3253 at 0000:41:00.0 bound to vfio-pci

 

 

Two VMs in VM Manager.

Neither is bound to 0000:41:00.0

 

0000:41:00.0 is available to bind to either VM.

Is that what you meant?

2 minutes ago, Jimbo7136 said:

0000:41:00.0 is available to bind to either VM.

Is that what you meant?

Yes

4 minutes ago, Jimbo7136 said:

0000:41:00.0 is available to bind to either VM.

Is that what you meant?

OK the other card is not in its own iommu group.

 

/sys/kernel/iommu_groups/15/devices/0000:01:00.0
/sys/kernel/iommu_groups/15/devices/0000:01:00.1
/sys/kernel/iommu_groups/15/devices/0000:01:00.2
/sys/kernel/iommu_groups/15/devices/0000:02:00.0
/sys/kernel/iommu_groups/15/devices/0000:02:02.0
/sys/kernel/iommu_groups/15/devices/0000:02:04.0
/sys/kernel/iommu_groups/15/devices/0000:04:00.0
/sys/kernel/iommu_groups/15/devices/0000:05:00.0

 

You could try the acs override options in VM settings to see if that will add into its own group.

I don't know what you mean by "acs override options" in VM settings. Could you elaborate, please?

3 minutes ago, Jimbo7136 said:

I don't know what you mean by "acs override options" in VM settings. Could you elaborate, please?

Confirmed.

Setting PCIe ACS override to "both" and rebooting, the 0000:05:00.0 is available to bind in Tools|System Devices|PCI Devices and IOMMU Groups.

Thanks for your help.

 

The device is bindable to a VM now.