Help Passing Through Capture Card

[s57o2]

Hello,

 

I recently build an unRAID server with the intention of also using it to run a VM to serve as a secondary PC for streaming.  I was successfully able to pass through a Black Magic Intensity Pro 4k, however the card itself was performing unsatisfactorily, so I returned it in lieu of a Elgato HD60 pro, however the same techniques are proving to be ineffective for passing through this other capture card.

 

01:00.0 Non-VGA unclassified device: Yuan Yuan Enterprise Co., Ltd. Device 0380 (rev ff)

 

Is the associated line from a lspci call, and with the -n flag:

 

01:00.0 0000: 12ab:0380 (rev ff)

 

I have add the pci-stub.ids=12ab:0380 line to the syslinux and added the following to the VM's XML:

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

 

When attempting to start the VM, the following error(s) appear:

 

internal error: early end of file from monitor: possible problem:
2016-02-17T05:36:22.229030Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev3,bus=pci.0,addr=0x7: vfio: Error: Failed to setup INTx fd: Device or resource busy
2016-02-17T05:36:22.229498Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev3,bus=pci.0,addr=0x7: Device initialization failed
2016-02-17T05:36:22.229516Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev3,bus=pci.0,addr=0x7: Device 'vfio-pci' could not be initialized

 

I have attempted with and without ACS override and cannot think of anything else to try and was wondering if anyone had some suggestions?

[iphillips77]

Does it give you that error if you start the VM from a cold boot?  And I mean totally cold, physically turn the server off then back on again.

 

I was trying for a while to pass through a USB card that didn't support PCI reset..  and it would work the very first time, but any subsequent times it would give me an error that the card couldn't be initialized.  Only way to get it to work again was a cold boot, a reboot wouldn't do the trick.

 

Nothing you can do if that's the case.

[s57o2]

No, it does not give me that error from cold boot, however the device also does not appear to be functional in the first run of the VM.

[tinimun]

Currently in the same situation with vfio-pci.

 

lspci -vnn output:

 

07:00.0 Non-VGA unclassified device [0000]: Yuan Yuan Enterprise Co., Ltd. Device [12ab:0380]
        Subsystem: Device [1cfa:0003]
        Flags: bus master, fast devsel, latency 0, IRQ 18, NUMA node 0
        Memory at f6000000 (32-bit, non-prefetchable) [size=32M]
        I/O ports at 0000
        Memory at f8000000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Kernel driver in use: vfio-pci

[bungee91]

I don't know your hardware, but you can try to add iommu=pt in your syslinux.cfg file, and also vfio_iommu_type1.allow_unsafe_interrupts=1 (I don't think the order matters).

 

It would look like this:

kernel /bzimage
append iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

 

Try one, or both, and see if it helps.

[tinimun]

I'm using an Asus X99-A motherboard with a Core i7-5930K. Isn't

iommu=pt

usually for AMD hardware? In any case, I tried that (with unsafe interrupts enabled), and don't think unsafe interrupts are the issue. The device is instantly being taken over by vfio-pci at boot, and issues seem to arise when a PCI reset occurs.

 

Here's the -vvv output for lspci on that device, at a fresh boot of the hypervisor:

 

07:00.0 Non-VGA unclassified device: Yuan Yuan Enterprise Co., Ltd. Device 0380
Subsystem: Device 1cfa:0003
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 18
NUMA node: 0
Region 0: Memory at f6000000 (32-bit, non-prefetchable) [size=32M]
Region 4: I/O ports at 0000
Region 5: Memory at f8000000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 3
	Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
	Status: D3 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
	Address: 0000000000000000  Data: 0000
Capabilities: [70] Express (v2) Endpoint, MSI 00
	DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
		ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
	DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
		RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
		MaxPayload 128 bytes, MaxReadReq 512 bytes
	DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
	LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
		ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
	LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
		ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
	LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
	DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
	LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
		 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
		 Compliance De-emphasis: -6dB
	LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
		 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
	UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
	UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
	UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
	CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
	CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
	AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Virtual Channel
	Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
	Arb:	Fixed- WRR32- WRR64- WRR128-
	Ctrl:	ArbSelect=Fixed
	Status:	InProgress-
	VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
		Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
		Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
		Status:	NegoPending- InProgress-
Kernel driver in use: vfio-pci

 

 

In this state, /sys/kernel/iommu_groups/34/devices/0000:07:00.0/config seems to contain useful data when I look at it with hexdump.

 

And, after booting the guest OS:

07:00.0 Non-VGA unclassified device: Yuan Yuan Enterprise Co., Ltd. Device 0380 (rev ff) (prog-if ff)
!!! Unknown header type 7f
Kernel driver in use: vfio-pci

 

 

A hexdump of the config in this state contains all ff.

 

If I run:

echo 1 > /sys/kernel/iommu_groups/34/devices/0000:07:00.0/remove

... the device is removed, but:

echo 1 > /sys/bus/pci/rescan

doesn't make it come back.

 

Maybe this device is broken when PCI resets happen?

Edited April 15, 2017 by tinimun

[tinimun]

I will try rebuilding the kernel with quirk_no_pm_reset set for this device and report back.

Edited April 15, 2017 by tinimun

[tinimun]

Progress. Enabled both quirk_no_pm_reset and quirk_no_bus_reset for this device, and it enumerates in the Windows guest.

 

Here is a very high-resolution picture of the board after removing the cover. The actual SoC used (chip on the right) is the Mozart 395S combo PCIe controller/H.264 encoder. Here is the specsheet.

 

http://i.imgur.com/JMdBp2h.jpg

 

Finally, a kernel patch: https://gist.github.com/numinit/40fc5ad96fd0990b0a63/

 

I will keep this thread updated. Somehow managed to screw up my Windows user account during this process.

Edited April 15, 2017 by tinimun

[tinimun]

For some reason, the card enumerates, but stays on "No Signal." Looking into this.

 

I think bus resets are what breaks it, so keeping that quirk and removing the PM reset quirk.

[tinimun]

I'm very happy to report that today's work has paid off. Everything works! Not bad for a Saturday.

 

Here's a picture of my Wii U's output displaying on the Elgato application in a GPU passthrough Windows guest on the right, with the hypervisor's GPU on the left. The HDMI output from the card also works flawlessly, so you can mirror it on an external monitor as Elgato intended.

 

http://i.imgur.com/nrbZssD.jpg

 

Here's the kernel patch you need (tested against 4.4.5): https://gist.github.com/numinit/40fc5ad96fd0990b0a63

 

Raw URL: https://gist.githubusercontent.com/numinit/40fc5ad96fd0990b0a63/raw/de80ac51ba846b733968ddcac56f71e4dc070206/0001-Add-quirks-for-Yuan-Yuan-Enterprises-capture-cards.patch

 

And, in /etc/modprobe.d/vfio.conf or similar:

options vfio-pci ids=12ab:0380 disable_idle_d3=1

 

Essentially, idle d3 and bus resets are what hose the card. I'm going to see if I can get my patch merged into mainline.

Edited April 15, 2017 by tinimun

[zarroc24]

Is there a tutorial for getting this to work?

[allanp81]

Would any of this help me to get my tv tuner card working properly in a vm? While I can get the card recognised in the VM, I always have signal issues when tuning.

[tinimun]

9 hours ago, zarroc24 said:

Is there a tutorial for getting this to work?

 

Read up on recompiling your kernel. I don't actually use unraid (I'm on arch), but my patch should get you half of the way there. I'm still running it successfully on Linux 4.9.

 

The patch incantation I used in the Linux kernel source tree (once you've downloaded the file in my gist) was:

patch -Np1 -i /path/to/0001-Add-quirks-for-Yuan-Yuan-Enterprises-capture-cards.patch

 

6 hours ago, allanp81 said:

Would any of this help me to get my tv tuner card working properly in a vm? While I can get the card recognised in the VM, I always have signal issues when tuning.

 

The other half is setting an option for vfio-pci (specifically, the disable_idle_d3=1 one). @allanp81 - you might not need the kernel patch if the card is recognized in the VM. No guarantee that disable_idle_d3 will fix your case, but it's worth trying.

Edited April 15, 2017 by tinimun

[zarroc24]

When i try to run your patch i get an error saying:

Can't find file to patch at input line 15

Perhaps you used the wrong -p or --strip option?

 

It gets to this part of the patch and the asks for file to patch: 

From b8425aaca49e8f88d114426626cd8266e1e37d34 Mon Sep 17 00:00:00 2001
	From: Morgan Jones <[email protected]>
	Date: Sun, 13 Mar 2016 00:56:04 +0000
	Subject: [PATCH] Add quirks for Yuan Yuan Enterprises capture cards
	---
	drivers/pci/quirks.c | 7 +++++++
	include/linux/pci_ids.h | 4 ++++
	2 files changed, 11 insertions(+)
	diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
	index 7e32730..77d48d9 100644
	--- a/drivers/pci/quirks.c
	+++ b/drivers/pci/quirks.c

[tinimun]

6 hours ago, zarroc24 said:

When i try to run your patch i get an error saying:

Can't find file to patch at input line 15

Perhaps you used the wrong -p or --strip option?

 

Make sure you're running patch from the root directory of the Linux kernel source tree.

[zarroc24]

I ran it again in the root directory of the Linux Kernel and nothing. This is driving me crazy because i am able to see the sound portion of the card in my windows VM but not the Capture Card portion of it. There has to be something in the background still keeping a hold of it somehow. I have looked at all the tutorials to pass through a pci device and nothing. I am really out of ideas here specially since Linux is new territory for me and so is unRAID.

[TreeHair]

Can this thread come alive again?

I am stuck on the same issue listed above. I have the variables added to the bzroot portion of the Unraid startup. That being "pci-stub.ids=12ab:0380 vfio-pci.ids=12ab:0380 disable_idle_d3=1"

 

I am still stuck on where to patch this in. I will attach my logs, name and pci number for my card. Any help would be greatly appreciated.
D3 PCI Passthrough.txt

[jonfive]

On 10/7/2017 at 6:49 AM, TreeHair said:

Can this thread come alive again?

I am stuck on the same issue listed above. I have the variables added to the bzroot portion of the Unraid startup. That being "pci-stub.ids=12ab:0380 vfio-pci.ids=12ab:0380 disable_idle_d3=1"

 

I am still stuck on where to patch this in. I will attach my logs, name and pci number for my card. Any help would be greatly appreciated.
D3 PCI Passthrough.txt

Has anyone been able to get this to work?

[jieiku]

I had hoped to post a complete step by step for EndeavourOS (arch based distro), because I recently got an HD60 pro, I have been using a windows VM underlinux for more than a few years now, I only use windows for gaming at this point. I did build the kernel with the new patch, update grub, and can boot using the new kernel, but my entire machine still locks up whenever I try to passthrough this card (just as it did before trying this patch). Instead I will just show what I have done so far, most of the instruction came from here https://wiki.archlinux.org/title/Kernel/Arch_build_system:

 

sudo pacman -S devtools base-devel

mkdir ~/build/
cd ~/build/
pkgctl repo clone --protocol=https linux
cd ~/build/linux

sed -i 's/^pkgbase=linux/pkgbase=linux-hd60pro/' ~/build/linux/PKGBUILD
cp /media/storage/docs/hd60pro/hd60pro.patch ~/build/linux/src/
uname -r

# Edit the PKGBUILD (notice the two new hd60pro.patch lines):

prepare() {
  cd $_srcname

  echo "Setting version..."
  echo "-$pkgrel" > localversion.10-pkgrel
  echo "${pkgbase#linux}" > localversion.20-pkgname

  echo "Applying patch hd60pro.patch..."
  patch -Np1 -i ../src/hd60pro.patch

  local src


# I had to add the folowing two keys or makepkg failed with unknown key.

gpg --recv-keys 38DBBDC86092693E
gpg --recv-keys 3B94A80E50A477C7

makepkg -s

# This is the feedback that it actually applied the patch:

==> Starting prepare()...
Setting version...
Applying patch hd60pro.patch...
patching file drivers/pci/quirks.c
Hunk #1 succeeded at 3765 with fuzz 2 (offset 642 lines).
patching file include/linux/pci_ids.h
Hunk #1 succeeded at 1806 (offset 39 lines).
Applying patch linux-v6.6.3-arch1.patch...


# Now install the newly created kernel and then update grub

sudo pacman -U linux-hd60pro-headers-6.6.3.arch1-1-x86_64.pkg.tar.zst linux-hd60pro-docs-6.6.3.arch1-1-x86_64.pkg.tar.zst linux-hd60pro-6.6.3.arch1-1-x86_64.pkg.tar.zst

sudo grub-mkconfig -o /boot/grub/grub.cfg

 

The only thing I can think to do is to try and find some debug info following the crash, but honestly linux is normally rock solid stable for me so I am not even sure where to look first, and I have limited time to try and figure this out. If I make any further progress I will be sure to revisit and update my post.