Passthrough eSATA PCI-e Card


N4TH4N

Recommended Posts

Hey,

 

I'm trying to passthrough a PCI-e eSATA card to a Windows VM i plan to use for data recovery. The reason i wish to passthrough the whole card is so i can use the port multiplier function with my 4 Bay HDD Dock directly with the Windows VM.

 

The problem i'm facing is that when i pci-stub the card according to what i have read it also disables an onboard SATA controller.

 

04:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
04:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
06:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
06:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)

04:00.0 0106: 197b:2363 (rev 03)
04:00.1 0101: 197b:2363 (rev 03)
06:00.0 0106: 197b:2363 (rev 03)
06:00.1 0101: 197b:2363 (rev 03)

 

Is it possible for me to passthrough only:

 

06:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
06:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)

 

Whilst leaving the other controller connected to unRAID.

 

Thank-you.

Link to comment

Buy a different eSata card that doesn't use a JMB363 chip on it. Or don't stub it just use the change to the XML.

 

 

The only reason I stubbed mine was I didn't want the drive letters in unRAID to go to double digits (sdaa) but it worked fine NOT to stub it.  You just have to be careful not to add one of those drives to the array but they showed up as NTFS file system for me so I just made sure not to select a drive with NTFS as the file system.

Link to comment

Buy a different eSata card that doesn't use a JMB363 chip on it. Or don't stub it just use the change to the XML.

 

 

The only reason I stubbed mine was I didn't want the drive letters in unRAID to go to double digits (sdaa) but it worked fine NOT to stub it.  You just have to be careful not to add one of those drives to the array but they showed up as NTFS file system for me so I just made sure not to select a drive with NTFS as the file system.

 

Thanks for the reply,

 

I didn't realise i could pass it through without disabling it first.

 

Thanks.

Link to comment

When i add the following to my XML

 

<hostdev mode='subsystem' type='pci' managed='yes'>
  <driver name='vfio'/>
  <source>
    <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
  </source>
</hostdev>

 

I get the following error

 

internal error: early end of file from monitor: possible problem:
2016-02-15T09:36:51.468435Z qemu-system-x86_64: -device vfio-pci,host=06:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: error, group 6 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2016-02-15T09:36:51.468468Z qemu-system-x86_64: -device vfio-pci,host=06:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: failed to get group 6
2016-02-15T09:36:51.468480Z qemu-system-x86_64: -device vfio-pci,host=06:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization failed
2016-02-15T09:36:51.468490Z qemu-system-x86_64: -device vfio-pci,host=06:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device 'vfio-pci' could not be initialized

 

Does this mean that i also have to include the 06:00.1 of the controller as well or is this because i didn't stub it.

 

Sorry if i'm asking so many questions, i have just come from ESXi which made all of these things easy.

Link to comment

I would try to pass that through as well.  It is complaining about not all devices on the same iommu_group are not being passed through so if you pass that through it might work.

 

Thanks for the reply,

 

I am not exactly sure how i can assign both functions to passthrough.

 

I have tried the following, with no luck still.

 

Thanks.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </hostdev>

Link to comment

It looks like other devices are in the same IOMMU Group ?

 

PCI Devices

00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 12)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 12)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 12)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 12)
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 12)
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 12)
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 12)
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 12)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.1 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 2
00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 3
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series]
03:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
04:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
04:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
06:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
06:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03)
ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05)
ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05)
ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05)
ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05)
ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05)
ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05)
ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05)
ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05)
ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05)
ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05)
ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05)
ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05)
ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05)
ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05)
ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05)
ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05)
ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05)
ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05)

 

IOMMU Groups

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:03.0
/sys/kernel/iommu_groups/3/devices/0000:00:07.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.1
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/4/devices/0000:00:14.3
/sys/kernel/iommu_groups/5/devices/0000:00:1a.0
/sys/kernel/iommu_groups/5/devices/0000:00:1a.1
/sys/kernel/iommu_groups/5/devices/0000:00:1a.2
/sys/kernel/iommu_groups/5/devices/0000:00:1a.7
/sys/kernel/iommu_groups/6/devices/0000:00:1c.0
/sys/kernel/iommu_groups/6/devices/0000:00:1c.1
/sys/kernel/iommu_groups/6/devices/0000:00:1c.2
/sys/kernel/iommu_groups/6/devices/0000:00:1c.4
/sys/kernel/iommu_groups/6/devices/0000:04:00.0
/sys/kernel/iommu_groups/6/devices/0000:04:00.1
/sys/kernel/iommu_groups/6/devices/0000:05:00.0
/sys/kernel/iommu_groups/6/devices/0000:06:00.0
/sys/kernel/iommu_groups/6/devices/0000:06:00.1
/sys/kernel/iommu_groups/7/devices/0000:00:1d.0
/sys/kernel/iommu_groups/7/devices/0000:00:1d.1
/sys/kernel/iommu_groups/7/devices/0000:00:1d.2
/sys/kernel/iommu_groups/7/devices/0000:00:1d.7
/sys/kernel/iommu_groups/8/devices/0000:00:1e.0
/sys/kernel/iommu_groups/9/devices/0000:00:1f.0
/sys/kernel/iommu_groups/9/devices/0000:00:1f.2
/sys/kernel/iommu_groups/9/devices/0000:00:1f.3
/sys/kernel/iommu_groups/10/devices/0000:01:00.0
/sys/kernel/iommu_groups/11/devices/0000:02:00.0
/sys/kernel/iommu_groups/11/devices/0000:02:00.1
/sys/kernel/iommu_groups/12/devices/0000:03:00.0
/sys/kernel/iommu_groups/13/devices/0000:ff:00.0
/sys/kernel/iommu_groups/13/devices/0000:ff:00.1
/sys/kernel/iommu_groups/14/devices/0000:ff:02.0
/sys/kernel/iommu_groups/14/devices/0000:ff:02.1
/sys/kernel/iommu_groups/15/devices/0000:ff:03.0
/sys/kernel/iommu_groups/15/devices/0000:ff:03.1
/sys/kernel/iommu_groups/15/devices/0000:ff:03.4
/sys/kernel/iommu_groups/16/devices/0000:ff:04.0
/sys/kernel/iommu_groups/16/devices/0000:ff:04.1
/sys/kernel/iommu_groups/16/devices/0000:ff:04.2
/sys/kernel/iommu_groups/16/devices/0000:ff:04.3
/sys/kernel/iommu_groups/17/devices/0000:ff:05.0
/sys/kernel/iommu_groups/17/devices/0000:ff:05.1
/sys/kernel/iommu_groups/17/devices/0000:ff:05.2
/sys/kernel/iommu_groups/17/devices/0000:ff:05.3
/sys/kernel/iommu_groups/18/devices/0000:ff:06.0
/sys/kernel/iommu_groups/18/devices/0000:ff:06.1
/sys/kernel/iommu_groups/18/devices/0000:ff:06.2
/sys/kernel/iommu_groups/18/devices/0000:ff:06.3

 

USB Devices

Bus 004 Device 007: ID 1058:10b8 Western Digital Technologies, Inc. Elements Portable (WDBU6Y, WDBUZG)
Bus 004 Device 009: ID 045e:0750 Microsoft Corp. Wired Keyboard 600
Bus 004 Device 008: ID 0557:8021 ATEN International Co., Ltd CS1764A [CubiQ DVI KVMP Switch]
Bus 004 Device 006: ID 0409:005a NEC Corp. HighSpeed Hub
Bus 004 Device 005: ID 1058:0820 Western Digital Technologies, Inc. My Passport Ultra (WDBMWV, WDBZFP)
Bus 004 Device 004: ID 058f:6387 Alcor Micro Corp. Flash Drive
Bus 004 Device 003: ID 13fe:5500 Kingston Technology Company Inc. 
Bus 004 Device 002: ID 174c:1053 ASMedia Technology Inc. 
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 010 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 009 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

 

SCSI Devices

[0:0:0:0]    disk             PATRIOT          0     /dev/sda 
[1:0:0:0]    disk             Patriot Memory   PMAP  /dev/sdb 
[2:0:0:0]    disk    JetFlash Transcend 4GB    8.07  /dev/sdc 
[3:0:0:0]    disk    WD       My Passport 0820 1012  /dev/sdd 
[3:0:0:1]    enclosu WD       SES Device       1012  -        
[4:0:0:0]    disk    WD       Elements 10B8    1012  /dev/sde 
[5:0:0:0]    disk    ATA      WDC WD30EZRX-00M 0A80  /dev/sdf 
[5:0:1:0]    disk    ATA      WDC WD15EARS-00M AB51  /dev/sdg 
[5:0:2:0]    disk    ATA      WDC WD20EARX-00P AB51  /dev/sdh 
[5:0:3:0]    disk    ATA      WDC WD30EZRX-00D 0A80  /dev/sdi 
[10:0:0:0]   disk    ATA      WDC WD30EZRX-00D 0A80  /dev/sdj 
[11:0:0:0]   disk    ATA      WDC WD30EZRX-19D 0A81  /dev/sdk 
[12:0:0:0]   disk    ATA      WDC WD30EZRX-00D 0A80  /dev/sdl 
[13:0:0:0]   disk    ATA      WDC WD30EZRX-00D 0A80  /dev/sdm 
[14:0:0:0]   disk    ATA      ST3000DM001-1ER1 CC25  /dev/sdn 
[15:0:0:0]   disk    ATA      WDC WD30EZRX-00D 0A80  /dev/sdo 
[18:0:0:0]   disk    ATA      INTEL SSDSC2BW12 0956  /dev/sdp 
[18:1:0:0]   disk    ATA      INTEL SSDSC2BW12 DC32  /dev/sdq 
[19:0:0:0]   disk    ATA      ST3000DM001-1CH1 CC27  /dev/sdr

Link to comment

I have a 2x USB 3.0 Card in a PCIe slot thats assigned to a different group.

 

I was hoping to forward this card to a VM also but if it means i can use my eSATA card instead ill sacrifice it.

 

01:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)

/sys/kernel/iommu_groups/10/devices/0000:01:00.0

 

Link to comment

After physically swapping the ports i got past the other error about grouping but now get the following error.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </hostdev>

 

internal error: early end of file from monitor: possible problem:
2016-02-16T04:40:43.996264Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x7: vfio: failed to set iommu for container: Operation not permitted
2016-02-16T04:40:43.996304Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x7: vfio: failed to setup container for group 10
2016-02-16T04:40:43.996312Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x7: vfio: failed to get group 10
2016-02-16T04:40:43.996324Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x7: Device initialization failed
2016-02-16T04:40:43.996335Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x7: Device 'vfio-pci' could not be initialized

Link to comment

After checking dmesg and it having this error present:

vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

 

I added this to my /boot/syslinux.cfg file after append and before bzroot

intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1

 

Then rebooted.

 

With the following code in my XML and the allow_unsafe_interrupts in the syslinux.cfg i managed to get my VM to boot with my PCIe card.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </hostdev>

 

Link to comment

Hey Guys,

 

I just pulled the card out of my unRAID system and put it into another to do a driver test which went well so the card is infact functional and works correctly with the drivers i was supplying it with.

 

The problem is that this is happening when passing through via KVM.

 

2je5i8j.png

 

Its a device cannot start error.

 

Any suggestions as to what i'm doing wrong ?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.