ethernet passthrough

[luca2]

Hi jon,

Did you passthrough an ethernet adapter (pci)? If yes or no, pls how should I proceed:

a) hostdev?

b) qemu ?

Feel free to post the corresponding code from any of your xml´s.

Rgds.

[jonp]

Hi jon,

Did you passthrough an ethernet adapter (pci)? If yes or no, pls how should I proceed:

a) hostdev?

b) qemu ?

Feel free to post the corresponding code from any of your xml´s.

Rgds.

So you want to pass through an entire PCI Ethernet controller to a guest.  The host will no longer have access to that controller then. Is that what you intend?  Is this a separate controller from what your host will use?

[luca2]

Hi Jonp,

 

Yes. I have 2 ethernet adapters.

 

My setup:

lspci -nn | grep net
05:05.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller [10ec:8169] (rev 10)
09:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller [10ec:8168] (rev 09)

I am trying to passthrough 05:05.0.

1) qemu is not working:

    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=05:05.0,bus=pcie.0'/>

2) hostdev not working:

<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
     <address domain='0x0000' bus='0x00' slot='0x14' function='0x4'/>
  </source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
     <address domain='0x0000' bus='0x05' slot='0x05' function='0x0'/>
  </source>
</hostdev>

My go file:

#!/bin/bash
mkdir -p /mnt/vm
mount /dev/disk/by-id/ata-SanDisk_SDSSDXP120G_132507400152-part1 /mnt/vm
# Start the Management Utility
/usr/local/sbin/emhttp &
/usr/local/sbin/vfio-bind 0000:01:00.0 0000:00:14.2 0000:00:14.4 0000:05:05.0

 

Rgds.

 

[johnodon]

Which OS is the guest?

[stottle]

How do you know it isn't working?  Is libvirt giving an error?  Or is the VM starting, but networking isn't available?  It isn't clear if this is a host or guest problem.

 

Did you install drivers for the networking controlling in the guest?

[johnodon]

How do you know it isn't working?  Is libvirt giving an error?  Or is the VM starting, but networking isn't available?  It isn't clear if this is a host or guest problem.

 

Did you install drivers for the networking controlling in the guest?

 

Exactly why I asked which guest OS?    The device may be present but no drivers loaded.

[jonp]

How do you know it isn't working?  Is libvirt giving an error?  Or is the VM starting, but networking isn't available?  It isn't clear if this is a host or guest problem.

 

Did you install drivers for the networking controlling in the guest?

 

Exactly why I asked which guest OS?    The device may be present but no drivers loaded.

Host = unraid. Guest = VM.

[jonp]

luca,

 

Ok, a few things.

 

#1 Confirm that after booting, your ethernet controller is assigned to VFIO.

 

You do this by using the following command:

 

lspci -k

 

You should see something like this:

 

04:00.0 Multimedia video controller: Device 1a0a:6202 (rev 01)
Subsystem: Avermedia Technologies Inc Device 620c
Kernel driver in use: vfio-pci

 

The "Kernel drive in use" should report vfio-pci, otherwise pass through using VFIO will not work.

 

#2 add the following code to your XML

 

<hostdev mode='subsystem' type='pci' managed='yes'>
  <driver name='vfio'/>
    <source>
      <address domain='0x0000' bus='0x05' slot='0x05' function='0x0'/>
    </source>
</hostdev>

 

The above code can be used as a template for any non-VGA PCI device you want to pass through.  The QEMU command line arguments are really only required to use VFIO with VGA devices (graphics cards).  Technically if you do it this way, you shouldn't even need to do vfio-bind in your go script, because libvirt should do all that for you.

 

If you do this and the VM starts, your pass through is successful, but if it doesn't appear to work inside your VM, it could be that you need to load the drivers into your VM.  What OS is the VM that you're trying to do this with?

[JustinChase]

Did you passthrough an ethernet adapter (pci)? If yes or no, pls how should I proceed:

 

Can I ask why?  the virtio driver is super fast and reliable.

 

I'm sure you've got a good reason, I'm just curious.

[jonp]

Did you passthrough an ethernet adapter (pci)? If yes or no, pls how should I proceed:

 

Can I ask why?  the virtio driver is super fast and reliable.

 

I'm sure you've got a good reason, I'm just curious.

One good reason would be to build a VM as a firewall / router for which you want complete isolation for that traffic from the host.  There are other reasons as well, but they really are niche compared to what the majority of folks will do.

[JustinChase]

Okay, that makes sense.  Maybe I'll get around to such a project someday.

[luca2]

Which OS is the guest?

 

Sorry for delay..but I had to remove from my main setup my r9290 and put it in my unraid test setup. It was watercooled ... and I only had one pump around ...

 

GUest OS is Windows 8.1 x64.

[luca2]

How do you know it isn't working?  Is libvirt giving an error?  Or is the VM starting, but networking isn't available?  It isn't clear if this is a host or guest problem.

 

Did you install drivers for the networking controlling in the guest?

 

I could not install the drivers since I could not plug in an usb-stick. I had to solve this first.

 

Funny thing I realised is that even if I passthrough my usb-stick, once I take it off ... and then back in ... now it is not recognized by windows. I have not found a solution for this. Maybe Jonp knows a solution for this?

[luca2]

 

Vfio is active:

05:05.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller (rev 10)
        Subsystem: Realtek Semiconductor Co., Ltd. RTL8169/8110 Family PCI Gigabit Ethernet NIC
        Kernel driver in use: vfio-pci
        Kernel modules: r8169

 

This is my output from vmmgr:

pci_0000_05_05_0 	0x10ec:0x8169 	vfio-pci 	Realtek Semiconductor Co., Ltd. 	RTL8169 PCI Gigabit Ethernet Controller

 

<device>
  <name>pci_0000_05_05_0</name>
  <path>/sys/devices/pci0000:00/0000:00:14.4/0000:05:05.0</path>
  <parent>pci_0000_00_14_4</parent>
  <driver>
    <name>vfio-pci</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>5</bus>
    <slot>5</slot>
    <function>0</function>
    <product id='0x8169'>RTL8169 PCI Gigabit Ethernet Controller</product>
    <vendor id='0x10ec'>Realtek Semiconductor Co., Ltd.</vendor>
    <iommuGroup number='11'>
      <address domain='0x0000' bus='0x00' slot='0x14' function='0x4'/>
      <address domain='0x0000' bus='0x05' slot='0x05' function='0x0'/>
    </iommuGroup>
    <numa node='0'/>
  </capability>
</device>

 

You only select this...

<address domain='0x0000' bus='0x05' slot='0x05' function='0x0'/>

and not this too ...

      <address domain='0x0000' bus='0x00' slot='0x14' function='0x4'/>

?

 

I still need to try adding "<driver name='vfio'/>"...

 

 

 

 

[luca2]

#2 add the following code to your XML

 

<hostdev mode='subsystem' type='pci' managed='yes'>
  <driver name='vfio'/>
    <source>
      <address domain='0x0000' bus='0x05' slot='0x05' function='0x0'/>
    </source>
</hostdev>

This is the error I get if i try this in my xml:

message: Error while starting domain: internal error: early end of file from monitor: possible problem: 2014-11-11T23:14:46.831971Z qemu-system-x86_64: -device vfio-pci,host=05:05.0,id=hostdev2,bus=pci.2,addr=0x5: vfio: Error: Failed to setup INTx fd: Device or resource busy 2014-11-11T23:14:47.167835Z qemu-system-x86_64: -device vfio-pci,host=05:05.0,id=hostdev2,bus=pci.2,addr=0x5: Device initialization failed. 2014-11-11T23:14:47.167901Z qemu-system-x86_64: -device vfio-pci,host=05:05.0,id=hostdev2,bus=pci.2,addr=0x5: Device 'vfio-pci' could not be initialized 

[luca2]

I will check if my card shows up in windows without doing hostdev.