Solved - Unable to Transmit from VM on BR1

[Security_IE]

All,

 

Interesting situation. My server 6.5.3 has the onboard NIC and a two port NIC. When adding a VM to BR0, tied to the onboard/management network, everything is good. I want to place some VMs on my second NIC (ETH1) and bridge that to a different VLAN on my switch. I have no IP address on ETH1, only a bridge (BR1) configuration. The VMs can't talk out to my switch. Wireshark shows that the VM is receiving frames from my switch (EIGRP hellos, BPDUs, etc) but nothing back. See below for troubleshooting steps taken:

 

1. Validated inbound traffic from switch with wireshark on VM

2. Validated network card in separate computer (OK)

3. Moved network card to different PCIe slot

4. Validated cable and physical port/configuration on switch to separate machine (OK)

5. Validated wireshark in one VM can see frames from another VM spun up on BR1

 

At this point, it appears that frames sent out from the VM make it on the bridged virtual LAN inside the server, but don't transmit out the NIC. Any troubleshooting I can do internally from here? I must be missing something simple. Any help appreciated.

Edited June 25, 2019 by Security_IE
Add software version

[testdasi]

Why is IP address assignment is none? Should be static or DHCP?

[Security_IE]

The IP address for my server is on ETH0. These VMs should get an IP from my DHCP. I just need to extend the VLAN from my switch port to these VMs. The switch is the gateway for the subnet.

Edited June 25, 2019 by Security_IE
Attach Diagram

[Vr2Io]

Does your VM also set correct VLAN ?

[Security_IE]

Just now, Benson said:

Does your VM also set correct VLAN ?

My VMs are all set to BR1. The VLAN should be determined by the port on the switch They can communicate with each other and hear broadcast/multicast from the switch but nothing returns. I get EIGRP, CDP,ARP etc in the VM from a wireshark capture, but nothing out. Example: Ping from switch to VM and ARP tables don't populate. Add the ARP entry on the switch and pings are seen at the VM but no responses, since the VM can't get an ARP entry for the switch. Add an ARP entry on the VM via netsh and I capture echo request and reply (VM doesn't need to arp for switch MAC) but the response never makes it to the switch.

 

Here is one of my VMs. The others are the same except the "target dev" changes per host.

 

<interface type='bridge'>
      <mac address='52:54:00:4c:25:27'/>
      <source bridge='br1'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

I'm configuring as per the use case shown here: https://lime-technology.com/wordpress/wp-content/uploads/networking-vm-use-cases.pdf for physical segregation on page 5. VMs work fine when on BR0 but fail to transmit to the switch when on BR1. I have other hosts on the same VLAN as BR1 that are not running on my server.

[testdasi]

Try giving your eth1 an IP from the switch (either via DHCP or static). I can only get my bridge to work if the eth1 device has an IP.

[Security_IE]

Just now, testdasi said:

Try giving your eth1 an IP from the switch (either via DHCP or static). I can only get my bridge to work if the eth1 device has an IP.

I did try that earlier. Configured a static IP in the same subnet. I'll try again with DHCP to see if that changes anything. I really don't need the host to be accessible through the interface. After adding ARP entries on hosts and switch, I'm down to troubleshooting the bridge.

 

root@achilles:~# brctl showmacs br1
port no mac addr                is local?       ageing timer
  1     00:0a:cd:2d:3d:2f       yes                0.00
  1     00:0a:cd:2d:3d:2f       yes                0.00
  1     00:19:06:95:26:91       no                 1.26
  1     00:19:06:95:26:c1       no                 1.26
  4     52:54:00:06:48:0a       no                39.05
  2     52:54:00:4c:25:27       no                 4.64
  1     b8:8a:ec:37:f8:e2       no               102.17
  4     fe:54:00:06:48:0a       yes                0.00
  4     fe:54:00:06:48:0a       yes                0.00
  2     fe:54:00:4c:25:27       yes                0.00
  2     fe:54:00:4c:25:27       yes                0.00
  5     fe:54:00:ff:b0:eb       yes                0.00
  5     fe:54:00:ff:b0:eb       yes                0.00
root@achilles:~#

 

[Vr2Io]

23 minutes ago, Security_IE said:

VMs work fine when on BR0 but fail to transmit to the switch when on BR1.

Note, if work on BR0, then it seems Unraid issue. If BR0 and BR1 were same subnet, then it may have problem, pls also try set both bridge network not in same subnet if they currently was.

Edited June 25, 2019 by Benson

[Security_IE]

Both bridges are in different subnets. I'm going to call it an issue between unraid and my NIC. Unfortunately, I don't have a spare PCIe card to test with. After putting an IP on my server for ETH1, I see the server traffic being broadcast out from within wireshark in the VM but it does not show up on the switch. For background, I run a Cisco switch and have a span set up on the port to my server. I capture this on a separate machine, where I can see all of the inbound/outbound traffic from the switch on that physical port. As mentioned, that outbound broadcast/multicast traffic also shows up on the wireshark capture running on my VM tied to BR1. I can ping between two VMs tied to BR1 with no issues. I'll look at getting an intel PCIe card and swap for testing. After going all the way down to the arp entries shown in BR1, I think this is a bug of some sort. If it resolves with a different NIC, I'll post a follow up.

 

Thanks to those who commented.

[Security_IE]

Solved - Network Card Issue with Unraid. I replaced my two-port NIC with two Intel EXPI9301CTBLK cards. All is well. No IP address needed on the server for ETH1 or 2. My access port in VLAN 2 is passed through with no issues to the VMs through ETH1 via BR1. I also configured ETH2 with VLAN sub-interfaces and mapping the VM to BR2.2 works as well, with 802.1q trunking on my switch port. Unraid is managed through my onboard NIC, ETH0

 

See the information on my Realtek two-port card in the output from LSPCI in my original post. This card is physically good. It works fine in my Win10 desktop.

 

Thank You

Edited June 28, 2019 by Security_IE