VMs and Unraid unable to connect to eachother

[NerdyGriffin]

Starting last night, my two ubuntu VMs are unable to connect to unraid and vice-versa.

 

In the following examples, I will be replacing the real IP addresse of the Unraid server with 10.0.0.XX and that of the VM with 10.0.0.YY

 

If I try to ssh from Unraid to a VM, I get the following error

ssh: connect to host 10.0.0.YY port 22: No route to host

If I run a ping to the VM IP from Unraid I get the following error:

ping -c 4 10.0.0.YY
PING 10.0.0.YY (10.0.0.YY) 56(84) bytes of data.
From 10.0.0.XX icmp_seq=1 Destination Host Unreachable
From 10.0.0.XX icmp_seq=2 Destination Host Unreachable
From 10.0.0.XX icmp_seq=3 Destination Host Unreachable
From 10.0.0.XX icmp_seq=4 Destination Host Unreachable

--- 10.0.0.YY ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3041ms
pipe 2

The result of running traceroute on the unraid server:

UNRAID:~$ traceroute 10.0.0.YY
traceroute to 10.0.0.YY (10.0.0.YY), 30 hops max, 60 byte packets
 1 unraid.lan (10.0.0.XX)  3055.969 ms !H  3055.943 ms !H  3055.933 ms !H

 

If I ssh from the VM to unraid it hangs for several minutes before giving me `Connection timed out'

If I ping from the VM to Unraid:

VM:~$ ping -c 4 10.0.0.XX
PING 10.0.0.XX (10.0.0.XX) 56(84) bytes of data.

--- 10.0.0.XX ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3057ms

If I traceroute from VM to Unraid:

VM:~$ traceroute 10.0.0.XX
traceroute to 10.0.0.XX (10.0.0.XX), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

 

However, when I ssh to the VMs from my Windows desktop or from a Raspberry Pi it works just fine.

The same goes for ssh to the Unraid server from those physical devices.

 

It seems like all the physical devices on the network can connect to Unraid and the VMs as normal, but the VMs and Unraid are unable to connect to eachother.

[NerdyGriffin]

I have found a temporary workaround in Unraid's routing table which allows me to restore the connection between Unraid and the VMs.

 

If I add an entry to the Unraid routing table using the VM IP address as the route entry and the IP of the router as the gateway, it fixes the connection issue until the next time I reboot Unraid.

 

For now I have added the following lines to my /boot/config/go to add these routes at startup:

# Workaround for Unraid VM's unable to connect to host server
ip route add 10.0.0.8/29 via 10.0.0.1 metric 1
ip route add 10.0.0.16/29 via 10.0.0.1 metric 1

 

This results in the following routing table:

VM:~$ ip route list
default via 10.0.0.1 dev bond0 metric 1
default via 10.10.128.1 dev bond0.10 metric 10
10.0.0.0/16 dev bond0 proto kernel scope link src 10.0.0.2 metric 1
10.0.0.8/29 via 10.0.0.1 dev bond0 metric 1
10.0.0.16/29 via 10.0.0.1 dev bond0 metric 1
10.10.128.0/17 dev bond0.10 proto kernel scope link src 10.10.128.2 metric 1
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev br-52fc2ba579fe proto kernel scope link src 172.18.0.1 linkdown
172.19.0.0/16 dev br-3be2b68f3367 proto kernel scope link src 172.19.0.1 linkdown
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

 

Notice that the default table entry for the `10.0.0.0/16` subnet does not include a `via`, which would imply that it should be using the `default via 10.0.0.1 ...`. However, in practice it is failing to route packets between Unraid and the VMs (as explained in my original post). By adding routing table entries that explicitly route the VM IPs to the router (`via 10.0.0.1`) I am able to restore the connection.

 

 

This allows me to continue using the VMs as normal for now, but it does not identify and fix the root cause of the problem. I would like to understand why this is happening in the first place.