Running UnRaid off Virtual NIC from OPNSense VM

[Damarine]

I'm in the process of upgrading my internal network to 2.5 gbe, and after looking at the consumer routers and their prices, I realized I'm better off creating my own using OPNSense.. but I'm having some trouble with getting Unraid (Version: 6.11.5) to run off that network.

 

This is my current network. I have simplified the diagram by removing the irrelevant connections (ie: Phones, tvs, etc) but essentially I have cascading routers for redundancy since I know running OPNSense in a VM isn't always the most pleasant experience.

As it stands, I have two wires running from my Standalone router going into my server running Unraid which is also running the OPNSense VM. The VM has a quad 2.5 GBe NIC being passed through. My PC is now connected to the VM NIC but the issue of course is, in order to connect to Unraid I'm going back to the standalone router causing a bottleneck. This is what I'm trying to accomplish, but for some reason I can't figure it out. This is what i have setup so far:

1) OPNSense running in VM:

* 2.5 gbe PCIe Quad Nic passed through
* VM also has a VMXNet3 virtual NIC being passed through
* I set up a bridge between the LAN and VMXNet3 port
* Setup firewall to allow traffic in/out through the bridge

The issue I'm having is how do I get UnRaid to recognize the VMXNet3 port? From my understanding, the virtual NIC runs off of br0. The issue is my Unraid doesn't have br0 listed as an interface. After doing more research I found out some information essentially stating when Docker is using br0, it becomes exclusive to docker and UnRaid doesn't have access to it. Currently I have AdGuard Home running in a docker container which has the network br0. How do I go about solving this issue? I tried creating a new custom docker network and running AdGuard Home off of that, but for some reason AdGuard Home doesn't work off of any network except br0, so I'm not sure what I'm missing. I'm relatively new to networking so I apologize if I've done anything wrong so far.

[Damarine]

Update: I've distilled down the issue to a simple question:

 

How do I assign an IP to UnRaid through br0 instead of eth0. I have br0 forwarded as a VMXNet3 to OPNSense, OPNSense sees the virtual port, it's connected and has a DHCP lease, it has access to internet; yet UnRaid doesn't see any of this at all. It detects no IP, and no internet.

Edited March 26, 2023 by Damarine

[itimpi]

4 minutes ago, Damarine said:

Update: I've distilled down the issue to a simple question:

 

How do I assign an IP to UnRaid through br0 instead of eth0. I have br0 forwarded as a VMXNet3 to OPNSense, OPNSense sees the virtual port, it's connected and has a DHCP lease, it has access to internet; yet UnRaid doesn't see any of this at all. It detects no IP, and no internet.

Unraid always expects eth0 to be its main LAN link.    You therefore need to get an IP address assigned to that.

 

it might be a good idea to provide your system’s diagnostics zip file assigned to your next post in this thread so we can see how you have your networking set at the Unraid level.

[Damarine]

19 minutes ago, itimpi said:

Unraid always expects eth0 to be its main LAN link.    You therefore need to get an IP address assigned to that.

 

it might be a good idea to provide your system’s diagnostics zip file assigned to your next post in this thread so we can see how you have your networking set at the Unraid level.

 

I've changed up my network quite a bit since initially posting because I've been trying to figure this out, so I'm not sure how helpful the diagnostics will be but I have attached them.

 

In the set up you'll notice I have since reconnected  Eth0 (and by extension br0) back up to the main router. So it's getting a static IP from there. I have a second onboard NIC (eth1) on the host machine which I'm using for testing. (Basically emulating what it's like to not have Eth0 connected, but br0 be the one with internet access except on the eth1/br1 interface)

 

Additionally what I have done is reverse the PC <-> VM passthrough, before I had all four PCIe NIC ports being passed through to OPNSense and now i have one of those ports remaining assigned to UnRaid (eth2) for 2.5 Gbe to the NAS and it's corresponding virtual br2 is assigned to OPNSense which gives it internet access.

damarine-diagnostics-20230326-0754.zip

Edited March 26, 2023 by Damarine