archedraft Posted January 24, 2015 Share Posted January 24, 2015 To start with I have successfully passed through GPU's / USB controllers so I know that my server is able. Also, my unRAID server is using the onboard LAN port. So, I installed the NIC card and turn on the server and everything (including VM's) works just fine. I then add 0000:04:00.0 & 0000:04:00.1 to my vfio-bind command in my go file and reboot the server. The server and VM's start fine; however, I no longer can access unRAID through the web or SSH. The weird thing is my Windows VM can access the internet just fine. Why am I losing connectivity with unRAID when I pass through a PCIe NIC card?? Link to comment
archedraft Posted January 25, 2015 Author Share Posted January 25, 2015 SO I have noticed what I think is strange behavior. When I start my server without the Dual NIC installed my server starts and has a MAC address of AA:AA:AA:AA:AA:AA:AA. When I put install my Dual NIC card and start unRAID, the MAC address changes significantly to BB:BB:BB:BB:BB:BB:BB. The Ethernet cord is still plugged into the same onboard LAN port so why is would its MAC address change? Also, I have found that if I virsh nodedev-detach pci_0000:04:00.0 I can then pass through the device to my VM; however, if I add that code to vifo-bind in my GO file then I loose access to unRAID... I'm so confused! Link to comment
CHBMB Posted January 25, 2015 Share Posted January 25, 2015 I noticed something similar when I tried a dual-nic card on my UnRAID machine. Didn't really mess around with anything and all three Ethernet cables were plugged in and I would find that although I have a reserved address in my router for the onboard NIC sometimes Unraid was using one of the dual NICs as it's primary Ethernet adapter and therefore my MAC and IP address of my Unraid box would change. I didn't get any further investigating the issue as all I really wanted to know was that the card was working. I do intend at some point to run a VM with PFSense to experiment with, but it'll be a good while before I get to trying that. I know that hasn't helped solve your problem at all, but sometimes it's nice to know you're not the only one! My observation led me to believe that Unraid randomly selects a NIC on boot to use, my plan was to try and bind a specific controller at boot time, but I don't yet know how to go about it. Good Luck though! Link to comment
archedraft Posted January 25, 2015 Author Share Posted January 25, 2015 Thanks for the reply! It is nice to know that I am not the only one. The thing I find weird is currently I only have 1 Ethernet cable plugged into my server and the dual NIC card ports are both unplugged, so why would it change its MAC address if it only has access to the onboard port? I wonder if there is a way to bind unRAID to only use a certain network port? Link to comment
CHBMB Posted January 25, 2015 Share Posted January 25, 2015 Well I think that a MAC address is hardware bound and not dependent on whether an IP address is allocated or not. So Unraid boots, sees three NICs, randomly chooses one as the primary Ethernet whether it has an IP address or not, hence you're seeing a change in the MAC address you expect and not able to access the server as it's not on the LAN and doesn't have a IP address. Link to comment
CHBMB Posted January 25, 2015 Share Posted January 25, 2015 This thread looks like it gives some clues on network setup. You don't want to bond your NICs together but might be able to get some clues on how to go about things. http://lime-technology.com/forum/index.php?topic=35575.0 Link to comment
archedraft Posted January 25, 2015 Author Share Posted January 25, 2015 This looks like what I want: http://lime-technology.com/forum/index.php?topic=34872.msg325436#msg325436 Link to comment
dmacias Posted January 25, 2015 Share Posted January 25, 2015 This looks like what I want: http://lime-technology.com/forum/index.php?topic=34872.msg325436#msg325436 I was gonna point you to that thread. I have it in my favorites and been meaning to try that but just haven't made time. Let us know how it goes. Link to comment
archedraft Posted January 25, 2015 Author Share Posted January 25, 2015 I'll be attempting this within the next few days. It looks like I disable the bridge in unRAID and then setup eth1 & eth2 as bridges. Then I guess all my VM's would use eth2 (in0) as the bridge? Link to comment
CHBMB Posted January 26, 2015 Share Posted January 26, 2015 Please let us know how you got on, I for one will be very interested to see how things go, and may follow in your footsteps! Link to comment
archedraft Posted February 2, 2015 Author Share Posted February 2, 2015 UPDATE: I removed the br0 setup and modified my GO file to create a out0 and in0 bridge. pfSense 2.2 is up and running. I have noticed one issue so far: The virtIO network drivers need some work (my upload speed is 0.03 Mbps ... SO BAD) I changed the network model type to e1000 and that fixes the upload issue. I'll have to wait until the pfSense team has time to fix the virtIO driver before I switch back. Other than that it seems to be stable. Link to comment
CHBMB Posted February 18, 2015 Share Posted February 18, 2015 Got around to putting my dual NIC in today and found the exact same behaviour. The MAC address changed, I think k I got a bit confused when I read your post but now I get what you mean. Did you ever find the reason for this? Link to comment
archedraft Posted February 18, 2015 Author Share Posted February 18, 2015 Got around to putting my dual NIC in today and found the exact same behaviour. The MAC address changed, I think k I got a bit confused when I read your post but now I get what you mean. Did you ever find the reason for this? Nope, but I have been running pfSense as a VM for the last couple of weeks. Haven't had any issues with it so that has been nice. I also changed the the e1000 drivers back to VirtIO and my upload is working as it should... Not sure why it's working now though but I am not complaining. Link to comment
CHBMB Posted February 18, 2015 Share Posted February 18, 2015 Nope, but I have been running pfSense as a VM for the last couple of weeks. Haven't had any issues with it so that has been nice. I also changed the the e1000 drivers back to VirtIO and my upload is working as it should... Not sure why it's working now though but I am not complaining. Cool, mind posting that bit of your go file, I'd be interested to see how you did that. Link to comment
archedraft Posted February 18, 2015 Author Share Posted February 18, 2015 Go file contents: # Start Bridges brctl addbr out0 brctl addbr in0 brctl stp out0 on brctl stp in0 on brctl addif out0 eth2 brctl addif in0 eth1 ifconfig eth1 up ifconfig eth2 up ifconfig out0 up ifconfig in0 up Link to comment
pcwiz13 Posted February 18, 2015 Share Posted February 18, 2015 archedraft - I'm hoping to start setting up a PfSense VM tonight using bridges. I plan to do basically the same thing you've done. Have you seen any problems with allowing your VM's to use eth2 br0? Is this port connected to a switch for other machines to use as the connection to pfsense? Does each VM get an IP address from PfSense? Thanks Link to comment
archedraft Posted February 18, 2015 Author Share Posted February 18, 2015 archedraft - I'm hoping to start setting up a PfSense VM tonight using bridges. I plan to do basically the same thing you've done. Have you seen any problems with allowing your VM's to use eth2 br0? With mine I have my modem connected to out0 and then all my VM's use in0. The VM's haven't had any issues that I have noticed. Is this port connected to a switch for other machines to use as the connection to pfsense? My in0 port's ethernet connects to a switch and then that switch is used for other computers (including unRAID as eth0). All the machines connected to the switch and VM's are controlled by the pfSense VM. Does each VM get an IP address from PfSense? Yup Link to comment
pcwiz13 Posted February 19, 2015 Share Posted February 19, 2015 archedraft - I'm hoping to start setting up a PfSense VM tonight using bridges. I plan to do basically the same thing you've done. Have you seen any problems with allowing your VM's to use eth2 br0? With mine I have my modem connected to out0 and then all my VM's use in0. The VM's haven't had any issues that I have noticed. Is this port connected to a switch for other machines to use as the connection to pfsense? My in0 port's ethernet connects to a switch and then that switch is used for other computers (including unRAID as eth0). All the machines connected to the switch and VM's are controlled by the pfSense VM. Does each VM get an IP address from PfSense? Yup Easy enough, I have PfSense up and running using bridges. Tomorrow I add my new switch and access point. Thanks for the info! Link to comment
CHBMB Posted February 20, 2015 Share Posted February 20, 2015 archedraft - I'm hoping to start setting up a PfSense VM tonight using bridges. I plan to do basically the same thing you've done. Have you seen any problems with allowing your VM's to use eth2 br0? With mine I have my modem connected to out0 and then all my VM's use in0. The VM's haven't had any issues that I have noticed. Is this port connected to a switch for other machines to use as the connection to pfsense? My in0 port's ethernet connects to a switch and then that switch is used for other computers (including unRAID as eth0). All the machines connected to the switch and VM's are controlled by the pfSense VM. Does each VM get an IP address from PfSense? Yup Easy enough, I have PfSense up and running using bridges. Tomorrow I add my new switch and access point. Thanks for the info! I'm confused now I'm starting my setup, so you guys are no longer passing through the NIC but instead setting up bridges in Unraid to be used in the VMs? Link to comment
pcwiz13 Posted February 20, 2015 Share Posted February 20, 2015 I'm confused now I'm starting my setup, so you guys are no longer passing through the NIC but instead setting up bridges in Unraid to be used in the VMs? Thats exactly what I'm doing, my board doesn't support VT-D so I am using bridges instead. Its only been a couple of days, but so far it seems stable enough to replace my router, for home use at least. Link to comment
archedraft Posted February 20, 2015 Author Share Posted February 20, 2015 Yeah, I could not successfully pass through the NIC because the unRAID web GUI would fail to load. Link to comment
CHBMB Posted March 4, 2015 Share Posted March 4, 2015 Yeah, I could not successfully pass through the NIC because the unRAID web GUI would fail to load. Right, not sure whether to start a new thread for this but got around to trying it at last. Got my bridges all set up as your config. Confirmed they're both working by using them in Windows VMs When I come to install PFsense it can't find the WAN interface. Tried to load the drivers but it couldn't find them either. Running brctl show in0 8000.0015176fd75a yes eth1 vnet0 vnet2 out0 8000.0015176fd75b yes eth2 vnet1 Tried entering both out0 and vnet1 as WAN names but no luck and it's not automatically discovering it either. The WAN interface is connected to an unmanaged switch which then goes to my modem/router. Any ideas where I'm going wrong? EDIT: Here's a copy of my xml entry for completeness. <interface type='bridge'> <mac address='52:54:00:a0:18:6e'/> <source bridge='out0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/> </interface> <interface type='bridge'> <mac address='52:54:00:06:43:3a'/> <source bridge='in0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/> </interface> Link to comment
archedraft Posted March 4, 2015 Author Share Posted March 4, 2015 When I come to install PFsense it can't find the WAN interface. Tried to load the drivers but it couldn't find them either. So when you try to assign devices on pfsense it does not show 2 available? pfsense should give you the option to assign vtnet0 and vtnet1 as your 2 ethernet ports. Or do you mean that pfsense is not giving you an external ip address after you assign the 2 ethernet ports? Link to comment
CHBMB Posted March 4, 2015 Share Posted March 4, 2015 I'm not given any opportunity to assign vtnet0 or vtnet1, Boot the iso to a launcher. Checked all the option, tried to use multi boot and to install it by pressing i. Clearly I'm missing something. EDIT: Just checked, during the install it states no interfaces found, do I need to load any drivers? Link to comment
CHBMB Posted March 4, 2015 Share Posted March 4, 2015 OK, getting somewhere now, my problem was with my xml. Once archedraft made me realise that I should have the option to assign vtnet0 & 1 and I saw the no interfaces found message I did a bit more experimenting, Changing my xml to <interface type='bridge'> <mac address='52:54:00:a0:18:6e'/> <source bridge='out0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> and only using out0 pfsense was able to see the interface and I could assign it. Now I'm left with the problem of how to define in0 I tried <interface type='bridge'> <mac address='52:54:00:06:43:3a'/> <source bridge='in0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/> </interface> But that gives me an error and won't save. The bridges are bound to two ports on a dual NIC, and I know I'm just getting confused conceptually. I just can't see where. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.