85% packet loss - 6.2rc3 and 6.1.9

[ryanhaver]

I've been struggling to even get 6.2.0-rc3 working at all on my setup. I am getting upwards of 85% packet loss on a brand new install. This was not an upgrade of any sort...., no existing settings or data....just plugged in a new USB with UNRAID 6.2.0-rc3 and four drives that were not initialized.

 

The web interface is virtually inaccessible and the hardware I am using works fine with any number of other operating systems. I did notice that by default it is trying to bond all 4 built in NICs, although I'm not sure this is the problem because I was able to manually tell UNRAID to use only one of the NICs.

 

I have tested multiple cables, connected to various network switches all with the same result. There is no IP address conflict, but I was using DHCP and then manually configured a static IP as well, but same result. I have seen the packet loss from various client to the UNRAID server and from the UNRAID server to clients on the network.

 

Motherboard: Supermicro A1SRi-2758F

NICs: Built in Intel i354 quad gigabit Ethernet

UNRAID Version: 6.2.0-rc3

 

 

Edit: Seems like this it a bigger problem than just 6.2.0-rc3. I've also tested 6.1.9 with the same results. I had a USB with an Older version of UNRAID installed and it appears to be working, but anything new is just foobar at the moment.

[Squid]

Could be something else on your network bombing the communications.  Or an app, etc.

 

Diagnostics might be of help.

[ryanhaver]

I'll try to stick it on a separate network segment...alone, to remove other network traffic as a variable. Just seems weird that an older v6 build of unRAID worked while newer ones are having this issue.

[John_M]

Do you happen to have Jumbo frames enabled on some hosts but not on others?

[ryanhaver]

Jumbo frames are not enabled anywhere. I still haven't had time to look into this further, but do find it strange that two other operating systems did not exhibit this issue on this hardware.

[Frank1940]

Is the issue you are referring to Receive Drops as reported on the Dashboard? 

 

If so, those are not really indicative of a big problem.  ...Unless you are have an other problem with networking.  If you want to look for the source of the problem be prepared to spend a lot of time. 

 

Right now I have 63,674,945 Receive packets and 28,067,275 Receive Drop packets.  Now, I  have done  careful investigation into my issue and know that 98%+ of those drops happened whenever I play a BluRay iso on any one of my three Netgear NTV550 Media Players.  I suspect that these players flood the server with packet requests when playing a BluRay iso!  My INTEL network card is simply pitching the duplicates.  (I can also tell you that the on-board RealTek NIC does NOT have the problem with  the drops but it has other issues in my setup!)

[ryanhaver]

Is the issue you are referring to Receive Drops as reported on the Dashboard? 

 

If so, those are not really indicative of a big problem.  ...Unless you are have an other problem with networking.  If you want to look for the source of the problem be prepared to spend a lot of time. 

 

Right now I have 63,674,945 Receive packets and 28,067,275 Receive Drop packets.  Now, I  have done  careful investigation into my issue and know that 98%+ of those drops happened whenever I play a BluRay iso on any one of my three Netgear NTV550 Media Players.  I suspect that these players flood the server with packet requests when playing a BluRay iso!  My INTEL network card is simply pitching the duplicates.  (I can also tell you that the on-board RealTek NIC does NOT have the problem with  the drops but it has other issues in my setup!)

 

The worst part for me is that the packet loss is on a brand new install, out of the gate. No applications, plugins, tweaks....100% just plugged a USB in and booted to it. I can understand some packet loss under heavy load, but the server is doing nothing. The Web GUI is completely unusable due to the excessive amount of packet loss.

[Frank1940]

 

 

The worst part for me is that the packet loss is on a brand new install, out of the gate. No applications, plugins, tweaks....100% just plugged a USB in and booted to it. I can understand some packet loss under heavy load, but the server is doing nothing. The Web GUI is completely unusable due to the excessive amount of packet loss.

 

OK, The GUI being not usable is a problem.  (If you google dropped packets, you will find that most of the time, they are harmless.) 

 

First thing,  get a diagnostics file down and attach it to your next post.  If you can't get the file from 'Tools'  >>  'Diagnostics' icon, use the command line and type  diagnostics  .  As I recall, the command line will put the file on your flash drive in the 'logs' folder.  ( I did a quick lookup on your MB and the NIC manufacturer was not one that I recognized.)

 

Second thing, double check that you haven't set some unusual type of flow control on your switches.  If it is a managed switch, set it back to the factory default. 

 

Thrid thing, Go in through the IPMI port and make sure that those other four network ports are not setup in some type of bridge formation. 

[ryanhaver]

Upgraded motherboard BIOS and then defined a static address for the dedicated IPMI port. Things are working now. 0 Packets dropped at this point.

 

I found a few results in google searches regarding issues others people have seen with the IPMI port when not configuring a static address. It appears that the IPMI interface bridges into the other NICs without a static IP assigned to it.

 

Unfortunately SuperMicro doesn't offer many options to manage their IPMI port....and the default config seems to break things under certain circumstances.

[ryanhaver]

Issue came back again after a reboot. I guess I'll just install ESXi for now.

 

Edit: I continue to expedience issue with network connectivity on any of the newer build of unRAID. To me to appears this is an issue with how network stack is implemented. Earlier version of unRAID are not affected by these issues.

[RobJ]

I don't know if this will make a difference at all, but check out the Tips and Tweaks wiki page, and use the Tips and Tweaks plugin to tweak various network settings.  In particular, try turning off network offloading.  I believe several users have seen their drops drop to near zero.

[ryanhaver]

I finally took some time to do some real troubleshooting. This is now resolved. Had nothing to do with unRAID.

 

I have never had this happen before, but it was a mac address conflict. The vmkernel port (for management network) on my new ESXi host, for my home lab, was using the same mac address as the physical NIC of the unRAID machine. Really weird....but glad it wasn't hardware or even software on the unRAID side.

 

All good now. Thanks for the recommendations.