I've been troubleshooting some network issues today (spotify and local iTunes playback buffering, pausing and my wife's work laptop connection seemed unstable).  During this I came across the subject recurring message in my unraid log.  I've been researching in the forums and I gather this is most likely a network switch configuration issue.  Network is all unmanaged switches and I can't seem to find a culprit connection/device (though truly haven't disconnected everything across the LAN yet as I was working on just the 8 port switch where unraid is connected).  I was on Unriad 6.11.1 when I started investigating this and figured it was a good excuse to upgrade to 6.11.5 and I moved Docker over to ipvlan too for good measure.  Unfortunately problem is persisting.  All my dockers seem fine - I don't run any vm's currently.

 

I'd appreciate any thoughts (attached diagnostics in event my configuration is messed up somehow?).  Meanwhile I'll keep trying to isolate stuff across the network (my house is heavily wired so there's quite a bit of potential places to look at).

Aside from getting to root cause of these messages am I correct that this would cause excessive network traffic and possibly impact overall network stability?  Would be glad to get to bottom of this and learn it also stabilizes my network overall!

Appreciate any help/thoughts.  Thanks!

tick-diagnostics-20230123-1637.zip

Updating here for "future reference" to possibly aid anyone else with similar issue.

After several hours of network troubleshooting I think I've narrowed it down to a specific 5 port switch (completely across the house of course).  Switch is actually < 1 month old so possibly an OOB failure.  I left that area of the network "off" for a few hours (disconnected switch from LAN) and the messages in Unraid stopped during that time (and all my other network problems seem to stabilize too).  Within 5 minutes of adding the switch back the messages appeared.  I swapped it out with spare switch and seems good now - will keep monitoring in event it's a device that location.

I did observe while trying to find the source that whenever those broadcast messages in Unraid popped up in waves I'd start to see the network issues I was initially noticing (I left Spotify streaming across house while working on this and it would typically drop out when I'd see the message flood).  So it seems this was likely all related.  Given I just recently observed these network issues and I only recently added this switch it does make sense.

I assume this a multi switch setup? Are the switches managed? It's possible you have incorrect network loop prevention setting (like spanning tree control disabled) causing a broadcast storm

 

Of course, the switch may be bad too, but I have been burned once long ago with same issue on my network where the switch turned out to be fine, just did not have the right setup for handling network loops

Yea it's' a multi-switch setup and all are unmanaged switches so not much I can do there to configure them.  After I posted last night to say I narrowed it down to a specific switch I saw the Unraid messages pop up again with that switch removed - so clearly something else going on.

I've read a bit on STP (but can't say I completely understand it).  I have a Sonos system which can cause broadcast storms if configured with multiple Sonos devices hardwired if STP isn't properly handled (Sonos uses STP in this configuration).  I had recently adjusted my Sonos setup to improve performance (was seeing some speakers drop out audio on occasion) and that included hardwiring additional speakers.  From what I've read this setup should be ok if the switches aren't managed so I assumed that wouldn't be the culprit.  Just to be safe though last night I reconfigured Sonos again (only 1 Sonos hardwired - a more standard setup).  I still saw some of the broadcast loopback messages in Unraid log after that change but the frequency is significantly reduced and I haven't seen anything today for past 12+ hours.  I'm don't think I've solved this completely but improved it I guess.

Much appreciate you sharing thoughts on this!  Helpful to brainstorm it (especially since I'm pretty uniformed on network stuff like this).

 

As for ideas, I would start with 2 things

# check for any physical network loops. A switch connected to another switch twice, directly or indirectly

# setup another client on same network to monitor broadcast packets. Something like Wireshark or similar, that can tell you where the traffic is coming from. This may be helpful, but ideally should be handled at network layer rather than taming the broadcast originator. So your solution may be elsrwhere

Also worth note that source address in this case is same as unraid. You I don't think it's your Sonos. Do you have multiple separate interfaces on unraid connected to a switch? 

Thanks again for suggestions/help!
 

On 1/25/2023 at 3:51 AM, apandey said:

As for ideas, I would start with 2 things

# check for any physical network loops. A switch connected to another switch twice, directly or indirectly

# setup another client on same network to monitor broadcast packets. Something like Wireshark or similar, that can tell you where the traffic is coming from. This may be helpful, but ideally should be handled at network layer rather than taming the broadcast originator. So your solution may be elsrwhere

 

Thanks!  I've been methodically (I think 🙂) working through your first suggestion - checking the network configuration and removing sections when I can as a test.  So far I go a few hours thinking I found the culprit only to see the error show up in Unraid logs a bit later.

I have found since reconfiguring my Sonos I'm getting these errors much less frequently - I went > 24 hours with nothing and then a burst for about a minute.  Some more reading about Sonos tells me that hard-wiring multiple speakers (as I had done) definitely requires proper handling of STP and if any network switches fail to properly handle that it can cause major issues.  Most who run that configuration seem to have managed switches configured for STP and loopback prevention.  Over the years I just added unmanaged switches across my LAN (didn't think I had need for managed and obviously they cost more) which may not have been advisable.  Regardless since I setup the Sonos with just 1 connected device I've seen improved stability across the LAN and these broadcast issues on Unraid log are much less frequent - so that's something!
 

On 1/25/2023 at 3:56 AM, apandey said:

Also worth note that source address in this case is same as unraid. You I don't think it's your Sonos. Do you have multiple separate interfaces on unraid connected to a switch? 

 

I an confirm there is only 1 LAN Connection to the Unraid Server - no other interface (wired or wireless).  I was wondering though if a docker in bridged network configuration could cause this?  I do have a docker running with vpn connection and acting as proxy for other dockers (such that multiple dockers are all routing through the one VPN connection).  I'll occasionally see in the docker log that the VPN connection dropped and it's struggling to reconnect.  A restart of that docker usually resolves it.  I'm assuming that's not the cause of my issues but just thinking through other possibilities.
 

This is a tricky problem to solve (much crawling around the house and checking hardwired connections) but at least I made big improvements on network stability to this point (it was becoming unusable and impacting ability to WFH).  I'll keep monitoring and maybe play around with wireshark or something to see if I can learn more (good chance to learn how to use wireshark I guess!)

 

Thanks once again - the ideas are much appreciated!

 

21 minutes ago, tmembrino said:

This is a tricky problem to solve

 

22 minutes ago, tmembrino said:

good chance to learn how to use wireshark

Agree both