Constant video buffering

[nwootton]

I used to run unRAID Plus v5 on a 780G AMD board with 2Gb RAM and a single Supermicro AOC-SASLP-MV8 board. Everything ran fine and the setup was perfectly able to serve BR rips across the network without buffering issues. A recent gift of a MSI Z87-G45 motherboard complete with i5 CPU and 8Gb RAM combined with the demise of the current board saw me migrating the system, including upgrading to v6, across into a new chassis.

 

Since then, we have had nothing but trouble with video buffering issues.

 

Initial issues with the on-board KillerNIC not working at all were resolved by disabling in in the BIOS and adding a dual port Intel Gb card. On the unRAID dashboard the card never gave speeds over 100Mbps. The local switch also showed the network speed running at 100Mbps.

 

My initial thought was that I had used the wrong cable between the switch and the server when connecting it. So I went through and ensured that every cable was Cat5e rated or higher at all points. The dashboard and switch still reported 100Mbps and the buffering still happened. Multiple restarts with cables in various states of connection did nothing to change the speeds.

 

Assuming that the dual-port NIC was the cause of the issue, I replaced the card with an Intel 82573L PCI-Ex1 card. Both switch and dashboard now happily report a 1Gbps connection, but the buffering still occurs on playback.

 

I’m running Plex, Maria and SB/CP/SAB as Docker containers, but turning these off makes no difference. CPU, network, IO and memory are all running low - there are no periods of sustained activity. The only other plugin I am running is the Docker container search tool - everything else is default.

 

Buffering occurs on both of our Kodi (15.2) boxes, OpenELEC (Kodi 14), VLC on Win10 & OSX 10.9, as well as on a Roku 3 using the Plex client.

 

In desperation, last night I rolled back to v5 and the same thing still occurs serving files from that version as well.

 

The remainder of the network hasn’t changed, and the end point devices are still the same hardware that used to play these files without issue. While the Kodi boxes have been updated from 14 to 15.2 at various points, the fact that buffering occurs on VLC as well indicates to me that Kodi isn’t the problem. As far as I can tell, it’s not the network - the same buffering occurs no matter where I connect a playback device.

 

unRAID was up to date (v6.1.3). No discs are showing errors on the dashboard, although I have had a ‘Warning [TOWER] - offline uncorrectable is 1’ message yesterday against my cache drive.

 

I’ve not got any logs available - I’m posting this from work, but I can get tonight if required.

 

The family are getting VERY upset that things are unavailable/unwatchable. Does anyone have any suggestions about other things I can try?

[RobJ]

Download the diagnostics (Tools -> Diagnostics) and check the system folder for ifconfig and ethtool, and check them for problems.  Post them here if unsure (or attach the diagnostics zip).

[nwootton]

@RobJ

 

Thanks - I'll get them out ASAP - just need to upgrade back to v6 tonight.

[nwootton]

As requested - here's the diagnostic dump. I can't see anything that indicates an issue.... Any opinions appreciated 

tower-diagnostics-20151002-2354.zip

[ashman70]

What if the problem is not network so much as something to do with the data going from the hard drives out of the network? Are all the drives connected to onboard SATA ports? I assume everything you have tried in terms of streaming results in buffering issues? Have you tried connecting your UnRaid server to a small switch, then connecting a laptop running kodi to the same switch to see if the buffering is still there? I am new to UnRaid myself and not familiar with it enough to know if any of its settings or configuration could be causing this.

[nwootton]

@ashman70. The unRAID box is connected to an eight port Netgear GB switch. Both my desktop and a Mac laptop suffer the same issue when connected directly to this switch. I've tried Kodi and VLC on both.

 

The drives are split between the on-board SATA and a Supermicro AOC-SASLP-MV8 card. The card was the same one used in my previous build, but I have also tried swapping it out for a different on of the same type.

[Russ Uno]

Have you tried playback of a file from each drive to see if it follows a drive or set of drives?

I noticed that even though I have all Seagate Barracudas, 7200 rpm, 1TB drives they do vary in speed.

I have 2 slower than the rest,  about half the speed, even though they are all on the MB SATA ports and all my SATA Ports are the same speed.

Probably not enough to effect playback of 1080p files but worth checking.

Or you can also try SpeedDisk. http://lime-technology.com/forum/index.php?topic=31073.0

[ashman70]

But everything under the old setup worked, so what changed?

 

Motherboard, CPU, RAM and Network card, this still didn't remedy the problem.

 

Could it be a setting in the MB BIOS? What are the CPU and RAM doing when you try to play a video file? Are they maxing out for some reason?

 

Could it be some compatibility issue with your motherboard? As suggested, have you tried playing a file that resides on a drive that is connected to an onboard SATA port and one that is connected to a drive connected to the Supermicro card?

 

Just throwing ideas out there.

[nwootton]

@Ashman70 - The bits that changed were the unRAID server and the Gb switch at the other end of the cable in the AV rack. I've put the old switch back in the rack and it hasn't made any difference.

 

@Russ - I've tried playing back SD content from each drive through Kodi, but even that failed.

 

In fact the situation appears to be worsening  :'( the Sonos now reports that it can't play back the music from unRAID as 'there isn't enough bandwidth to maintain a buffer'.

 

Most of the day has been spent swapping out switches and checking cable types. I'm leaning towards either the motherboard BIOS is doing something to limit output (and that is going to be painful to figure out) or physical damage to the cables in the walls. I add that last one, but every switch reports that its got a gig connection (both lights lit or light colour etc etc). Swapping to lower capability cables does change the reported state on the switch - just in case you ask 

 

I think I'm going to try the drive speed test that @Russ suggested, but the next course of action is to acquire some proper network testing kit and see what that says about the entire data path. If that passes with full GB capability, the BIOS must be the answer.... Assuming the diagnostics posted above don't show something I've missed.

[ashman70]

I would focus on the BIOS, your network can't be the culprit even though the problem is presenting as if it is. My feeling is its something motherboard related.

[opentoe]

Did you go back to basics and just copy a large file from each disk to your client computer and see what speeds were obtained? What are your speeds on a copy with a large file, like an ISO or MKV file that's around 5-10GB?

 

 

[RobJ]

Assuming the diagnostics posted above don't show something I've missed.

Sorry, should have commented earlier, but I didn't find anything of note in your diagnostics, and forgot to say so.

[nwootton]

@RobJ glad to know that I didn't miss anything  Thanks for looking though.

[RXWatcher]

Does it say in that log the link speed and duplex? I might have missed it.

 

This sounds a lot like what I  used to see in 100mbit / half duplex connections. You'd have to force the nic and switch to FULL to prevent connection issues.

Gbit should usually be set to Auto but I wonder if even that is set to half duplex?

 

[RobJ]

Does it say in that log the link speed and duplex? I might have missed it.

 

Speed: 1000Mb/s

Duplex: Full

Oct  2 22:59:47 Tower kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

[Russ Uno]

Ethernet cables? I ran in to a speed issue yesterday with my MacPro backing up to my backup server.

I was only getting 3.4 MB/s, thought it would pick up but 9 hours later if finally finished a 115 GB backup.

I checked the link speed on UnRaid and it was 1Gbit, all good, checked my Mac Pro, 4 feet away from the server, with Network utility on the Mac and it said 100 Mbit.

Turns out in a pinch I used an old Ethernet cable to the switch and forgot to replace it. I put a new cable in and I'm back to 1Gbit.

So even though UnRaid and my other Macs had 1Gbit connections the MacPro didn't. 

 

Also run htop in UnRaid in Terminal and watch the CPU usage.. I was shocked at how much more UnRaid v6 uses compared to v5.

I had to go back to v5 for my Media server until I can upgrade. My backup server is no issue with v6, plenty of HP there.

[nwootton]

@Russ all the cables are Cat5E or Cat6. I also bought a mini LAN tester and none of them show faults. In fact I actually binned all the older cat5 cables as part of the checking, just to prevent one accidentally ending up somewhere it shouldn't in the future 

 

The CPU is an i5 backed with 8Gb RAM, and it's not shown any sign of straining. Rolling back to v5 on this box had no effect on the issue either.

[nwootton]

Just a general update, I reflashed the BIOS last night and I've not yet had time to see if that made any difference. I've also used a new mini LAN tester to check the cables and they all pass - no shorts, crosses or faults reported.

 

I'm also planning on sticking a software LAN speed tester on each end of the data path to see what it says, but that will have to wait until the weekend as I can't run up and down stairs all night - it'll wake the kids and then I'll be in even more trouble 

 

Thanks for all the help and suggestions. I'll update once I've had a chance to do some more testing.

[Russ Uno]

You know you can use "Activity Monitor" in "Utilities" from the Mac to watch Network transfer rates..

After swapping cables yesterday and doing a new backup today I'm seeing 72 MB/s as apposed to 3.4 MB/s I was getting with the old cable.

[nwootton]

Just an update before I mark this issue as solved.

 

It looks like the issue is a failing Netgear GS608 switch in the study next to the unRAID server. Actually to be 100% accurate, it looks like one or two of the ports is starting to fail, but that means it all goes!

 

A long evening of testing every link in the data path indicated that the issue was with a switch in my loft patch panel. Spending most of yesterday swapping out this switch (four times!) still didn't solve the issue. Eventually I went back and tested every stage again, and this time the issue looked like it was the study switch.

 

Swapping the study switch for another Netgear (GS105 this time) meant everything started to work. I'm planning to do some more testing tonight, but I will mark this as closed for now.

 

Thank you for all your help & suggestions. I might find some time to actually watch my content now rather than just swear at a stuttering stream 

 

[opentoe]

Just an update before I mark this issue as solved.

 

It looks like the issue is a failing Netgear GS608 switch in the study next to the unRAID server. Actually to be 100% accurate, it looks like one or two of the ports is starting to fail, but that means it all goes!

 

A long evening of testing every link in the data path indicated that the issue was with a switch in my loft patch panel. Spending most of yesterday swapping out this switch (four times!) still didn't solve the issue. Eventually I went back and tested every stage again, and this time the issue looked like it was the study switch.

 

Swapping the study switch for another Netgear (GS105 this time) meant everything started to work. I'm planning to do some more testing tonight, but I will mark this as closed for now.

 

Thank you for all your help & suggestions. I might find some time to actually watch my content now rather than just swear at a stuttering stream 

 

You have passive switches all over the house like that? So far in the loft and in the study. Switches give headaches. If you don't live in a mansion best practices are one major switch (for a newbie, one big hub) then direct lines running to each client/computer. At least for all the non portable stuff. Like TV's, Receivers, workstations, etc. Glad you found the problem. Even those switches are unmanaged and act more like hubs they do carry a lifetime warranty from Netgear. Netgear will replace it for you.