NotYetRated

I have been experiencing server lock-up issues lately. I first attributed it to updating to 6.5RC, but have since downgraded back to 6.4.1 and have been experiencing the same issues. Randomly, my dashboard, main, docker and VM tabs become unresponsive in the WebUI. I can still access my shares, users, settings, plugins, stats and tools tabs. System log shows this: Mar 6 22:43:17 BigBang nginx: 2018/03/06 22:43:17 [error] 8803#8803: *5794 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Main HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.20", referrer: "http://192.168.0.20/Main/Device?name=cache" Mar 6 22:44:34 BigBang nginx: 2018/03/06 22:44:34 [error] 8803#8803: *6324 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Dashboard HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.20", referrer: "http://192.168.0.20/Users" Mar 6 22:46:44 BigBang nginx: 2018/03/06 22:46:44 [error] 8803#8803: *6499 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Main HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.20" Mar 6 22:46:45 BigBang nginx: 2018/03/06 22:46:45 [error] 8803#8803: *7079 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /VMs HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.20", referrer: "http://192.168.0.20/Main" Mar 6 22:47:08 BigBang nginx: 2018/03/06 22:47:08 [error] 8803#8803: *7144 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Main HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "bigbang" Mar 6 22:47:10 BigBang nginx: 2018/03/06 22:47:10 [error] 8803#8803: *7145 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Docker HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "bigbang", referrer: "http://bigbang/Main" Mar 6 22:48:14 BigBang nginx: 2018/03/06 22:48:14 [error] 8803#8803: *6869 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.0.61, server: , request: "GET /Main HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.20", referrer: "http://192.168.0.20/Main/New?name=sdl" Diags will not generate. It just stays at "Please wait, collecting diagnostic information", but i did manage to putty in a generate one. They are attached. I recently replaced an SSD, but was having these same random issues from before that replacement. The SSD was one on unassigned devices, which I keep my VM's on. The new SSD is used the same, just more space. I also swapped my PCIe USB device to a different slot, as there was thinking that was helping drive some lockups. No such luck in fixing my issue. Any help would be appreciated, kind of pulling my hair out here. bigbang-diagnostics-20180306-2250.zip

That is not something I even thought about, thank you.

DOH, thank you!

I had some time to try and troubleshoot some server issues i have been having. Issues have been noticed since updating to 6.5.0rc4. 3 times now when "ctrl+t" on a keyboard on VM to open a new chrome tab, the entire VM locked up. Two of the 3 times the server itself became unresponsive and I had to force a reboot via IPMI. This last time it happened the server itself did not lock up, and I was able to reboot to VM. VM is a windows 10 VM, with an entire PCIe USB controller passed through. Video card is also passed through. Diags attached, though the log entries relating to my issue appear to be below. Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: AER: Uncorrected (Non-Fatal) error received: id=8018 Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=8018(Requester ID) Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: device [8086:3c08] error status/mask=00004000/00000000 Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: [14] Completion Timeout (First) Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: broadcast error_detected message Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: broadcast mmio_enabled message Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: broadcast resume message Mar 4 15:49:05 BigBang kernel: pcieport 0000:80:03.0: AER: Device recovery successful bigbang-diagnostics-20180304-1557.zip

Just a couple of quick notes from me: 1. Power supplies seem vastly over-spec'd. I am running a server with 12 HDD's, 4 SSD's, Dual V1 E5's, dual Nvidia GeForce 1060 GTX and have 750 WATT platinum power supply. Even with both gaming VM's going, by Blue Iris server running, 1 or 2 Plex streams, I still only hit ~600 Watts pulled. My idle puts me right around the sweet spot for power supply efficiency. Running way over spec'd is running at quite low efficiency. I know you say the power bill is no problem, its just my observation. I am not on solar yet so try and watch efficiencies here and there... 2. Do you live in a hot climate? Is the garage cooled? Summers by me can hit 100 F, a server in my garage would seriously cook 10 or so days a year. (though my winters are nice and cold, so that would be nice on the opposite end!) 3. Everything else looks pretty damn good to me. Though I am not intimately familiar with backpanes to be of much use there... Enjoy!

I ask a sh!t load of questions, and most always get some fantastic replies. Sometimes those replies are links to other threads of my exact same question(yes, I am poor at forum searching), but I have never gotten a "go google it".

How much space are you thinking you will need?

Awesome videos, hoping tips here will help me with my 2 VM's experiencing audio/video desync as well as random audio static type issues I have been experiencing. Sorry if I missed this, but how are people checking latency on their VM's?

To chime in here, I have been using 2 gaming VM's in unRaid since the unRaid 6 release. Initially, it was all perfectly smooth sailing with rarely ever a hiccup. Over the past.... I dont know 6-12 months I have been getting sound stuttering, locking up, audio/video desync issues left and right. With old AMD 6450 cards as well as new Nvidia 1060's. No amount of tinkering has fixed any of it yet, nor the most recent unRaid update. I personally am giving up and will be transitioning away from gaming/video pass through on my unraid box until the issues are resolved. Just my $0.02.

Awesome write up, thanks! Dumb question, is this more secure than my current method of Firewall blocking all but certain IP's? I feel like my method is more secure, but less accessible, that sound accurate?

I have got to say I am incredibly surprised at how constant the temperature in the rack stays despite the garage temperature! I suppose that comes from cutting the rack fans below a set temperature. How high to rack temps get in your summer? I live in Northern Ohio, USA and get a month or so of 80 F outside, with 85-90F at times in my garage. I would think thats too high an ambient for my server/case. Do you see dust making it in to your server at all? I used to use those style mesh filters for intake on my closet server room, but found I was still getting a ton of fine dust through. I ended up replacing my 2x 120mm fans for 2x 240mm fans and built an intake filter using a furnace filter. Dust is now near 0 into the room, though i do have to change the filter every 2 months.

Another update, girlfriend accidentally took and installed windows updates and viola, the sync issues have reappeared. Im out of patience and ideas.

I just use CPU pinning option in the "Extra Parameters" field of my dockers to specifically pin certain cores to certain dockers. This way, no docker, VM or otherwise can lock all of my cores up and affect other systems within my unRaid. "--cpuset-cpus=9,10,11,12,13,25,26,27,28,29" is the command, filling in the cores you want to pin.

Nice links, glad to see those worked too! I also forgot to mention, I put powered USB hubs at the end of all of my extensions. I was getting occasional drops when I was using Xbox One or Steam Pad controllers without a powered hub. With the hubs, no drops at all.

I used what I would consider cheap units from Monoprice, linked below. Working fantastic so far. Only issue is, 2 of the 3 I got, female ports are loose and I had to rubber band the connection to hold it together. Annoying once, then I have not had to deal with it since fixing. https://www.monoprice.com/product?p_id=6042

I just use 50' HDMI cables, and I do USB over ethernet. I have had no issues so far. I even have 2 runs where I have 50' HDMI, into a keystone jack, then out to a 6' HDMI to a receiver. No issues there apart from my original HDMI keystone being bad on arrival.

Hi all, just a quick update on my status/progress/findings. I was never able to correct the issue using AMD 6450 w/Windows 10. I was able to get it working perfectly using Windows 7 and the AMD 6450. I replaced the AMD 6450 and dropped an Nvidia 1050 Ti into the original Win10 machine having issues, and all is well with that! I did have to extract the ROM from the NVIDIA card, but it was only about a 5 minute process performing it for my first time.

I am using pfSense on unRaid for routing control, multi-WAN, firewall and some tp-link EAP's for wireless access points. The system works well for me, though I do wish pfSense had some better visual reporting of traffic etc as the photos above in this convo have. My current weak point, is I do not have a suitable backup for my pfSense VM. If unRaid goes down, I would need to plug my old Asus flashed to dd-wrt in. I would lose any of the configuration changes I have made in the last year as well as dual WAN setup. But its a risk i am fine with.

Subscribing here for data. I have been checking in to this as well, in terms of trying various cards with no luck. 3 of the same cards as Prograde mentioned in original post.

What VM OS? And what video card you passing through?

After installing a Windows 7 VM and letting Windows pick the audio/video drivers for my card, I have had a problem free experience for multiple hours across multiple different shows. On chrome, Internet Explorer, within Plex Media Player as well as Open Plex Home Theater. I have plex set to not transcode anything, all playing direct.

I am on frugaluse and they provide 40 connections. I set my sab to allow for 38 connections. Setting to 40 sometimes gives me too many connection errors. Not sure if nzbget functions different than sab, but you wanna set your total connections available to at or just below your providers given stats. As for your performance, I DL to my cache(1 TB SSD), extract to it, and my mover moves once a week. I do notice when a large file is extracting and im topping my DL speed @250mbps, things slow down a bit. Perhaps would be better if I separated my DL/extract locations, but it doesnt bother me just yet.

OpenPHT is my go-to for Windows based VM's for playback. I get the audio/video out of sync issues on the VM's in plex media player as well as any webbrowser, when using Windows 10. I swapped my VM's back to Windows 7 on Saturday and so far have had no audio/video sync issues in web browser, OpenPHT or PMP. However, I do not have any 4k content, so I cannot help much with the original question of the thread, sorry!

Update: Fresh VM's with Windows 10 did not fix the problem for me. Going to try a Windows 7 install tonight to see if that has any affect. No modifications within the Windows VM such as drivers, audio settings, etc worked. Replacing HDMI cables had no change. Running out of ideas!

I have pinned CPU cores for a while now, so doubt that will fix the issue. Just checked, and both of my VM's shows MSI as enabled. I have never tinkered with this to turn it on or off, so I assume its always been on, even when I was not having audio/video issues. It is quite maddening! I too have been using chromecast for a lot of stuff now. Also, I noticed I have this issue on my VMs using Plex Media Player, but not Plex Home Theater. Quite odd.... Still no luck identifying the issue or any solutions.