Rudder2 Posted December 27, 2018 Share Posted December 27, 2018 (edited) I've been having a problem with all my web UIs becoming unresponsive every 26 to 48 hours. This is unRAID, Plex, Sonarr, etc. During the shutdown process they start working again until the server kills the web UI processes. I'm sure it's something hanging up that is killed early in the shutdown process but don't know what. I SSH in to the server and do a diagnostics collection and the diagnostics appears to hang till I initiate the shutdown then it does the diagnostics before it shuts down. Here is that diagnostics files from yesterday and today I did before shutdown from the SSH terminal since the Web UIs are unresponsive. If anyone could give me a clue I would greatly appreciate it. Thank you! rudder2-server-diagnostics-20181227-0928.zip rudder2-server-diagnostics-20181226-0431.zip Edited January 1, 2019 by Rudder2 Quote Link to comment
Rudder2 Posted December 27, 2018 Author Share Posted December 27, 2018 It just happened in 8 hours this time. Here is the New Diagnostics File. rudder2-server-diagnostics-20181227-1744.zip Quote Link to comment
BRiT Posted December 28, 2018 Share Posted December 28, 2018 I didn't see anything obvious from Syslog but looking at your Process and Top files it seems like LIDARR and Plex Media Server and Sonarr are using all your CPU: 30575 nobody 20 0 1171120 314952 23484 S 205.9 1.0 12:47.24 Plex Medi+ 31265 nobody 20 0 2010944 1.2g 17144 S 70.6 4.0 54:48.23 mono 23826 nobody 20 0 1945300 753844 2680 S 41.2 2.3 471:15.74 mono root 29040 0.0 0.0 7500 4068 ? Sl 09:31 0:00 | \_ docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/53a7490dbb07a01da06de7ed3f931d775f9a61aabeece176f5f1589695063a86 -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc root 29059 0.0 0.0 204 4 ? Ss 09:31 0:00 | | \_ s6-svscan -t0 /var/run/s6/services root 29227 0.0 0.0 204 4 ? S 09:31 0:00 | | \_ s6-supervise s6-fdholderd root 30569 0.0 0.0 204 4 ? S 09:31 0:00 | | \_ s6-supervise avahi 102 30576 0.0 0.0 45128 896 ? Ss 09:31 0:00 | | | \_ avahi-daemon: running [Rudder2-Server.local] root 30571 0.0 0.0 204 4 ? S 09:31 0:00 | | \_ s6-supervise dbus 101 30574 0.0 0.0 47496 848 ? Ss 09:31 0:00 | | | \_ dbus-daemon --system --nofork root 30572 0.0 0.0 204 4 ? S 09:31 0:00 | | \_ s6-supervise plex nobody 30575 2.5 1.0 1171120 342488 ? Ssl 09:31 12:47 | | \_ /usr/lib/plexmediaserver/Plex Media Server root 20890 0.0 0.0 7500 4120 ? Sl 09:31 0:00 | \_ docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/6dc5186956fbd6afcd068ff905d0fba65ae30ef88a37323515397ff5131bb6fa -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc root 20908 0.0 0.0 196 4 ? Ss 09:31 0:00 | | \_ s6-svscan -t0 /var/run/s6/services root 21798 0.0 0.0 196 4 ? S 09:31 0:00 | | \_ s6-supervise s6-fdholderd root 23824 0.0 0.0 196 4 ? S 09:31 0:00 | | \_ s6-supervise lidarr nobody 23826 95.5 2.3 1945300 753844 ? Ssl 09:31 471:15 | | \_ mono --debug /app/lidarr/Lidarr.exe -nobrowser -data=/config root 30731 0.0 0.0 7500 4564 ? Sl 09:31 0:00 | \_ docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/d919eb2e2e294620b1478862c22a1f60bbdf368f25d6506b1c5c81319e0d83fe -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc root 30759 0.0 0.0 196 4 ? Ss 09:31 0:00 | | \_ s6-svscan -t0 /var/run/s6/services root 30975 0.0 0.0 196 4 ? S 09:31 0:00 | | \_ s6-supervise s6-fdholderd root 31260 0.0 0.0 196 4 ? S 09:31 0:00 | | \_ s6-supervise sonarr nobody 31265 11.1 3.9 2009968 1293188 ? Ssl 09:31 54:48 | | \_ mono --debug NzbDrone.exe -nobrowser -data=/config Quote Link to comment
BRiT Posted December 28, 2018 Share Posted December 28, 2018 And from your UPTIME / Loads file: 17:44:58 up 8:14, 1 user, load average: 111.49, 114.23, 81.57 Cores: 4 😱 Quote Link to comment
Rudder2 Posted December 28, 2018 Author Share Posted December 28, 2018 I've found Lidarr to take WAY more CPU than I think it should. Every time I think the CPU is 100% it looks like it's Lidarr. Usually when the system is @ 100% cpu it still functions fine. This is the first time I've had a problem with the Web UI hanging...I will stop Lidarr and see what happens. Quote Link to comment
Rudder2 Posted December 30, 2018 Author Share Posted December 30, 2018 So, I undid my CPU isolation for my VM and the problem happened again. Honestly, I've never had this happen before and I've been running everything I have on this server for years. Here is the new diagnostics file if you could please review it and see if anything stands out to you. Thank you for your time and help in this very important matter. I can't have my server going down every 8 - 18 hours because this problem. I have a CPU I was told was over kill and it just started acting up the same week I started this post. I've been considering getting a much beefier system even though I didn't think I had to. I'm starting to think that it wouldn't matter because even after unisolating my cores it's the same. rudder2-server-diagnostics-20181230-1627.zip Quote Link to comment
Rudder2 Posted December 30, 2018 Author Share Posted December 30, 2018 After google searching my problem I might have an answer. I forgot, I just configured Jackett on my server a couple weeks ago. It's the only process that's new on my server. and there are comments that it causes this behavior here: https://github.com/Jackett/Jackett/issues/2198 Quote Link to comment
Rudder2 Posted January 1, 2019 Author Share Posted January 1, 2019 (edited) I'm playing with Dockers, CPU Pinning, and Hardware Trans-coding in Plex to band-aid the problem but I guess I just need to upgrade to a briefer system to do what I want. It's hard for me to believe that a 4790K couldn't handle this. It has for years. Seam like hen I upgraded to 6.6.x allot of things went wrong. Thank you for pointing out they load files. I now have learned to read my CPU load logs because of y'all! Thank you! I just wish they would do away with Mono for the Lidarr, Sonarr, Radarr, etc. and write them be able to natively run on Linux. Edited January 1, 2019 by Rudder2 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.