Unraid GUI and dockers locks up regularly (solved)

[gregtamaki]

Hello,

I have regular freezups of my Unraid machine where everything slows to a crawl (remote users on plex, any GUI from an installed docker, the UI, etc). I have been chasing this for a year - sometimes it seems to be ok, other times I make some sort of change and think that I have solved it but it always returns. These lockups happen either a few times an hour or go for days without issue.  Restarting all my dockers sometimes helps. A full reboot always brings  things back nice and responsive until it inevitably bogs down again.

 

-Upgraded to 6.9.2 last week

-Switched to NZBGet instead of sabnzb based on a comment in this thread where a user thought sabnzb was crippling their server.

-Plex stuff - I noticed that plex does occasionally get the dreaded SQLITE DB lockup in the logs occasionally, but I only have plex scan for library changes once a day. I made other recommended modifications to server settings which includes turning off scanning for video thumbnails, chapters, audio loudness, etc.

-Dashboard UI issue - there is a nginx error (similar to this) in the log sometimes when I navigate the UI and the pages fail to load that I haven't been able to solve. I think it might just be a symptom of the GUI not able to respond in time when everything is locked up. The lockups still happen without the UI being open on any computer.

-I have noticed high utilization for  "/usr/local/sbin/shfs /mnt/user -disk..." process when it's happening (and I can get the processes tab to load) similar to this issue

- My shares seem to be set up properly. App data only on my 1TB cache SSD. All other shares set to use cache.

- When the UI locks up, I can still usually use the terminal (but it's a bit slow to respond) and "docker stats" usually never has the sum of all running dockers over 40% CPU, so it seems like I am waiting on a slow I/O process or something.

- I previously posted about this and thought that it was Sonarr/Radarr causing the issue, but I can stop both and things will still lock up.

 

Does anything pop out in the logs?

 

Thanks in advance!

 

 

Edited May 14, 2021 by gregtamaki

[gregtamaki]

For anyone else struggling with something similar, I think I finally figured it out.

 

It seemed like when NZBDGet, SabNZBD, or Deluge were actively downloading/processing is when the system was locking up. Although CPU utilization was only getting to 40-60% at most, it seems like some I/O operations were seizing up the cache where all the apps were running. I added a new SSD into the system, mounted it as an unassigned device, then set that to hold my downloads (instead of downloading into shares located on the cache).

The difference is night and day - cpu utilization is at 0-10% while downloads are going, GUI isn't locking up, etc. I wish I could find the post where a random user mused this was the issue they were having so I could thank them

[ChatNoir]

12 hours ago, gregtamaki said:

I added a new SSD into the system, mounted it as an unassigned device, then set that to hold my downloads (instead of downloading into shares located on the cache).

I am curious, why use UD and not an additional pool ? I see you are using 6.9.2.

 

I wonder if the post you are referring to predates 6.9 and the multiple pools.

 

In any case, it is good that your issue is solved.

[gregtamaki]

7 hours ago, ChatNoir said:

I am curious, why use UD and not an additional pool ? I see you are using 6.9.2.

 

I wonder if the post you are referring to predates 6.9 and the multiple pools.

 

In any case, it is good that your issue is solved.

I didn't even realize that was an option! Yes, I shall add another pool and use that for my temp downloads. Should be cleaner and do the same thing.

[gregtamaki]

9 hours ago, ChatNoir said:

I am curious, why use UD and not an additional pool ? I see you are using 6.9.2.

 

I wonder if the post you are referring to predates 6.9 and the multiple pools.

 

In any case, it is good that your issue is solved.

Ha - seems to exhibit the original behavior even when I add a 2nd cache pool on a fresh drive. Really weird. I put it back to the UD and it works fine again!

[gregtamaki]

I was mistaken - I didn't configure Sonarr/Radarr to watch the proper download folders on the UD, so things were only half working. When I added the new Cache pool, Sonarr/Radarr *were* already pointing at the right spot, which is why that behavior continued. Pointing S/R at the UD downloads folder caused the behavior to continue.

 

I think this narrows down the culprits to be Sonarr/Radarr, though.