Slow server with 100% usage and massive lsof process usage

[DrSpaldo]

1 hour ago, bucketphobia said:

Hey @DrSpaldo, I was having a very similar issue, but slightly different - the cause of mine was a dockerd log process.

 

The solution to mine was to ensure that "Permit exclusive shares" is on, and that "Exclusive Access" is indeed active on my cache pool.

 

Hope this helps.

Thanks @bucketphobia, I will also turn that on. Fortunately mine has died down heaps since I made the above other changes, but who knows what the actual root cause is. Seems to be so many people in similar (but not exactly) the same situation

[DrSpaldo]

@bucketphobiaI've turned on the permit exclusing shares, however, I don't see how to turn on the exclusive access part anywhere. 

 

Although, I do remember that I did change the default for my dockers to point directly to the drive and not use fuse... /mnt/cachetwo/appdata/opera rather than /mnt/user/appdata/opera

[itimpi]

1 hour ago, DrSpaldo said:

@bucketphobiaI've turned on the permit exclusing shares, however, I don't see how to turn on the exclusive access part anywhere. 

 

Although, I do remember that I did change the default for my dockers to point directly to the drive and not use fuse... /mnt/cachetwo/appdata/opera rather than /mnt/user/appdata/opera

Once you have enabled the global option it should be automatic if a share is set up to have all its files to be on a single pool/drive and that is how they are actually currently placed   If it is not enabling then you have probably not met that criteria.

[bucketphobia]

10 hours ago, DrSpaldo said:

@bucketphobiaI've turned on the permit exclusing shares, however, I don't see how to turn on the exclusive access part anywhere. 

 

Although, I do remember that I did change the default for my dockers to point directly to the drive and not use fuse... /mnt/cachetwo/appdata/opera rather than /mnt/user/appdata/opera

 

If it says no - you likely need to (1): disable secondary storage; (2): make sure that nothing on the share is on other drives

[DrSpaldo]

No worries, thanks @bucketphobia@itimpi

[MrK]

Do you think the issues have gone after removing frigate from the server?

When playing about with frigate (2 cams go2rtc streams and HW encoding enabled intel with coral). Only seems to happen when I am messing about with settings trying to get go2rtc in to home assistant etc.

Nov  5 13:00:58 Tower kernel: ffmpeg invoked oom-killer: gfp_mask=0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_COMP|__GFP_ZERO), order=0, oom_score_adj=0
Nov  5 13:00:58 Tower kernel: CPU: 3 PID: 7505 Comm: ffmpeg Tainted: P           O       6.1.49-Unraid #1

 

It says invoking oom-killer but I guess it doesn't work?

 

It makes the server unreachable. I have to power cycle it.

 

I have added the "--memory=3G" line to the frigate docker and installed the swap file plugin.

[DrSpaldo]

4 hours ago, MrK said:

Do you think the issues have gone after removing frigate from the server?

When playing about with frigate (2 cams go2rtc streams and HW encoding enabled intel with coral). Only seems to happen when I am messing about with settings trying to get go2rtc in to home assistant etc.

Nov  5 13:00:58 Tower kernel: ffmpeg invoked oom-killer: gfp_mask=0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_COMP|__GFP_ZERO), order=0, oom_score_adj=0
Nov  5 13:00:58 Tower kernel: CPU: 3 PID: 7505 Comm: ffmpeg Tainted: P           O       6.1.49-Unraid #1

 

It says invoking oom-killer but I guess it doesn't work?

 

It makes the server unreachable. I have to power cycle it.

 

I have added the "--memory=3G" line to the frigate docker and installed the swap file plugin.

 

Hi @MrK, yes, I do think that was either the primary cause or a big part of it. Although I have no concrete proof. 

 

I know originally when I was deep into the issue, I found out that the --memory line doesn't always work with Unraid. I did a quick search earlier and didn't find the exact post where I remember seeing it though... Here are a few to look through that may help - Unraid OOM Issues · blakeblackshear/frigate

 

Edit: check this post to see if you are using the fuse file system for your Unraid Frigate container - FUSE post - Frigate Github

Edited November 6, 2023 by DrSpaldo