Please help; I keep getting this error on my my server after upgrading to 6.2.8

Your server has run out of memory, and processes (potentially required) are being killed off. You should post your diagnostics and ask for assistance on the

Server Configuration:

Memory: 160 GiB DDR4 Single-bit ECC

Intel Corporation S2600STB , Version J17012-551
Intel Corporation, Version SE5C620.86B.02.01.0016.032120230338
BIOS dated: Tue 21 Mar 2023 12:00:00 AM EDT
Dual CPU Intel® Xeon® Silver 4114 CPU @ 2.20GHz

unraidserver-diagnostics-20240312-0843.zip

Killed process 38055 (ffmpeg)

 

This was using a lot of RAM, so whatever container uses it may be the problem, also make sure you change your docker network to ipvlan.

I'm using MACVLAN all of my Dockers are on a VLAN on my local network (can I use ipvlan) and assign local IP to the dockers?)

I'm using Frigate and Immich can one of this be the problem 
All was working fine before 6.12.8 

You should be able to use ipvlan, but if you really need macvlan you can use, just need to disable bridging first, see the 6.12.4 release notes.

Thank you for the reply,
I tried with IPVLAN and MACVLAN but the issue still happening with IPVLAN, something is causing the CPU to overload 
this wasn't an issue until I upgrade to 6.2.8. 

I have this server running for over a year and Frigate using GPU for the last 6 months, been having this issue for the last week or so 
Note: I also installed coral TPU to free-up the GPU that was 3 days ago. (issue was happening before TPU installation)

13 minutes ago, LugoCloud said:

I tried with IPVLAN and MACVLAN but the issue still happening with IPVLAN

What issue? The OOM is unrelated to that.

2 hours ago, LugoCloud said:

I'm using Frigate and Immich can one of this be the problem

 

Here's the container that was running the process, see if any part of the string matches either of the container IDs (advanced view on the docker tab) - Frigate is the most likely as there were quite a few processes running at the time (via coral/cuda-EvtHandlr), and on the 10th it was Frigate capture that invoked the reaper (each time ffmpeg ran away with the RAM)

 

Most likely Frigate:
42895499d47e566044b88aaad34d27527654e862a99663365824e302465beb4d

Whatever you're using for GPU offload:
130189b80344b2fc2eb03b176497c5eba0aa27c18268a83afec3ba62771debc9

 

 

This are my Frigate and Immich

Frigate Container ID: 178a51d9c2a2
Immich Container ID: 0166ed80c64a
this is docker stats it looks like Frigate is is the issue 
Thank you for the information, need todo more digging, any suggestions will be greatly appreciated... 

 

21 minutes ago, LugoCloud said:

any suggestions will be greatly appreciated

 

Make sure it's not configured to transcode to a directory that isn't mapped to a drive

Pardon my ignorance, but I don't understand 
Thank you

this is my Docker config 
docker run
  -d
  --name='frigate'
  --net='br0.50' (VLAN)
  --ip='172.20.50.253' (STATIC IP)
  --privileged=true
  -e TZ="America/New_York"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="uNRAIDServer"
  -e HOST_CONTAINERNAME="frigate"
  -e 'TCP_PORT_5000'='5000'
  -e 'TCP_PORT_8554'='8554'
  -e 'FRIGATE_RTSP_PASSWORD'='******************'
  -e 'PLUS_API_KEY'='***********************'
  -e 'NVIDIA_VISIBLE_DEVICES'='GPU-8e704c73-6f19-b95f-12ac-a907b26a2221'
  -e 'NVIDIA_DRIVER_CAPABILITIES'='compute,utility,video'
  -e 'TCP_PORT_8555'='8555'
  -e 'UDP_PORT_8555'='8555'
  -e 'TCP_PORT_1984'='1984'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:5000]'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/yayitazale/unraid-templates/main/frigate.png'
  -v '/mnt/user/appdata/frigate/config/':'/config':'rw' (this is map to SSD Pool)
  -v '/mnt/user/CCTVStorage/frigate/':'/media/frigate':'rw' (this is map to a single drive Pool) 
  -v '/etc/localtime':'/etc/localtime':'rw'
  --device='/dev/apex_0' (M.2 TPU1)
  --device='/dev/apex_1' (M.2 TPU2)
  --mac-address 02:42:ac:14:32:53 (STATIC MAC only change the last 2 digits) 
  --runtime=nvidia
  --shm-size=256mb
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=100000000
  --restart unless-stopped 'ghcr.io/blakeblackshear/frigate:stable'

c8c50231d4c01b6c123b1f7928ddcef5f481a20c9637b06a28bb6f96d1aed8a5

16 hours ago, LugoCloud said:

Pardon my ignorance, but I don't understand 

 

No worries

 

I'm not familiar with Frigate so I can't tell you where exactly to go, but in its settings or config file there should be storage folders and working folders, make sure that wherever ffmpeg is set to place its work files (which briefly looking at the docs appears it may be /tmp/cache) is either limited or mapped to a disk folder

I check on unraid the directory /tmp/ and don't see a cache directory 

20 minutes ago, LugoCloud said:

I check on unraid the directory /tmp/ and don't see a cache directory 

Since you have not mapped it to a location outside the container this path is only internal to the container.

Now I have a bigger problem 

I decided to rebuilt the USB so I formated and created a new USB and the server is not booting up Getting error 110 

IMG_3326.HEIC

4 minutes ago, LugoCloud said:

Getting error 110 

 

try another port, or retry creating the USB

Done that 

Now after creating the USB agin it just hang on bzroot ok

I also tried to boot on savemode and it pass the bzroot ok and it hangs on bzroot-gui ok its been there for 10 minutes or so 

Tried a different USB now but is a very old (that worked) 
DOn't have any other that I can use at this time 
what USB model you suggest for me  to get I can order Amazone 
Thank you  

16 minutes ago, LugoCloud said:

Tried a different USB now but is a very old (that worked) 
DOn't have any other that I can use at this time 
what USB model you suggest for me  to get I can order Amazone 
Thank you  

 

Cant help you there, but I'd suggest a reputable brand from a reputable seller - there's a lot of shadiness in the world of flash drives. From what I remember, a 16/32GB USB 2.0 device is what's usually what's recommended but don't quote me on that  

How can I get new key for my USB or should I wait until I get a good quality USB

52 minutes ago, LugoCloud said:

How can I get new key for my USB or should I wait until I get a good quality USB

 

https://docs.unraid.net/unraid-os/manual/changing-the-flash-device/