LugoCloud Posted March 12 Share Posted March 12 Please help; I keep getting this error on my my server after upgrading to 6.2.8 Your server has run out of memory, and processes (potentially required) are being killed off. You should post your diagnostics and ask for assistance on the Server Configuration: Memory: 160 GiB DDR4 Single-bit ECC Intel Corporation S2600STB , Version J17012-551 Intel Corporation, Version SE5C620.86B.02.01.0016.032120230338 BIOS dated: Tue 21 Mar 2023 12:00:00 AM EDT Dual CPU Intel® Xeon® Silver 4114 CPU @ 2.20GHz unraidserver-diagnostics-20240312-0843.zip Quote Link to comment
JorgeB Posted March 12 Share Posted March 12 Killed process 38055 (ffmpeg) This was using a lot of RAM, so whatever container uses it may be the problem, also make sure you change your docker network to ipvlan. Quote Link to comment
LugoCloud Posted March 12 Author Share Posted March 12 I'm using MACVLAN all of my Dockers are on a VLAN on my local network (can I use ipvlan) and assign local IP to the dockers?) I'm using Frigate and Immich can one of this be the problem All was working fine before 6.12.8 Quote Link to comment
JorgeB Posted March 12 Share Posted March 12 You should be able to use ipvlan, but if you really need macvlan you can use, just need to disable bridging first, see the 6.12.4 release notes. Quote Link to comment
LugoCloud Posted March 12 Author Share Posted March 12 Thank you for the reply, I tried with IPVLAN and MACVLAN but the issue still happening with IPVLAN, something is causing the CPU to overload this wasn't an issue until I upgrade to 6.2.8. I have this server running for over a year and Frigate using GPU for the last 6 months, been having this issue for the last week or so Note: I also installed coral TPU to free-up the GPU that was 3 days ago. (issue was happening before TPU installation) Quote Link to comment
JorgeB Posted March 12 Share Posted March 12 13 minutes ago, LugoCloud said: I tried with IPVLAN and MACVLAN but the issue still happening with IPVLAN What issue? The OOM is unrelated to that. Quote Link to comment
Michael_P Posted March 12 Share Posted March 12 2 hours ago, LugoCloud said: I'm using Frigate and Immich can one of this be the problem Here's the container that was running the process, see if any part of the string matches either of the container IDs (advanced view on the docker tab) - Frigate is the most likely as there were quite a few processes running at the time (via coral/cuda-EvtHandlr), and on the 10th it was Frigate capture that invoked the reaper (each time ffmpeg ran away with the RAM) Most likely Frigate: 42895499d47e566044b88aaad34d27527654e862a99663365824e302465beb4d Whatever you're using for GPU offload: 130189b80344b2fc2eb03b176497c5eba0aa27c18268a83afec3ba62771debc9 Quote Link to comment
LugoCloud Posted March 12 Author Share Posted March 12 This are my Frigate and Immich Frigate Container ID: 178a51d9c2a2 Immich Container ID: 0166ed80c64a this is docker stats it looks like Frigate is is the issue Thank you for the information, need todo more digging, any suggestions will be greatly appreciated... Quote Link to comment
Michael_P Posted March 12 Share Posted March 12 21 minutes ago, LugoCloud said: any suggestions will be greatly appreciated Make sure it's not configured to transcode to a directory that isn't mapped to a drive Quote Link to comment
LugoCloud Posted March 12 Author Share Posted March 12 Pardon my ignorance, but I don't understand Thank you this is my Docker config docker run -d --name='frigate' --net='br0.50' (VLAN) --ip='172.20.50.253' (STATIC IP) --privileged=true -e TZ="America/New_York" -e HOST_OS="Unraid" -e HOST_HOSTNAME="uNRAIDServer" -e HOST_CONTAINERNAME="frigate" -e 'TCP_PORT_5000'='5000' -e 'TCP_PORT_8554'='8554' -e 'FRIGATE_RTSP_PASSWORD'='******************' -e 'PLUS_API_KEY'='***********************' -e 'NVIDIA_VISIBLE_DEVICES'='GPU-8e704c73-6f19-b95f-12ac-a907b26a2221' -e 'NVIDIA_DRIVER_CAPABILITIES'='compute,utility,video' -e 'TCP_PORT_8555'='8555' -e 'UDP_PORT_8555'='8555' -e 'TCP_PORT_1984'='1984' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:5000]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/yayitazale/unraid-templates/main/frigate.png' -v '/mnt/user/appdata/frigate/config/':'/config':'rw' (this is map to SSD Pool) -v '/mnt/user/CCTVStorage/frigate/':'/media/frigate':'rw' (this is map to a single drive Pool) -v '/etc/localtime':'/etc/localtime':'rw' --device='/dev/apex_0' (M.2 TPU1) --device='/dev/apex_1' (M.2 TPU2) --mac-address 02:42:ac:14:32:53 (STATIC MAC only change the last 2 digits) --runtime=nvidia --shm-size=256mb --mount type=tmpfs,target=/tmp/cache,tmpfs-size=100000000 --restart unless-stopped 'ghcr.io/blakeblackshear/frigate:stable' c8c50231d4c01b6c123b1f7928ddcef5f481a20c9637b06a28bb6f96d1aed8a5 Quote Link to comment
Michael_P Posted March 13 Share Posted March 13 16 hours ago, LugoCloud said: Pardon my ignorance, but I don't understand No worries I'm not familiar with Frigate so I can't tell you where exactly to go, but in its settings or config file there should be storage folders and working folders, make sure that wherever ffmpeg is set to place its work files (which briefly looking at the docs appears it may be /tmp/cache) is either limited or mapped to a disk folder Quote Link to comment
LugoCloud Posted March 13 Author Share Posted March 13 I check on unraid the directory /tmp/ and don't see a cache directory Quote Link to comment
itimpi Posted March 13 Share Posted March 13 20 minutes ago, LugoCloud said: I check on unraid the directory /tmp/ and don't see a cache directory Since you have not mapped it to a location outside the container this path is only internal to the container. Quote Link to comment
LugoCloud Posted March 14 Author Share Posted March 14 Now I have a bigger problem I decided to rebuilt the USB so I formated and created a new USB and the server is not booting up Getting error 110 IMG_3326.HEIC Quote Link to comment
Michael_P Posted March 14 Share Posted March 14 4 minutes ago, LugoCloud said: Getting error 110 try another port, or retry creating the USB Quote Link to comment
LugoCloud Posted March 14 Author Share Posted March 14 Done that Now after creating the USB agin it just hang on bzroot ok I also tried to boot on savemode and it pass the bzroot ok and it hangs on bzroot-gui ok its been there for 10 minutes or so Quote Link to comment
LugoCloud Posted March 14 Author Share Posted March 14 Tried a different USB now but is a very old (that worked) DOn't have any other that I can use at this time what USB model you suggest for me to get I can order Amazone Thank you Quote Link to comment
Michael_P Posted March 14 Share Posted March 14 16 minutes ago, LugoCloud said: Tried a different USB now but is a very old (that worked) DOn't have any other that I can use at this time what USB model you suggest for me to get I can order Amazone Thank you Cant help you there, but I'd suggest a reputable brand from a reputable seller - there's a lot of shadiness in the world of flash drives. From what I remember, a 16/32GB USB 2.0 device is what's usually what's recommended but don't quote me on that 1 Quote Link to comment
LugoCloud Posted March 14 Author Share Posted March 14 How can I get new key for my USB or should I wait until I get a good quality USB Quote Link to comment
Solution Michael_P Posted March 14 Solution Share Posted March 14 52 minutes ago, LugoCloud said: How can I get new key for my USB or should I wait until I get a good quality USB https://docs.unraid.net/unraid-os/manual/changing-the-flash-device/ Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.