A random scan threw this error (Fix Common Problems)

unraidzfs-diagnostics-20250305-0829.zip

If your hitting out of memory. You need to buy more ram and increase the ram in the machine.

Otherwise, to help alienate:

Swap file so when your out of ram it writes the ram contents to disk...
 

CA install a swap file to assist with low memory issues

requires a btfs formatted disk. Install the CA swap plugin.

General. as unraid sits in ram. clear logs.... Limit memory that a resource is able to have, example the docker in question:
docker extra parm limit ram it gets.
 

Setting Memory Limit in Unraid Docker:
-Open the Unraid Web UI.
-Navigate to Docker > Click on the container you want to modify.
-Click on Edit.
-Scroll down to the Extra Parameters field (you may need to enable advanced view by clicking the toggle at the top-right).
-Add the following parameter to set a memory limit:

 

example:

--memory=1g

-m 1g

- Click Apply and Restart the container.

 

Explanation of the Flag:
--memory=1g (or -m 1g): Limits the container's memory usage to 1GB.
You can also specify other values like:
512m for 512MB
2g for 2GB

 

Docker container is already restricted to 16g.
Also, have plenty of memory available out of total 64gb installed.

 

6 hours ago, sabertooth said:

Docker container is already restricted to 16g.
Also, have plenty of memory available out of total 64gb installed.

 

 

Whatever container was running that java instance ran the host OOM, if you were doing something at the time, then you can narrow it down - else you can run ps auxf from the CLI to figure out which container is kicking it off

 

Mar  4 19:12:18 UnraidZFS kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Mar  4 19:12:18 UnraidZFS kernel: [1130142]  1001 1130142     1843      898    53248        0             0 bash
Mar  4 19:12:18 UnraidZFS kernel: [1130845]  1001 1130845      723      389    40960        0             0 sh
Mar  4 19:12:18 UnraidZFS kernel: [1131059]  1001 1131059  5665517  4073503 33357824        0             0 java
Mar  4 19:12:18 UnraidZFS kernel: [1131387]  1001 1131387     1451      449    45056        0             0 tail
Mar  4 19:12:18 UnraidZFS kernel: [1157589]  1001 1157589      723      389    40960        0             0 sh
Mar  4 19:12:18 UnraidZFS kernel: [1270535]  1001 1270535      723      390    40960        0             0 sh
Mar  4 19:12:18 UnraidZFS kernel: [1270541]  1001 1270541      723      389    40960        0             0 ldapsearch
Mar  4 19:12:18 UnraidZFS kernel: [1270543]  1001 1270543      723      387    45056        0             0 _client-script.
Mar  4 19:12:18 UnraidZFS kernel: [1270576]  1001 1270576  4994726   122374  1310720        0             0 java
Mar  4 19:12:18 UnraidZFS kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=a36642b942ed2e5ad6e110c86d428bebeaaf36997e6a55d45fab3878a001182e,mems_allowed=0,oom_memcg=/docker/a36642b942ed2e5ad6e110c86d428bebeaaf36997e6a55d45fab3878a001182e,task_memcg=/docker/a36642b942ed2e5ad6e110c86d428bebeaaf36997e6a55d45fab3878a001182e,task=java,pid=1131059,uid=1001
Mar  4 19:12:18 UnraidZFS kernel: Memory cgroup out of memory: Killed process 1131059 (java) total-vm:22662068kB, anon-rss:16269568kB, file-rss:24444kB, shmem-rss:0kB, UID:1001 pgtables:32576kB oom_score_adj:0

 

Edited March 5, 20251 yr by Michael_P

your clearly using zfs which is equivalent to 1 gb of ram to 1 TB of data. *General Rule of Thumb on ZFS pools...

Most likely a some form of Database in the docker writing large amounts of data to the zfs pool of disks that caused a spike in zfs ram usage and the docker database required similar ram resource demanding.

 

Regardless if the limit was set to 16 gb of ram to the docker(LOWER IT!). You hit OOM which means you didn't have enouth resources to preform that operation.

as this seems more zfs related, I highly recommend running a swap file or lookinto alcoteing ram and making a zil/zram disk to allocate physical ram as a swap file to assist with issues that can lead into OOM before it happens...

You may need to lower zfs arc memory.

 

Unraid web UI > system drivers > zfs

 

you may need to lower the number to decrease zfs arch cahce and ram usage...

13 hours ago, bmartino1 said:

your clearly using zfs which is equivalent to 1 gb of ram to 1 TB of data

 

By default it only uses 1/8th of system RAM

 

13 hours ago, bmartino1 said:

Regardless if the limit was set to 16 gb of ram to the docker(LOWER IT!)

Docker treats limits as a suggestion, you have to add extra flags to force it to respect RAM limits (which has the downside of possibly crashing the host)

 

In this case, that java process ran away with all the available RAM (much higher than the set 16GB Limit) and the OOM reaper killed it accordingly