Jump to content

Out of Memory


Recommended Posts

Hi, I recently had an 'Out of Memory' error on my server. Any clues on what might have caused that? Here's the relevant log as well as the 'Diagnostics'. I suspect that it might have to do something with Home Assistant since it mentions it in the log but I wasn't able to find anything related to that in the Home Assistant log. Any help would be much appreciated.

 

Mar 25 22:51:20 unraid-server kernel: avahi-daemon invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
Mar 25 22:51:20 unraid-server kernel: CPU: 1 PID: 8857 Comm: avahi-daemon Not tainted 5.10.28-Unraid #1
Mar 25 22:51:20 unraid-server kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./B450 Gaming-ITX/ac, BIOS P3.50 08/02/2019
Mar 25 22:51:20 unraid-server kernel: Call Trace:
Mar 25 22:51:20 unraid-server kernel: dump_stack+0x6b/0x83
Mar 25 22:51:20 unraid-server kernel: dump_header+0x45/0x1e8
Mar 25 22:51:20 unraid-server kernel: oom_kill_process+0x7b/0xf6
Mar 25 22:51:20 unraid-server kernel: out_of_memory+0x3dd/0x410
Mar 25 22:51:20 unraid-server kernel: __alloc_pages_slowpath.constprop.0+0x665/0x74c
Mar 25 22:51:20 unraid-server kernel: __alloc_pages_nodemask+0x1a1/0x1fc
Mar 25 22:51:20 unraid-server kernel: __get_free_pages+0x8/0x35
Mar 25 22:51:20 unraid-server kernel: __pollwait+0x52/0xc1
Mar 25 22:51:20 unraid-server kernel: sock_poll_wait+0x18/0x1f
Mar 25 22:51:20 unraid-server kernel: unix_poll+0xa/0x7f
Mar 25 22:51:20 unraid-server kernel: sock_poll+0x81/0x8a
Mar 25 22:51:20 unraid-server kernel: do_sys_poll+0x221/0x441
Mar 25 22:51:20 unraid-server kernel: ? ip_cmsg_recv_offset+0xaf/0x2ef
Mar 25 22:51:20 unraid-server kernel: ? memcg_slab_free_hook+0x8a/0xfc
Mar 25 22:51:20 unraid-server kernel: ? udp_recvmsg+0x3ab/0x41c
Mar 25 22:51:20 unraid-server kernel: ? kmem_cache_free+0xa3/0x11d
Mar 25 22:51:20 unraid-server kernel: ? poll_initwait+0x33/0x33
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: ? get_compat_sigset_argpack.part.0+0x27/0x27
Mar 25 22:51:20 unraid-server kernel: __do_sys_poll+0x51/0xd4
Mar 25 22:51:20 unraid-server kernel: do_syscall_64+0x5d/0x6a
Mar 25 22:51:20 unraid-server kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Mar 25 22:51:20 unraid-server kernel: RIP: 0033:0x146058cf6dc3
Mar 25 22:51:20 unraid-server kernel: Code: 49 8b 45 10 5d 41 5c 41 5d 41 5e c3 66 2e 0f 1f 84 00 00 00 00 00 90 64 8b 04 25 18 00 00 00 85 c0 75 14 b8 07 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 55 c3 0f 1f 40 00 48 83 ec 28 89 54 24 1c 48
Mar 25 22:51:20 unraid-server kernel: RSP: 002b:00007ffe4a966758 EFLAGS: 00000246 ORIG_RAX: 0000000000000007
Mar 25 22:51:20 unraid-server kernel: RAX: ffffffffffffffda RBX: 0000000000428a80 RCX: 0000146058cf6dc3
Mar 25 22:51:20 unraid-server kernel: RDX: 0000000000001aac RSI: 000000000000000b RDI: 000000000044edd0
Mar 25 22:51:20 unraid-server kernel: RBP: 0000146058beab00 R08: 00000000e4b0dcfc R09: 00000000004b1a50
Mar 25 22:51:20 unraid-server kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000428b30
Mar 25 22:51:20 unraid-server kernel: R13: 000000000042c7d0 R14: 0000000000424a10 R15: 0000000000000000
Mar 25 22:51:20 unraid-server kernel: Mem-Info:
Mar 25 22:51:20 unraid-server kernel: active_anon:360650 inactive_anon:2980317 isolated_anon:0
Mar 25 22:51:20 unraid-server kernel: active_file:782 inactive_file:14456 isolated_file:123
Mar 25 22:51:20 unraid-server kernel: unevictable:16 dirty:180 writeback:16
Mar 25 22:51:20 unraid-server kernel: slab_reclaimable:62505 slab_unreclaimable:74357
Mar 25 22:51:20 unraid-server kernel: mapped:124257 shmem:309185 pagetables:16504 bounce:0
Mar 25 22:51:20 unraid-server kernel: free:33340 free_pcp:124 free_cma:0
Mar 25 22:51:20 unraid-server kernel: Node 0 active_anon:1442600kB inactive_anon:11921268kB active_file:3128kB inactive_file:57824kB unevictable:64kB isolated(anon):0kB isolated(file):492kB mapped:497028kB dirty:720kB writeback:64kB shmem:1236740kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 5781504kB writeback_tmp:0kB kernel_stack:27376kB all_unreclaimable? yes
Mar 25 22:51:20 unraid-server kernel: Node 0 DMA free:15900kB min:76kB low:92kB high:108kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15984kB managed:15900kB mlocked:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Mar 25 22:51:20 unraid-server kernel: lowmem_reserve[]: 0 1234 13766 13766
Mar 25 22:51:20 unraid-server kernel: Node 0 DMA32 free:56180kB min:6052kB low:7564kB high:9076kB reserved_highatomic:0KB active_anon:84232kB inactive_anon:1249536kB active_file:220kB inactive_file:6036kB unevictable:0kB writepending:16kB present:1496880kB managed:1479252kB mlocked:0kB pagetables:6348kB bounce:0kB free_pcp:4kB local_pcp:0kB free_cma:0kB
Mar 25 22:51:20 unraid-server kernel: lowmem_reserve[]: 0 0 12532 12532
Mar 25 22:51:20 unraid-server kernel: Node 0 Normal free:61280kB min:61452kB low:76812kB high:92172kB reserved_highatomic:0KB active_anon:1358368kB inactive_anon:10671732kB active_file:3076kB inactive_file:51332kB unevictable:64kB writepending:768kB present:13094144kB managed:12833168kB mlocked:64kB pagetables:59668kB bounce:0kB free_pcp:492kB local_pcp:0kB free_cma:0kB
Mar 25 22:51:20 unraid-server kernel: lowmem_reserve[]: 0 0 0 0
Mar 25 22:51:20 unraid-server kernel: Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15900kB
Mar 25 22:51:20 unraid-server kernel: Node 0 DMA32: 1691*4kB (UME) 1115*8kB (UE) 609*16kB (UME) 277*32kB (UME) 128*64kB (UME) 59*128kB (UE) 16*256kB (UM) 2*512kB (U) 1*1024kB (U) 0*2048kB 0*4096kB = 56180kB
Mar 25 22:51:20 unraid-server kernel: Node 0 Normal: 46*4kB (M) 2521*8kB (UME) 1470*16kB (UME) 544*32kB (UE) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 61280kB
Mar 25 22:51:20 unraid-server kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Mar 25 22:51:20 unraid-server kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Mar 25 22:51:20 unraid-server kernel: 324509 total pagecache pages
Mar 25 22:51:20 unraid-server kernel: 0 pages in swap cache
Mar 25 22:51:20 unraid-server kernel: Swap cache stats: add 0, delete 0, find 0/0
Mar 25 22:51:20 unraid-server kernel: Free swap  = 0kB
Mar 25 22:51:20 unraid-server kernel: Total swap = 0kB
Mar 25 22:51:20 unraid-server kernel: 3651752 pages RAM
Mar 25 22:51:20 unraid-server kernel: 0 pages HighMem/MovableOnly
Mar 25 22:51:20 unraid-server kernel: 69672 pages reserved
Mar 25 22:51:20 unraid-server kernel: 0 pages cma reserved
Mar 25 22:51:20 unraid-server kernel: Tasks state (memory values in pages):
Mar 25 22:51:20 unraid-server kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Mar 25 22:51:20 unraid-server kernel: [   1282]     0  1282     2480      962    53248        0         -1000 udevd
Mar 25 22:51:20 unraid-server kernel: [   1638]     0  1638     2054     1111    53248        0             0 haveged
Mar 25 22:51:20 unraid-server kernel: [   1724]     0  1724      720      520    40960        0             0 dhcpcd
Mar 25 22:51:20 unraid-server kernel: [   1807]    81  1807      865      570    45056        0             0 dbus-daemon
Mar 25 22:51:20 unraid-server kernel: [   1816]    32  1816     1245      536    45056        0             0 rpcbind
Mar 25 22:51:20 unraid-server kernel: [   1821]    32  1821     2232     1510    57344        0             0 rpc.statd
Mar 25 22:51:20 unraid-server kernel: [   1850]    44  1850    19321     1155    65536        0             0 ntpd
Mar 25 22:51:20 unraid-server kernel: [   1857]     0  1857      619       23    40960        0             0 acpid
Mar 25 22:51:20 unraid-server kernel: [   1871]     0  1871      634      455    45056        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [   1875]     0  1875      657      389    40960        0             0 atd
Mar 25 22:51:20 unraid-server kernel: [   8728]     0  8728      644      434    45056        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8729]     0  8729      644      456    45056        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8730]     0  8730      644      434    40960        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8731]     0  8731      644      438    40960        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8732]     0  8732      644      457    40960        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8733]     0  8733      644      446    40960        0             0 agetty
Mar 25 22:51:20 unraid-server kernel: [   8767]     0  8767      632      432    45056        0             0 inetd
Mar 25 22:51:20 unraid-server kernel: [   8769]     0  8769    87313     1548    94208        0             0 emhttpd
Mar 25 22:51:20 unraid-server kernel: [   8857]    61  8857     1635      886    49152        0             0 avahi-daemon
Mar 25 22:51:20 unraid-server kernel: [   8859]    61  8859     1457       64    49152        0             0 avahi-daemon
Mar 25 22:51:20 unraid-server kernel: [   8867]     0  8867     1180       28    45056        0             0 avahi-dnsconfd
Mar 25 22:51:20 unraid-server kernel: [   8884]     0  8884    25636     7060   184320        0             0 php
Mar 25 22:51:20 unraid-server kernel: [   8933]     0  8933    24910     3073   180224        0             0 php-fpm
Mar 25 22:51:20 unraid-server kernel: [   9016]     0  9016     2888     1335    57344        0             0 ttyd
Mar 25 22:51:20 unraid-server kernel: [   9019]     0  9019    37189      960    65536        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   9020]     0  9020    38186     2987    81920        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  10951]     0 10951      947      754    40960        0             0 diskload
Mar 25 22:51:20 unraid-server kernel: [  11092]     0 11092    36430      146    69632        0             0 shfs
Mar 25 22:51:20 unraid-server kernel: [  11103]     0 11103   307558    18212   389120        0             0 shfs
Mar 25 22:51:20 unraid-server kernel: [  11281]     0 11281    71365     1063    81920        0             0 rsyslogd
Mar 25 22:51:20 unraid-server kernel: [  11473]     0 11473    21851     4503   155648        0             0 smbd
Mar 25 22:51:20 unraid-server kernel: [  11476]     0 11476    22964     2610   155648        0             0 smbd-notifyd
Mar 25 22:51:20 unraid-server kernel: [  11477]     0 11477    21361     1629   147456        0             0 cleanupd
Mar 25 22:51:20 unraid-server kernel: [  11478]     0 11478    17011     2307   106496        0             0 nmbd
Mar 25 22:51:20 unraid-server kernel: [  11485]     0 11485      629       25    40960        0             0 wsdd
Mar 25 22:51:20 unraid-server kernel: [  11488]     0 11488    18868     2834   126976        0             0 winbindd
Mar 25 22:51:20 unraid-server kernel: [  11491]     0 11491    46063    30628   348160        0             0 winbindd
Mar 25 22:51:20 unraid-server kernel: [  12061]     0 12061    19045     2828   131072        0             0 winbindd
Mar 25 22:51:20 unraid-server kernel: [  12439]     0 12439   590440    23798   647168        0             0 dockerd
Mar 25 22:51:20 unraid-server kernel: [  12455]     0 12455   306243    11476   303104        0             0 containerd
Mar 25 22:51:20 unraid-server kernel: [  13831]     0 13831    11489     2441   126976        0             0 virtlockd
Mar 25 22:51:20 unraid-server kernel: [  13836]     0 13836    11540     2718   118784        0             0 virtlogd
Mar 25 22:51:20 unraid-server kernel: [  13850]     0 13850   333531     4702   290816        0             0 libvirtd
Mar 25 22:51:20 unraid-server kernel: [  13945]    99 13945      732       48    45056        0             0 dnsmasq
Mar 25 22:51:20 unraid-server kernel: [  13946]     0 13946      732       48    45056        0             0 dnsmasq
Mar 25 22:51:20 unraid-server kernel: [  14181]     0 14181  1205051  1123438  9871360        0             0 qemu-system-x86
Mar 25 22:51:20 unraid-server kernel: [   9615]     0  9615    18804     2833   131072        0             0 winbindd
Mar 25 22:51:20 unraid-server kernel: [  28539]     0 28539     1387      126    53248        0             0 mount.ntfs
Mar 25 22:51:20 unraid-server kernel: [   1654]     0  1654      968      625    45056        0             0 docker_load
Mar 25 22:51:20 unraid-server kernel: [   2542]     0  2542    25845     1362    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   2599]     0  2599    28345     3889    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   2625]     0  2625     5717     3541    81920        0             0 gunicorn
Mar 25 22:51:20 unraid-server kernel: [  18440]     0 18440    25845     1348    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  18478]     0 18478    28345     2639    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  18498]     0 18498       51        1    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  18592]     0 18592       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  19001]     0 19001       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  19003]    99 19003    39514     4735   208896        0             0 mono
Mar 25 22:51:20 unraid-server kernel: [  19048]    99 19048   618067    78802  1908736        0             0 mono-sgen
Mar 25 22:51:20 unraid-server kernel: [   7541]     0  7541    28345     3243    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   7563]     0  7563       51        1    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [   7684]     0  7684       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   7738]     0  7738    28345     3671    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   7762]     0  7762    19043     3659   188416        0             0 supervisord
Mar 25 22:51:20 unraid-server kernel: [   8163]     0  8163    28345     3742    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   8184]     0  8184    19100     3704   188416        0             0 supervisord
Mar 25 22:51:20 unraid-server kernel: [   8706]     0  8706       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   8707]     0  8707       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   8708]     0  8708       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   8709]     0  8709       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   8710]     0  8710       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   8712]     0  8712      935       58    45056        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [   8713]     0  8713      935       65    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [   8714]     0  8714      935       66    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [   8722]     0  8722    25845     2375    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   8745]     0  8745     1379       62    45056        0             0 cron
Mar 25 22:51:20 unraid-server kernel: [   8752]    33  8752     2659      274    57344        0             0 lighttpd
Mar 25 22:51:20 unraid-server kernel: [   8781]   999  8781   134390     2207   135168        0             0 pihole-FTL
Mar 25 22:51:20 unraid-server kernel: [   8889]     0  8889    28345     3777    98304        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   8905]    33  8905    49420     1664   139264        0             0 php-cgi
Mar 25 22:51:20 unraid-server kernel: [   8918]     0  8918       51        1    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [   9603]     0  9603    25845     1875    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   9621]     0  9621       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [   9873]     0  9873    28345     4017    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   9893]     0  9893      537       17    40960        0             0 dumb-init
Mar 25 22:51:20 unraid-server kernel: [  10281]     0 10281   316235    22019   417792        0             0 vaultwarden
Mar 25 22:51:20 unraid-server kernel: [  10899]    33 10899    49545     2346   135168        0             0 php-cgi
Mar 25 22:51:20 unraid-server kernel: [  10900]    33 10900    49544     2234   135168        0             0 php-cgi
Mar 25 22:51:20 unraid-server kernel: [  10901]    33 10901    49559     2202   135168        0             0 php-cgi
Mar 25 22:51:20 unraid-server kernel: [  10902]    33 10902    49549     2192   135168        0             0 php-cgi
Mar 25 22:51:20 unraid-server kernel: [  10960]     0 10960    25845     2381    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  11016]     0 11016    28345     3870    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  11038]     0 11038      654       16    40960        0             0 start.sh
Mar 25 22:51:20 unraid-server kernel: [  12712]     0 12712    25845     1858    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  12753]     0 12753    28345     3487    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  12774]     0 12774     6950     3755    90112        0             0 supervisord
Mar 25 22:51:20 unraid-server kernel: [  12995]   102 12995    99704     3517   258048        0             0 named
Mar 25 22:51:20 unraid-server kernel: [  13760]     0 13760    28345     3769    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  13791]    99 13791   545590    49231  1134592        0             0 urbackupsrv
Mar 25 22:51:20 unraid-server kernel: [  14058]     0 14058    26421     2667    86016        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  14292]     0 14292    28345     3714    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  14312]   991 14312    93235    48354   786432        0             0 python
Mar 25 22:51:20 unraid-server kernel: [  14382]     0 14382     5015       65    77824        0             0 startnginx.sh
Mar 25 22:51:20 unraid-server kernel: [  14384]     0 14384   167542     1179   290816        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14385]    33 14385   184391      947   405504        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14386]    33 14386   184391      947   405504        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14387]    33 14387   184391      947   405504        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14388]    33 14388   184391      947   405504        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14401]    33 14401   167584      536   282624        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  14721]     0 14721    25845     1348    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  14733]     0 14733    25845     1362    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  14874]     0 14874    28345     3801    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  14895]  1000 14895     9925     1671   110592        0             0 murmur.x86
Mar 25 22:51:20 unraid-server kernel: [  15293]     0 15293    28345     3355    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  15315]     0 15315   263794     3584   237568        0             0 python2
Mar 25 22:51:20 unraid-server kernel: [  15518]     0 15518       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  15519]     0 15519       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  15520]     0 15520       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  15522]     0 15522     1586      269    49152        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  15523]     0 15523      578       60    36864        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  15531]     0 15531      401       17    40960        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [  15680]     0 15680    25845     1858    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  15724]     0 15724    28345     3466    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  15743]   101 15743     7032     1347    90112        0             0 php
Mar 25 22:51:20 unraid-server kernel: [  15958]     0 15958    25845     2361    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  16140]     0 16140    28345     3376    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  16160]   999 16160   358433    26030   614400        0             0 mongod
Mar 25 22:51:20 unraid-server kernel: [  16644]     0 16644    28345     3816    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  16664]     0 16664       52        8    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  16938]     0 16938       53        6    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  16946]     0 16946       51        1    28672        0             0 s6-linux-init-s
Mar 25 22:51:20 unraid-server kernel: [  17010]     0 17010    25845      822    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  17021]     0 17021       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  17022]     0 17022       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  17049]     0 17049       47        1    28672        0             0 s6-ipcserverd
Mar 25 22:51:20 unraid-server kernel: [  17139]     0 17139    28361     3364    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  17158]   999 17158    53279     3204   151552        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  17826]     0 17826       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  17828]     0 17828       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  17829]     0 17829       53        6    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  17830]     0 17830      752      230    49152        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  17831]     0 17831      755      231    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  17832]     0 17832      808      283    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  17902]     0 17902      400       16    40960        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [  17974] 10050 17974   181071     6125   143360        0             0 zabbix_agent2
Mar 25 22:51:20 unraid-server kernel: [  18293]     0 18293    28345     3299    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  18313]     0 18313       52        6    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  18516]     0 18516       53        4    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  18538]     0 18538       51        1    28672        0             0 s6-linux-init-s
Mar 25 22:51:20 unraid-server kernel: [  18540]   999 18540    53359    35176   413696        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18541]   999 18541    53312    34816   409600        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18542]   999 18542    53279     1547   135168        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18543]   999 18543    53451      950   159744        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18544]   999 18544    17060      610   122880        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18545]   999 18545    53387      690   143360        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18576]     0 18576       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  18577]     0 18577       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  18594]     0 18594    25845     1324    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  18604]     0 18604       47        1    28672        0             0 s6-ipcserverd
Mar 25 22:51:20 unraid-server kernel: [  18929]     0 18929    28345     3283    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  18953]     0 18953       51        1    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  19376]     0 19376       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  19378]     0 19378       53        4    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  19379]     0 19379       53        6    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  19380]     0 19380      751      230    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  19381]     0 19381      753      232    45056        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  19382]     0 19382      809      286    45056        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  19452]     0 19452      400       16    40960        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [  19524] 10050 19524   181071     6093   143360        0             0 zabbix_agent2
Mar 25 22:51:20 unraid-server kernel: [  19676]     0 19676       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  20101]     0 20101    28345     3805    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  20123]     0 20123       52        6    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  20236]     0 20236       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  20238]     0 20238      580       56    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  20245]    99 20245      429       36    40960        0             0 mysqld_safe
Mar 25 22:51:20 unraid-server kernel: [  20495]    99 20495   175280   105365   962560        0             0 mariadbd
Mar 25 22:51:20 unraid-server kernel: [  20567]     0 20567       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  20572]     0 20572       51        1    28672        0             0 s6-linux-init-s
Mar 25 22:51:20 unraid-server kernel: [  20603]     0 20603       53        6    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  20604]     0 20604       53        4    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  20659]     0 20659       47        1    28672        0             0 s6-ipcserverd
Mar 25 22:51:20 unraid-server kernel: [  20889]     0 20889    25845      822    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  21364]     0 21364       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  21366]     0 21366       53        5    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  21367]     0 21367       53        6    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  21368]     0 21368      751      229    45056        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  21369]     0 21369      809      284    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  21370]     0 21370      753      232    40960        0             0 bash
Mar 25 22:51:20 unraid-server kernel: [  21438]     0 21438      400       15    40960        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [  21510] 10050 21510   181007     5968   139264        0             0 zabbix_agent2
Mar 25 22:51:20 unraid-server kernel: [  21587]     0 21587    28345     4105    98304        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  21607]  1001 21607    32192      509   122880        0             0 redis-server
Mar 25 22:51:20 unraid-server kernel: [  22442]    99 22442    60904    12841   258048        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  22443]    99 22443    26554     5358   122880        0             0 gunicorn
Mar 25 22:51:20 unraid-server kernel: [  22444]    99 22444    61097    12632   253952        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  22565]   999 22565    81480    38300   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22570]   999 22570    81498    38300   581632        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22574]   999 22574    81360    38204   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22575]   999 22575    81363    38257   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22578]   999 22578    81503    38286   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22580]   999 22580    81513    38281   581632        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22581]   999 22581    81504    38285   581632        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22582]   999 22582    81458    38294   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22585]   999 22585    81479    38274   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22586]   999 22586    81491    38307   577536        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  22657]    99 22657   256139    62314  1363968        0             0 gunicorn
Mar 25 22:51:20 unraid-server kernel: [  22660]    99 22660   323077    75400  1540096        0             0 gunicorn
Mar 25 22:51:20 unraid-server kernel: [  23399]    99 23399    61268    12765   249856        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  23406]    99 23406    61332    13041   241664        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  23407]    99 23407    61204    12666   245760        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  13003]     0 13003     7125     3629    90112        0             0 supervisord
Mar 25 22:51:20 unraid-server kernel: [  13119]     0 13119     1394       67    49152        0             0 cron
Mar 25 22:51:20 unraid-server kernel: [  13120]     0 13120      654       16    45056        0             0 apachectl
Mar 25 22:51:20 unraid-server kernel: [  13122]     0 13122    69421     2786   225280        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13151]    33 13151    71088     8273   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13152]    33 13152    71084     7954   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13153]    33 13153    71080     8044   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13154]    33 13154    71091     7932   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13155]    33 13155    71086     7938   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  13177]    33 13177    71079     7939   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [   9040]    33  9040    71072     8094   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  12009]    33 12009    71071     7861   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  16008]     0 16008    25845      837    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  16022]     0 16022    25845      837    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  16034]     0 16034    26437      902    86016        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [  16091]     0 16091    28345     2430    90112        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [  16114]     0 16114       51        1    28672        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [  16230]     0 16230       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  16754]     0 16754       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  16758]    99 16758  1174529   385277  3919872        0             0 jellyfin
Mar 25 22:51:20 unraid-server kernel: [   8605]     0  8605    26197      936    69632        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   8617]     0  8617    25845      937    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   8630]     0  8630    25845      936    65536        0             0 docker-proxy
Mar 25 22:51:20 unraid-server kernel: [   8691]     0  8691    28345     3472    94208        0             1 containerd-shim
Mar 25 22:51:20 unraid-server kernel: [   8721]     0  8721       51        1    32768        0             0 s6-svscan
Mar 25 22:51:20 unraid-server kernel: [   8938]     0  8938       51        1    32768        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  10452]     0 10452       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  10453]     0 10453       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  10454]     0 10454       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  10458]     0 10458       51        1    28672        0             0 s6-supervise
Mar 25 22:51:20 unraid-server kernel: [  10463]     0 10463     7922     3025   102400        0             0 fail2ban-server
Mar 25 22:51:20 unraid-server kernel: [  10464]     0 10464      401       16    36864        0             0 crond
Mar 25 22:51:20 unraid-server kernel: [  10465]     0 10465    45448     1941    94208        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [  25754]     0 25754  2876677    41388  4640768        0             0 unraid-api
Mar 25 22:51:20 unraid-server kernel: [  25792]     0 25792     1245      947    40960        0             0 rc.flash_backup
Mar 25 22:51:20 unraid-server kernel: [  25962]     0 25962    37467     8463   167936        0             0 docker
Mar 25 22:51:20 unraid-server kernel: [   7933]    33  7933    71093     7928   286720        0             0 apache2
Mar 25 22:51:20 unraid-server kernel: [  17652]     0 17652    28518    23134   266240        0             0 gunicorn
Mar 25 22:51:20 unraid-server kernel: [   3185]    99  3185     1679      360    49152        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3186]    99  3186     1715      375    49152        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3187]    99  3187     1679      359    49152        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3188]    99  3188     1679      359    49152        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3190]     0  3190   557116     3032   204800        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3293]    99  3293   572563    12962   372736        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3294]    99  3294   572584    13003   380928        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3295]    99  3295   572541    12794   376832        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3296]    99  3296   572555    12799   376832        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3297]    99  3297   572560    12949   372736        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3298]    99  3298   572557    12821   372736        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   3517]    99  3517    45767     4533   110592        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3518]    99  3518    45663     4406   110592        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3519]    99  3519    45629     4357   110592        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3520]    99  3520    45629     3060   110592        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3521]    99  3521    45510     1785    90112        0             0 nginx
Mar 25 22:51:20 unraid-server kernel: [   3611]     0  3611    59959     2092   159744        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   2540]     0  2540    19314      494    61440        0             0 apcupsd
Mar 25 22:51:20 unraid-server kernel: [   1137]    99  1137    66734    25458   368640        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [  28160]    99 28160    68310    27604   376832        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [  28194]    99 28194    66766    25547   368640        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [  27551]     0 27551    22744     4209   159744        0             0 smbd
Mar 25 22:51:20 unraid-server kernel: [  27392]     0 27392     5015       65    77824        0             0 heartbeat.sh
Mar 25 22:51:20 unraid-server kernel: [  27395]     0 27395     1547       15    53248        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: [  31740]     0 31740      396        7    36864        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: [   4954]     0  4954      396        7    36864        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: [   8474]    99  8474   558891    10861   278528        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8496]    99  8496   558891    10759   278528        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8497]    99  8497   558894    10642   282624        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8502]    99  8502   558894    10578   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8503]    99  8503   558893    10396   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8504]    99  8504   558894    10595   278528        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8505]    99  8505   558897    10995   278528        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8527]    99  8527   558900    10691   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8528]    99  8528   558890    10633   278528        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   8529]    99  8529   558893    10841   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   5516]    99  5516   558885    10564   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   5519]    99  5519   558879    10164   274432        0             0 php-fpm7
Mar 25 22:51:20 unraid-server kernel: [   5858]    99  5858   584360   303323  3305472        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  13118]     0 13118      396        5    45056        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: [  16359]    99 16359   584779   301658  3309568        0             0 python3
Mar 25 22:51:20 unraid-server kernel: [  18852]   999 18852    53557     2564   196608        0             0 postgres
Mar 25 22:51:20 unraid-server kernel: [  18880]     0 18880    37467     8173   167936        0             0 docker
Mar 25 22:51:20 unraid-server kernel: [  19054]     0 19054      613      189    40960        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: [  19201]     0 19201      940      739    40960        0             0 sh
Mar 25 22:51:20 unraid-server kernel: [  19203]     0 19203    25620     6724   180224        0             0 php
Mar 25 22:51:20 unraid-server kernel: [  19208]     0 19208      940      695    40960        0             0 sh
Mar 25 22:51:20 unraid-server kernel: [  19209]     0 19209    37467     7736   172032        0             0 docker
Mar 25 22:51:20 unraid-server kernel: [  19247]     0 19247      613      189    40960        0             0 sleep
Mar 25 22:51:20 unraid-server kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/machine/qemu-1-HomeAssistant.libvirt-qemu,task=qemu-system-x86,pid=14181,uid=0
Mar 25 22:51:20 unraid-server kernel: Out of memory: Killed process 14181 (qemu-system-x86) total-vm:4820204kB, anon-rss:4470748kB, file-rss:24kB, shmem-rss:22980kB, UID:0 pgtables:9640kB oom_score_adj:0
Mar 25 22:51:20 unraid-server kernel: oom_reaper: reaped process 14181 (qemu-system-x86), now anon-rss:16kB, file-rss:20kB, shmem-rss:0kB
Mar 25 22:51:20 unraid-server kernel: br0: port 2(vnet0) entered disabled state
Mar 25 22:51:20 unraid-server kernel: device vnet0 left promiscuous mode
Mar 25 22:51:20 unraid-server kernel: br0: port 2(vnet0) entered disabled state

 

unraid-server-diagnostics-20220403-1931.zip

Link to comment

You mention Home Assistant.  One thing you have to realize is that Unraid is actually installed in Memory---  Not on a physical device.  Any time a docker or plugin has to write a file, you have to make sure that it writes that file to a physical device.  If it just writes it to the OS , it will end up on a RAM disk.  IF you do have a Docker or Plugin that is storing data in RAM, it is possible that the operation is the source of your OOM problem.

 

As I recall, Unraid mounts all of the physical devices at   /mnt  directory in the file structure.  It is important to realize that when a file's  path points to the /mnt folder, it must also include a mounted physical device (For example; disk1, cache, or user) as a part of the path.  Another important thing is that Linux is Case sensitive (whereas, Windows is not).  As an example,  Test and test are two different (and unique) resources to Linux but only one to Windows.  (A path of  /mnt/disk1   will write the file on a physical device.  But a path of /mnt/Disk1   will write the file to RAM.)

Link to comment

Thanks for that hint. Unfortunately, I don't think it causes the problem. I just had another look at the log I posted above and (if I read it right) this line

Mar 25 22:51:20 unraid-server kernel: [  14181]     0 14181  1205051  1123438  9871360        0             0 qemu-system-x86

says that qemu used a lot of memory at this point. And qemu is a part of the virtualisation system on unraid as far as I understand. So maybe I do need to have a closer look at Home Assistant, since it is the only VM I run on my server. Please correct me, if you think different.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...