Good evening everyone! Loooooooonnngg time unraid user (10 years?), currently on 6.10.3 after uneventful updates from prior releases (6.10.2) about a week ago. Current uptime in the log is over 7 days. I first noticed these entries about 10 days ago, prior to 6.10.3 update being applied, hoped 6.10.3 would address this... It's good to have a dream!
Jun 28 00:51:28 MAXTOWER nginx: 2022/06/28 00:51:28 [error] 5362#5362: *5697534 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update2?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:28 MAXTOWER nginx: 2022/06/28 00:51:28 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update2
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 612. Increase nchan_max_reserved_memory.
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: *5697535 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/cpuload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /cpuload
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 934. Increase nchan_max_reserved_memory.
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: *5697538 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update3?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update3
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 503. Increase nchan_max_reserved_memory.
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: *5697539 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:29 MAXTOWER nginx: 2022/06/28 00:51:29 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /devices
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 612. Increase nchan_max_reserved_memory.
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: *5697541 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/cpuload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /cpuload
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 521. Increase nchan_max_reserved_memory.
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: *5697542 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/diskload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /diskload
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 902. Increase nchan_max_reserved_memory.
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: *5697548 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update3?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update3
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 503. Increase nchan_max_reserved_memory.
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: *5697549 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:30 MAXTOWER nginx: 2022/06/28 00:51:30 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /devices
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 612. Increase nchan_max_reserved_memory.
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: *5697551 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/cpuload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /cpuload
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 933. Increase nchan_max_reserved_memory.
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: *5697555 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update3?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update3
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 278. Increase nchan_max_reserved_memory.
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: *5697556 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update1?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update1
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 503. Increase nchan_max_reserved_memory.
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: *5697557 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:31 MAXTOWER nginx: 2022/06/28 00:51:31 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /devices
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 613. Increase nchan_max_reserved_memory.
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: *5697559 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/cpuload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /cpuload
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 521. Increase nchan_max_reserved_memory.
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: *5697560 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/diskload?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /diskload
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 934. Increase nchan_max_reserved_memory.
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: *5697566 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/update3?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /update3
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 470. Increase nchan_max_reserved_memory.
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: *5697567 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/apcups?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /apcups
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [crit] 5362#5362: ngx_slab_alloc() failed: no memory
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: shpool alloc failed
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: nchan: Out of shared memory while allocating message of size 503. Increase nchan_max_reserved_memory.
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: *5697568 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/devices?buffer_length=1 HTTP/1.1", host: "localhost"
Jun 28 00:51:32 MAXTOWER nginx: 2022/06/28 00:51:32 [error] 5362#5362: MEMSTORE:00: can't create shared message for channel /devices
First guess (about 10 days ago) was some type of DRAM error, but a full 72 hour burn-in with MemTest passed (about a week ago) with no errors, reducing suspicion of a hardware error. I did not notice anything else untoward (I wasn't using the box when these were logged however).
I found these errors again when checking the logs (via the UI) trying to track down why the drop-count on my main NIC is steadily increasing (no errors, just drops). (Will address that in a separate thread, just looking for any hints on the nginix errors here).
I have been leaving the UI (on the main "Dashboard") tab for the past few days while I have been attempting to track down the reason for the ethernet drops being reported, which is a change in the usual usage (it's headless and just does it's thing, only rarely needing to be checked in on for updates etc.) Shorter version: I do not normally leave the web UI loaded up and idling.
Chrome (on Windows 10) Version 102.0.5005.115 (Official Build) (64-bit) (Chrome started an update cycle when I went to grab the version info). However, there are *numerous* reports in this thread of this occurring w/o any open browser tabs to the unraid UI.
Reading this entire thread, the issue appears to have been present through several releases going back to 6.8.3 (2019) and other releases through, now 2022 (a few new posts today) at least.
Another correlate seems to be using the web terminals to access unraid: I have not done this in many months, across several restarts now, so for this report, that aspect is likely not in play.
The other suggestion was problematic dockers. I am only running the official Emby docker, with two variants (not active). All are current versions and appear to be working fine.
I have one VM (Debian Jessie) that has not been up for months (only need it rarely).
Lastly, I have not observed the log partition ever filling up (so far). All drives (including the flash) appear to have plenty of space left.
Seems like this one may be an actual memory leak in the daemon <facepalm>... I hate tracking those down!
Let me know if there's any more info you need! Thanks for taking a look.
maxtower-diagnositcs-20220628-1758.zip