Unraid 6.8.0

 

Hi guys,

 

This past saturday, out of the blue, i noticed the dashboard on my server wasn't showing any cpu stats... then i noticed none of my docker stats were working, then realized my syslog was at 100%. The whole system seemed very slow and unresponsive. Fix Common Problems showed nothing out of sorts, and the dockers themselves (plex for example) all seemed to be working fine. I tried to use the diagnostics zip tool and even that wouldn't work - unresponsive. I used terminal to capture the syslog to flash manually - the wiki said it would be ready to post to forums using this method, but if it's not anonymized let me know and i'll take it down straight away. this log file is below, titled "syslog-2020-01-11.zip". Clean reboot, everything seemed to return to normal.

 

Fast forward a couple days to just a few short moments ago, i logged in to find my syslog back up to 77%. FCP still not showing any problems. diagnostic zip tool worked this time, so i snagged the logs and have attached them below as well. file is "kal-el-diagnostics-20200113-1320.zip". I glanced at the logs from today and saw a few red lines, but unfortunately the rest is greek to me. 

 

I have a couple of suspicions... i installed folding@home last week, even with cpu and memory parameters, it caused problems so i uninstalled (no good deed... lol). other possibility is i have a semi permanent wireguard connection now with other remote unraid server. but that's been setup for weeks now with no issues (until now i guess). other than that, I can't think of any changes that might be the culprit, really don't know where to start. Thanks in advance

 

-Chaz

 

kal-el-diagnostics-20200113-1320.zipsyslog-2020-01-11.zip

 

desperate bump

All I can tell is problem lies with nginx.

 

Don't use it so don't know what's wrong with it. Try turning NGINX off and restart your server to see if the problem reappears.

Edited January 14, 20206 yr by testdasi

Try safe mode, if that doesn't help try redoing the flash drive and just copying the key and super.dat for the assignments (or use just a trial flash to test)

Thanks guys, i'll give those a shot and report back.

 

Off topic for a second, but, @johnnie.black i feel compelled to tell you how much i like your handle and avatar. not sure if you're familiar with Christopher Hitchens or not, but you might find this short youtube clip of general interest. "The best blended scotch in the history of the world..."

Edited January 15, 20206 yr by Cpt. Chaz
typo

I think I found the problem gentlemen... a mac on my lan was running the folding@home gui application, trying to connect to my previous installation of FAH docker on unraid. thanks for the help!

ok, nope, that wasn't the problem after all.

 

server locked up again. wasn't able to use the diagnostic tool to capture the syslog, used terminal again to capture "syslog.txt" and i also ran a short tail and exported it "tail syslog.txt". Attached below.

 

The tail output was running with docker service disabled, so this should disable nginx, right? I could try restarting in safe mode, but i don't know how to reproduce this problem. It just happened out of the blue again. i'm seeing

# Jan 17 16:27:00 Kal-El nginx: 2020/01/17 16:27:00 [crit] 4850#4850: ngx_slab_alloc() failed: no memory
# Jan 17 16:27:00 Kal-El nginx: 2020/01/17 16:27:00 [error] 4850#4850: shpool alloc failed
# Jan 17 16:27:00 Kal-El nginx: 2020/01/17 16:27:00 [error] 4850#4850: nchan: Out of shared memory while allocating channel /var. Increase nchan_max_reserved_memory.

over and over. nginx should be disabled with letsencrypt docker closed, right? and out of shared memory? getting out of my depth here, but with 192gb of ram, only ever getting up to 7-8% utilization with no more than i have running, seems strange. it says to "increase nchan_max_reserved_memory". should i do that? if so, how?

 

It also seems i'm not alone, but they had no resolution it looks like here.

 

edit: i also increased vm dirty ratios a few days ago per this and it obviously made no difference.

tail syslog.txt.zip syslog.txt.zip

Edited January 17, 20206 yr by Cpt. Chaz
addition

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=781601