It just happened again.. Exactly the same issue as a week or so before, see the original post:

 

 

Exactly the same situation. One difference, last time it looked like KVM was throwing a lot of errors, I have since then totally phased out KVM (Also not enabled anymore). Issues comes back.

 

Diagnostics attached.

 

I will now reboot my server, that will help, it did last time.

 

This is something that is only happening on this latest build, never before (and I have run them all).

 

 

tower-diagnostics-20170915-0711.zip

?

I can't seem to get stability out of anything beyond 6.3.5 at the moment.  I haven't dug deep enough to figure out why.

 

It's unfortunate too because the WEB UI is lightning quick on 6.4 releases compared to 6.3.5

 

Have you tried running in safe mode to see if that holds up?

6.4 worked without any issues in every beta but the last one.. Have not tried safe mode since it seems to only go wrong every week.. 

And again

 

Sep 24 07:38:18 Tower kernel: CPU: 6 PID: 26074 Comm: java Not tainted 4.12.10-unRAID #1
Sep 24 07:38:18 Tower kernel: Call Trace:
Sep 24 07:38:18 Tower kernel: out_of_memory+0x3ac/0x3ce
Sep 24 07:38:18 Tower kernel: Killed process 18162 (shfs) total-vm:24973196kB, anon-rss:21359480kB, file-rss:4kB, shmem-rss:600kB

...

tower-diagnostics-20170924-0839.zip

Edited September 24, 20178 yr by Helmonder

Looks like your CrashPlan docker caused your OOM.  Fairly sure it uses Java which is what caused the OOM.  I don't use CrashPlan but I do use a docker that uses Java.  What I did to avoid OOMs and causing the docker to crash was to limit the docker to a set amount of memory with the '--memory="16GB"' docker advanced option and then limiting the amount of memory Java was allowed to allocate to half that amount (8GB)

Edited September 26, 20178 yr by BobPhoenix

when my server freezes up, and stops answering it's ip, 192.168.11.53, how can I get in to look at old logs? I can do a hard reset with the power buttons, and it will come back up, but the log only seems to pick up from the last boot up - none of the old crash information.

 

ah, reading, I need to shutdown, and pull the flash drive

 

logfile was gone, I'm trying to run a 

tail -f /var/log/syslog > /mnt/disk1/syslog.txt

in Screen (from nerdtools) in hopes it will capture what's going on up to the crash. My headless server is a real hassle to get a monitor going!

 

On a side note, I'm doing a huge rsync process from my old unRaid server to this new one. That's also running in a Screen window, and going over the network.

Edited September 26, 20178 yr by dkerlee

13 minutes ago, dkerlee said:

when my server freezes up, and stops answering it's ip, 192.168.11.53, how can I get in to look at old logs? I can do a hard reset with the power buttons, and it will come back up, but the log only seems to pick up from the last boot up - none of the old crash information.

 

ah, reading, I need to shutdown, and pull the flash drive

 

https://forums.lime-technology.com/topic/60394-important-diagnosing-hard-crashes/

Perhaps this has something to do with a parity rebuild happening while I'm doing a network saturation of copying data.

 

Just did a fresh install on the flash drive of 6.3.5. I'll wait for the parity rebuild to finish before I start rsync'ing over the data and installing plug-ins.

 

I'm also not using a cache drive at the moment. Once I get the data copied over, I'll pull the cache drive from the old server and move that over too.

 

I was able to get a bit of a log from that Screen Tail command.

 

I'm also on a Ryzen CPU new build. Apparently there have been stability problems with unRaid 6.3.5, which were finally fixed in 6.4rc7+ this thread. I wanted to get on that pre-release, which is 6.4.0rc9f at the moment, but couldn't get that figured out for a while either. Until I found this thread, by limetech, on How to Install Pre-releases. I'm now on 6.4.0rc9f, and we'll see how it goes.

 

sent 2,549,330 bytes  received 9,330,674,645,339 bytes (9.3TB)  39,822,358.20 bytes/sec (40 MB/sec, or 318 Mbit/sec)
total size is 9,666,168,481,636 (9.7TB)  speedup is 1.04

that was in about 56 hours give or take

 

Next big transfer I'm going to turn off the Parity disc, do the copy, then rebuild the Parity after it's done.

 

hopefully this will help some folks: release candidate, pre-release, rc, prerelease, ryzen, instability, unstable, crash, crashing

Edited October 1, 20178 yr by dkerlee

And another crash:

 

I am faily confident its the syncthing docker... I have been restarting it every couple of days and it kept on running... Guess I waited to long..

Oct  5 07:28:11 Tower kernel: CPU: 2 PID: 14057 Comm: Plex Tuner Serv Not tainted 4.12.14-unRAID #1
Oct  5 07:28:11 Tower kernel: Call Trace:
Oct  5 07:28:11 Tower kernel: out_of_memory+0x3ac/0x3ce
Oct  5 07:28:11 Tower kernel: Killed process 12884 (shfs) total-vm:25180156kB, anon-rss:21844748kB, file-rss:4kB, shmem-rss:716kB

How about double checking your file locations in each dockers image to make sure they aren't filling up your dockers image with temporary files or something.