NAS crashing after a few days - regularly...

[theunraidhomeuser]

Hi there,

 

I have a problem I would please use some help with. My NAS, a self-built server with quite decent specs, constantly crashes after a few days. Not sure why as I don't know where to troubleshoot. A few months ago, somebody suggested to remove the RAM overclocking although the modules and system should be able to run 3200 MHz. Anyway, did that, and things improved. The server used to crash after around 24 hours and now it's up for 4-5 days before it crashes.

 

What do I mean by crashed?

GUI is no longer accessible, services like VMs don't respond anymore.

 

Appreciate your thoughts and suggestions. I'm currently not at the server location but can troubleshoot remotely.

 

Thanks!

super-nas-diagnostics-20220724-1407.zip

[ChatNoir]

You should set up a syslog server and post that file after a crash.

[theunraidhomeuser]

Hi there, I thought the issue had solved itself but with the recent update of unraid it's unfortunately back..

 

Dec 22 09:00:04 SUPER-NAS root: /etc/libvirt: 24 MiB (25128960 bytes) trimmed on /dev/loop3
Dec 22 09:00:04 SUPER-NAS root: /var/lib/docker: 59.9 GiB (64322048000 bytes) trimmed on /dev/loop2
Dec 22 09:00:04 SUPER-NAS root: /mnt/vm: 17.7 GiB (19046002688 bytes) trimmed on /dev/sdi1
Dec 22 09:00:04 SUPER-NAS root: /mnt/cache: 1.5 GiB (1661894656 bytes) trimmed on /dev/sdj1
Dec 22 09:00:07 SUPER-NAS  crond[1513]: exit status 1 from user root /usr/local/sbin/mover &> /dev/null
Dec 22 10:00:02 SUPER-NAS  crond[1513]: exit status 1 from user root /usr/local/sbin/mover &> /dev/null
Dec 22 10:00:04 SUPER-NAS root: /etc/libvirt: 24 MiB (25128960 bytes) trimmed on /dev/loop3
Dec 22 10:00:04 SUPER-NAS root: /var/lib/docker: 59.9 GiB (64321851392 bytes) trimmed on /dev/loop2
Dec 22 10:00:04 SUPER-NAS root: /mnt/vm: 17.7 GiB (18955059200 bytes) trimmed on /dev/sdi1
Dec 22 10:00:04 SUPER-NAS root: /mnt/cache: 1.5 GiB (1661894656 bytes) trimmed on /dev/sdj1

 

The syslog entries above are the very latest that happened before the machine went down. Nothing I can see that would explain what happened, all looks pretty regular to me except for the error that the move threw (however it does that every hour even when the machine doesn't crash).

 

Am attaching the diags once more, if anyone has an idea. There was this issue with AMD CPUs and UNRAID and I think there could be something linked to that..

 

any ideas? Thanks all!

 

super-nas-diagnostics-20221223-0857.zip

[JorgeB]

See if this helps:

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=819173

 

[theunraidhomeuser]

hi there, none of this worked unfortunately... any other idea, is this not something that should be fixed by the unraid devs? Should I file a bug report?

[JorgeB]

Enable the syslog server and post that after a crash.

[theunraidhomeuser]

13 minutes ago, JorgeB said:

Enable the syslog server and post that after a crash.

see above!

[JorgeB]

25 minutes ago, theunraidhomeuser said:

see above!

I don't see the persistent syslog posted above, only the regular diags.

[theunraidhomeuser]

I don't know how to do that, I enabled syslog to another NAS device and that's the output I got...

Meanwhile I've been playing with disabled C-States and am monitoring. For a few days things are good. Also did a BIOS update of my MSI board (though none of the changelog items suggested any fixes in this direction), so let's see.

 

Thanks again for your ongoing support!