Mr_Jay84 Posted August 15, 2021 Share Posted August 15, 2021 I've been suffering some strange crashes recently completely at random on 6.9.1. This morning it was at 00:46 The system can't be accessed at all via web or SSH. I did find this section in the log, can anyone tell what's happened here? New Text Document (4).txt Quote Link to comment
Mr_Jay84 Posted August 19, 2021 Author Share Posted August 19, 2021 Crashed again during the night. ultron-diagnostics-20210819-1002.zip Quote Link to comment
JorgeB Posted August 19, 2021 Share Posted August 19, 2021 Previous syslog shows many call traces, can't see the reason for them though, could be hardware or kernel related, try upgrading to v6.10 to see if the newer kernel helps, if it doesn't could be hardware. Quote Link to comment
Mr_Jay84 Posted August 19, 2021 Author Share Posted August 19, 2021 4 hours ago, JorgeB said: Previous syslog shows many call traces, can't see the reason for them though, could be hardware or kernel related, try upgrading to v6.10 to see if the newer kernel helps, if it doesn't could be hardware. Worth a try Jorge Quote Link to comment
Mr_Jay84 Posted August 30, 2021 Author Share Posted August 30, 2021 Upgrade to 6.10RC Two crashes this week so far. ultron-diagnostics-20210830-1554.zip Quote Link to comment
trurl Posted August 30, 2021 Share Posted August 30, 2021 https://wiki.unraid.net/Manual/Troubleshooting#Persistent_Logs_.28Syslog_server.29 Quote Link to comment
Mr_Jay84 Posted August 30, 2021 Author Share Posted August 30, 2021 (edited) I just so happened to have it already set up. It crashed some time after 1300BST Ultron Edited August 30, 2021 by Mr_Jay84 Quote Link to comment
trurl Posted August 30, 2021 Share Posted August 30, 2021 Problems with multiple disks. Probably controller or maybe power, splitter. Quote Link to comment
Mr_Jay84 Posted August 30, 2021 Author Share Posted August 30, 2021 1 minute ago, trurl said: Problems with multiple disks. Probably controller or maybe power, splitter. Yeah I know about that issue. Some of these disks (particular model) don't like being spun down, that's only a recent change though as this strange crash was happening before I set the to spin down. I have a post on the HD subject Quote Link to comment
trurl Posted August 30, 2021 Share Posted August 30, 2021 16 minutes ago, Mr_Jay84 said: Yeah I know about that issue. Seems unusable until you get that fixed. Quote Link to comment
Mr_Jay84 Posted August 30, 2021 Author Share Posted August 30, 2021 Yeah I thought as much. I'll keep them spun up for now and post back when another crash happens. Thanks for responding so far mate. Quote Link to comment
Mr_Jay84 Posted September 21, 2021 Author Share Posted September 21, 2021 Happened again today randomly at roughly <77>1 2021-09-21T18:49:01+01:00 Ultron ultron-diagnostics-20210921-1950.zip Quote Link to comment
trurl Posted September 21, 2021 Share Posted September 21, 2021 Why does you server have a public internet address? inet 200.200.1.111 netmask 255.255.255.0 broadcast 200.200.1.255 Quote Link to comment
Mr_Jay84 Posted September 21, 2021 Author Share Posted September 21, 2021 It's been on the internal network for years, never got around to switching everything over. Quote Link to comment
trurl Posted September 21, 2021 Share Posted September 21, 2021 That IP is owned by someone in Brazil Quote Link to comment
Mr_Jay84 Posted September 21, 2021 Author Share Posted September 21, 2021 It's my internal IP range. Quote Link to comment
trurl Posted September 21, 2021 Share Posted September 21, 2021 https://en.wikipedia.org/wiki/Private_network Quote Link to comment
Mr_Jay84 Posted September 21, 2021 Author Share Posted September 21, 2021 Yeah I'm aware but that's not the issue at hand mate. Quote Link to comment
trurl Posted September 21, 2021 Share Posted September 21, 2021 Last thing logged before reboot as you may have noticed was Parity Check Tuning plugin. Does it happen if you remove that? Also notice you have 100G docker.img. That is usually a sign of misconfiguration by the user, especially since you are actually using 70G of that 100. 20G for docker.img is often more than enough. The usual cause of filling docker.img is an application writing to a path that isn't mapped. What can be even more fatal is filling rootfs, since that is where the OS files are and will typically break the OS. One way to fill rootfs with a misconfigured docker is by specifying a host path that isn't a disk or user share. You also have a large number of .cfg files in config/shares on flash. That suggests you may have accidentally created user shares at some time, perhaps by specifiying a path to the top level of a disk or /mnt/user. You might try monitoring rootfs to see if you are filling it. What do you get from the command line with this? df -h Quote Link to comment
Mr_Jay84 Posted September 22, 2021 Author Share Posted September 22, 2021 I've removed Parity Check Tuning plugin as of now. I've got a rather large collection of containers, it's mainly the databases and PVR ones that are large. It's always been around 70G and is fairly constant. Having had the issue you described last year it usually just stops the docker service from running. In this case the UI completely crashes, I can't even reset by SSH, I need to use IMPI to reset the machine. I'll start going through the containers just to check the oaths anyway. Here's the command output root@Ultron:~# df -h Filesystem Size Used Avail Use% Mounted on rootfs 63G 1.7G 62G 3% / tmpfs 32M 5.9M 27M 19% /run /dev/sda1 7.2G 948M 6.3G 13% /boot overlay 63G 1.7G 62G 3% /lib/modules overlay 63G 1.7G 62G 3% /lib/firmware devtmpfs 63G 0 63G 0% /dev tmpfs 63G 264K 63G 1% /dev/shm cgroup_root 8.0M 0 8.0M 0% /sys/fs/cgroup tmpfs 128M 3.4M 125M 3% /var/log tmpfs 1.0M 0 1.0M 0% /mnt/disks tmpfs 1.0M 0 1.0M 0% /mnt/remotes /dev/md1 5.5T 2.0T 3.5T 37% /mnt/disk1 /dev/md2 5.5T 40G 5.5T 1% /mnt/disk2 /dev/sdb1 224G 187G 38G 84% /mnt/cache-docker /dev/sdf1 11T 5.2T 5.8T 48% /mnt/cache-downloads /dev/sdi1 932G 264G 669G 29% /mnt/cache-files /dev/sdd1 932G 6.6G 925G 1% /mnt/cache-media /dev/sde1 932G 6.6G 925G 1% /mnt/cache-tv shfs 11T 2.1T 8.9T 19% /mnt/user0 shfs 11T 2.1T 8.9T 19% /mnt/user /dev/loop2 100G 69G 30G 70% /var/lib/docker /dev/loop3 1.0G 6.1M 903M 1% /etc/libvirt //200.200.1.244/GusSync 43T 37T 6.3T 86% /mnt/remotes/200.200.1.244_GusSync //200.200.1.244/Mel Drive 43T 37T 6.3T 86% /mnt/remotes/200.200.1.244_Mel Drive //200.200.1.244/Public 43T 37T 6.3T 86% /mnt/remotes/200.200.1.244_Public //200.200.1.243/Public 43T 37T 6.3T 86% /mnt/remotes/200.200.1.243_Public root@Ultron:~# Quote Link to comment
itimpi Posted September 22, 2021 Share Posted September 22, 2021 1 hour ago, Mr_Jay84 said: I've removed Parity Check Tuning plugin as of now. The parity check tuning plugin never does nothing that can directly cause the system to become unresponsive, although it could be that a running parity check can cause this. The plugin periodically runs a monitor task to see if there is currently a parity check in progress so it is not abnormal to see entries in the syslog relating to this. Having said that I see that the version of the plugin you have installed is a few release out-of-date. Quote Link to comment
Mr_Jay84 Posted September 23, 2021 Author Share Posted September 23, 2021 Crashed again. Ultron Quote Link to comment
Mr_Jay84 Posted November 1, 2021 Author Share Posted November 1, 2021 It may have been down to the C6 power state in the BIOS. Now disabled, it's been stable so far. Quote Link to comment
Mr_Jay84 Posted December 17, 2021 Author Share Posted December 17, 2021 UPDATE: So the disabling the C states in the BIOS only worked for a few days. It's crashing almost daily now. Also added "rcu_nocbs=0-47" to the flash boot procedure. Still doesn't help. It's definitely related to the CPUs. Quote Link to comment
Squid Posted December 17, 2021 Share Posted December 17, 2021 What CPU are you running??? First time I've ever seen this without it actually stating what it is.... Vendor ID: GenuineIntel Model name: Genuine Intel(R) CPU @ 2.00GHz It's obviously a Xeon, but what model and what stepping? Does the system event log happen to show anything (in the BIOS) Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.