ohj Posted January 22, 2021 Share Posted January 22, 2021 (edited) I've god unRAID installed on an old stationary computer, with 2x4TB data storage, 4TB parity and 256GB SSD cache. On it I have one Debian VM running Home Assistant, and a number of docker containers. My problem is that my computer just freezes at some point, and I get no contact with any running services or the unRAID GUI. When starting the computer, it boots just fine into unRAID, with the typical white on black text screen. When the computer freezes, however, the text is gone and the screen is completely blank (black). How can I troubleshoot this? The only way to get the system up again is to hold the power button in for 5 seconds to force a shutdown, and then boot the computer again. So I have no way to get log files before rebooting. unraid-diagnostics-20210122-0855.zip Edited February 13, 2021 by ohj I've added the diagnostics file, downloaded after forcing a shutdown and rebooting unRAID. Quote Link to comment
JorgeB Posted January 22, 2021 Share Posted January 22, 2021 You can try this: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=781601 1 Quote Link to comment
ohj Posted January 24, 2021 Author Share Posted January 24, 2021 On 1/22/2021 at 10:05 AM, JorgeB said: You can try this: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=781601 Thanks, I've activated logging on my server. I've also removed a VM I was only using to run Home Assistant, and am running Home Assistant Core as a Docker container instead. So I'll see if the problem crops up again over the next days... Quote Link to comment
ohj Posted January 29, 2021 Author Share Posted January 29, 2021 Since I activate logging the server has hung twice. Unfortunately I don't see anything happening at the point when the server is hanging. The server is sending me status emails every once in a while, and in the log I'm only seeing an entry of sending a status email a few hours before the server hangs, and then new log entries once the server reboots. So I'm uncertain how I ought to handle this... Hang #1: Jan 27 12:20:01 unraid sSMTP[23451]: Creating SSL connection to host Jan 27 12:20:02 unraid sSMTP[23451]: SSL connection using TLS_AES_256_GCM_SHA384 Jan 27 12:20:03 unraid sSMTP[23451]: Sent mail for ***@***.no (221 2.0.0 Bye) uid=0 username=root outbytes=1134 Jan 27 12:23:46 unraid kernel: veth10fae82: renamed from eth0 Jan 27 12:23:46 unraid kernel: docker0: port 6(veth882542c) entered disabled state Jan 27 12:23:46 unraid kernel: docker0: port 6(veth882542c) entered disabled state Jan 27 12:23:46 unraid kernel: device veth882542c left promiscuous mode Jan 27 12:23:46 unraid kernel: docker0: port 6(veth882542c) entered disabled state Jan 27 12:24:44 unraid kernel: docker0: port 6(vethe18f83a) entered blocking state Jan 27 12:24:44 unraid kernel: docker0: port 6(vethe18f83a) entered disabled state Jan 27 12:24:44 unraid kernel: device vethe18f83a entered promiscuous mode Jan 27 12:24:44 unraid kernel: IPv6: ADDRCONF(NETDEV_UP): vethe18f83a: link is not ready Jan 27 12:24:45 unraid kernel: eth0: renamed from vethc36c89c Jan 27 12:24:45 unraid kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethe18f83a: link becomes ready Jan 27 12:24:45 unraid kernel: docker0: port 6(vethe18f83a) entered blocking state Jan 27 12:24:45 unraid kernel: docker0: port 6(vethe18f83a) entered forwarding state Jan 27 18:20:01 unraid sSMTP[26939]: Creating SSL connection to host Jan 27 18:20:01 unraid sSMTP[26939]: SSL connection using TLS_AES_256_GCM_SHA384 Jan 27 18:20:03 unraid sSMTP[26939]: Sent mail for ***@***.no (221 2.0.0 Bye) uid=0 username=root outbytes=1134 <== SERVER HUNG. FORCING SHUTDOWN AND REBOOTING SERVER. Jan 28 00:00:16 unraid root: Delaying execution of fix common problems scan for 10 minutes Jan 28 00:00:16 unraid emhttpd: Starting services... Jan 28 00:00:16 unraid emhttpd: shcmd (44): /etc/rc.d/rc.samba restart Jan 28 00:00:17 unraid rsyslogd: [origin software="rsyslogd" swVersion="8.1908.0" x-pid="2653" x-info="https://www.rsyslog.com"] start Jan 28 00:00:18 unraid root: Starting Samba: /usr/sbin/smbd -D Jan 28 00:00:18 unraid root: /usr/sbin/nmbd -D Jan 28 00:00:18 unraid root: /usr/sbin/wsdd Jan 28 00:00:18 unraid root: /usr/sbin/winbindd -D Jan 28 00:00:19 unraid emhttpd: shcmd (58): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20 Jan 28 00:00:19 unraid kernel: BTRFS: device fsid 900e8dad-69d0-4748-b5c1-c657b528f60e devid 1 transid 53866 /dev/loop2 Jan 28 00:00:19 unraid kernel: BTRFS info (device loop2): disk space caching is enabled Jan 28 00:00:19 unraid kernel: BTRFS info (device loop2): has skinny extents Jan 28 00:00:19 unraid kernel: BTRFS info (device loop2): start tree-log replay Jan 28 00:00:21 unraid kernel: BTRFS info (device loop2): checking UUID tree Jan 28 00:00:21 unraid root: Resize '/var/lib/docker' of 'max' Jan 28 00:00:21 unraid kernel: BTRFS info (device loop2): new size for /dev/loop2 is 21474836480 Jan 28 00:00:21 unraid emhttpd: shcmd (60): /etc/rc.d/rc.docker start Jan 28 00:00:21 unraid root: starting dockerd ... Hang #2: Jan 29 02:15:40 unraid CA Backup/Restore: ####################### Jan 29 02:15:40 unraid CA Backup/Restore: appData Backup complete Jan 29 02:15:40 unraid CA Backup/Restore: ####################### Jan 29 02:15:40 unraid sSMTP[32198]: Creating SSL connection to host Jan 29 02:15:41 unraid sSMTP[32198]: SSL connection using TLS_AES_256_GCM_SHA384 Jan 29 02:15:42 unraid sSMTP[32198]: Sent mail for ***@***.no (221 2.0.0 Bye) uid=0 username=root outbytes=628 Jan 29 02:15:43 unraid CA Backup/Restore: Deleting /mnt/user/backups/CommunityApplicationsAppdataBackup/[email protected] Jan 29 02:15:43 unraid CA Backup/Restore: Backup / Restore Completed Jan 29 03:02:38 unraid kernel: traps: OZW-driver[32583] general protection ip:15441fa9b5d0 sp:15441b875358 error:0 in libopenzwave.so.1.6[15441f9b2000+115000] Jan 29 03:40:01 unraid crond[1679]: exit status 1 from user root /usr/local/sbin/mover &> /dev/null Jan 29 06:20:01 unraid sSMTP[12148]: Creating SSL connection to host Jan 29 06:20:02 unraid sSMTP[12148]: SSL connection using TLS_AES_256_GCM_SHA384 Jan 29 06:20:03 unraid sSMTP[12148]: Sent mail for ***@***.no (221 2.0.0 Bye) uid=0 username=root outbytes=1137 <== SERVER HUNG. FORCING SHUTDOWN AND REBOOTING SERVER. Jan 29 09:01:14 unraid root: Delaying execution of fix common problems scan for 10 minutes Jan 29 09:01:14 unraid emhttpd: Starting services... Jan 29 09:01:14 unraid emhttpd: shcmd (44): /etc/rc.d/rc.samba restart Jan 29 09:01:15 unraid rsyslogd: [origin software="rsyslogd" swVersion="8.1908.0" x-pid="2655" x-info="https://www.rsyslog.com"] start Jan 29 09:01:16 unraid root: Starting Samba: /usr/sbin/smbd -D Jan 29 09:01:16 unraid root: /usr/sbin/nmbd -D Jan 29 09:01:16 unraid root: /usr/sbin/wsdd Jan 29 09:01:16 unraid root: /usr/sbin/winbindd -D Jan 29 09:01:16 unraid emhttpd: shcmd (58): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20 Jan 29 09:01:16 unraid kernel: BTRFS: device fsid 900e8dad-69d0-4748-b5c1-c657b528f60e devid 1 transid 57069 /dev/loop2 Jan 29 09:01:16 unraid kernel: BTRFS info (device loop2): disk space caching is enabled Jan 29 09:01:16 unraid kernel: BTRFS info (device loop2): has skinny extents Jan 29 09:01:16 unraid kernel: BTRFS info (device loop2): start tree-log replay Jan 29 09:01:16 unraid kernel: BTRFS info (device loop2): checking UUID tree Jan 29 09:01:16 unraid root: Resize '/var/lib/docker' of 'max' Jan 29 09:01:16 unraid kernel: BTRFS info (device loop2): new size for /dev/loop2 is 21474836480 Jan 29 09:01:16 unraid emhttpd: shcmd (60): /etc/rc.d/rc.docker start Jan 29 09:01:16 unraid root: starting dockerd ... Quote Link to comment
JorgeB Posted January 29, 2021 Share Posted January 29, 2021 One thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one. 1 Quote Link to comment
ohj Posted January 29, 2021 Author Share Posted January 29, 2021 I've just disabled C states in BIOS, after reviewing this thread and seeing that I have the same crash symptoms: https://forums.unraid.net/topic/61774-unraid-crashesfreezes-daily/ So I'll see over the next few days if it had any impact.. 1 Quote Link to comment
ohj Posted January 29, 2021 Author Share Posted January 29, 2021 (edited) 4 minutes ago, JorgeB said: One thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one. Thank you for the input. Yes, I will do that if it still crashes with C states disabled. Edited January 29, 2021 by ohj Quote Link to comment
ohj Posted February 13, 2021 Author Share Posted February 13, 2021 Well, it's been a couple of weeks since the last crash/hang. I've restarted unRAID for other reasons a couple of times, but as of now my uptime is a little more than 8 days - more than double of the longest uptime period I had before disabling the C states. So I think it's safe to say that disabling the C states was the solution in my case. Thanks for all input. 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.