JamminTK Posted December 7, 2019 Share Posted December 7, 2019 (edited) Hello, After a variable amount of time, Unraid becomes unresponsive. Loading the webgui shows a "The Connection has Timed Out" error, the server stops responding to SSH requests, shares are unavailable, and everything generally just grinds to a halt and I'm forced to manually shut down the server. The only things running aside from the base OS are a few docker containers (binhex's Plex, Sonarr, Radarr, Jackett, and Qbittorrentvpn ones), but I haven't even finished setting them up yet. This is a very new install I'm running Unraid 6.7.2 (Trial license) on the following hardware: AMD Ryzen 5 1600 (not overclocked) 1x8GB DDR4 memory (2400mhz) Gigabyte A320m-S2H motherboard EVGA GT 610 (just so the thing boots with the Ryzen 1600) As I'm just evaluating the software, I'm running an array of a single Seagate 6TB drive. It's brand new and has a clean SMART report. Since the server becomes unresponsive, I can't provide a diagnostic.zip, but here's the output of my syslog until it becomes unresponsive. This one was pretty quick. root@Homeserver:~# tail -f /var/log/syslog Dec 6 19:49:32 Homeserver kernel: md1: running, size: 5860522532 blocks Dec 6 19:49:32 Homeserver emhttpd: shcmd (27): udevadm settle Dec 6 19:49:33 Homeserver emhttpd: Autostart disabled Dec 6 19:49:33 Homeserver kernel: mdcmd (37): stop Dec 6 19:49:33 Homeserver kernel: md1: stopping Dec 6 19:49:33 Homeserver avahi-daemon[2450]: Server startup complete. Host name is Homeserver.local. Local service cookie is 627222976. Dec 6 19:49:34 Homeserver avahi-daemon[2450]: Service "Homeserver" (/services/ssh.service) successfully established. Dec 6 19:49:34 Homeserver avahi-daemon[2450]: Service "Homeserver" (/services/smb.service) successfully established. Dec 6 19:49:34 Homeserver avahi-daemon[2450]: Service "Homeserver" (/services/sftp-ssh.service) successfully established. Dec 6 19:49:35 Homeserver sshd[2529]: Accepted none for root from 192.168.0.3 port 54467 ssh2 Dec 6 19:51:34 Homeserver emhttpd: req (1): startState=STOPPED&file=&csrf_token=****************&cmdStart=Start Dec 6 19:51:34 Homeserver emhttpd: shcmd (150): /usr/local/sbin/set_ncq sdb 1 Dec 6 19:51:34 Homeserver kernel: mdcmd (38): set md_num_stripes 1280 Dec 6 19:51:34 Homeserver kernel: mdcmd (39): set md_sync_window 384 Dec 6 19:51:34 Homeserver kernel: mdcmd (40): set md_sync_thresh 192 Dec 6 19:51:34 Homeserver kernel: mdcmd (41): set md_write_method Dec 6 19:51:34 Homeserver kernel: mdcmd (42): set spinup_group 1 0 Dec 6 19:51:34 Homeserver emhttpd: shcmd (151): echo 128 > /sys/block/sdb/queue/nr_requests Dec 6 19:51:34 Homeserver kernel: mdcmd (43): start STOPPED Dec 6 19:51:34 Homeserver kernel: unraid: allocating 15740K for 1280 stripes (3 disks) Dec 6 19:51:34 Homeserver kernel: md1: running, size: 5860522532 blocks Dec 6 19:51:34 Homeserver emhttpd: shcmd (152): udevadm settle Dec 6 19:51:34 Homeserver root: Starting diskload Dec 6 19:51:34 Homeserver emhttpd: Mounting disks... Dec 6 19:51:34 Homeserver emhttpd: shcmd (154): /sbin/btrfs device scan Dec 6 19:51:34 Homeserver root: Scanning for Btrfs filesystems Dec 6 19:51:34 Homeserver emhttpd: shcmd (155): mkdir -p /mnt/disk1 Dec 6 19:51:34 Homeserver emhttpd: shcmd (156): mount -t xfs -o noatime,nodiratime /dev/md1 /mnt/disk1 Dec 6 19:51:34 Homeserver kernel: SGI XFS with ACLs, security attributes, no debug enabled Dec 6 19:51:34 Homeserver kernel: XFS (md1): Mounting V5 Filesystem Dec 6 19:51:34 Homeserver kernel: XFS (md1): Starting recovery (logdev: internal) Dec 6 19:51:34 Homeserver kernel: XFS (md1): Ending recovery (logdev: internal) Dec 6 19:51:34 Homeserver emhttpd: shcmd (157): xfs_growfs /mnt/disk1 Dec 6 19:51:34 Homeserver root: meta-data=/dev/md1 isize=512 agcount=6, agsize=268435455 blks Dec 6 19:51:34 Homeserver root: = sectsz=512 attr=2, projid32bit=1 Dec 6 19:51:34 Homeserver root: = crc=1 finobt=1, sparse=1, rmapbt=0 Dec 6 19:51:34 Homeserver root: = reflink=0 Dec 6 19:51:34 Homeserver root: data = bsize=4096 blocks=1465130633, imaxpct=5 Dec 6 19:51:34 Homeserver root: = sunit=0 swidth=0 blks Dec 6 19:51:34 Homeserver root: naming =version 2 bsize=4096 ascii-ci=0, ftype=1 Dec 6 19:51:34 Homeserver root: log =internal log bsize=4096 blocks=521728, version=2 Dec 6 19:51:34 Homeserver root: = sectsz=512 sunit=0 blks, lazy-count=1 Dec 6 19:51:34 Homeserver root: realtime =none extsz=4096 blocks=0, rtextents=0 Dec 6 19:51:34 Homeserver emhttpd: shcmd (158): sync Dec 6 19:51:35 Homeserver emhttpd: shcmd (159): mkdir /mnt/user Dec 6 19:51:35 Homeserver emhttpd: shcmd (160): /usr/local/sbin/shfs /mnt/user -disks 2 -o noatime,big_writes,allow_other -o remember=0 |& logger Dec 6 19:51:35 Homeserver shfs: stderr redirected to syslog Dec 6 19:51:35 Homeserver emhttpd: shcmd (162): /usr/local/sbin/update_cron Dec 6 19:51:35 Homeserver emhttpd: Starting services... Dec 6 19:51:35 Homeserver emhttpd: shcmd (164): /etc/rc.d/rc.samba restart Dec 6 19:51:37 Homeserver root: Starting Samba: /usr/sbin/nmbd -D Dec 6 19:51:37 Homeserver root: /usr/sbin/smbd -D Dec 6 19:51:37 Homeserver root: /usr/sbin/winbindd -D Dec 6 19:51:37 Homeserver emhttpd: shcmd (178): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20 Dec 6 19:51:37 Homeserver kernel: BTRFS: device fsid 7cc58970-9111-4c98-8010-9f316d5aa7af devid 1 transid 77 /dev/loop2 Dec 6 19:51:37 Homeserver kernel: BTRFS info (device loop2): disk space caching is enabled Dec 6 19:51:37 Homeserver kernel: BTRFS info (device loop2): has skinny extents Dec 6 19:51:37 Homeserver root: Resize '/var/lib/docker' of 'max' Dec 6 19:51:37 Homeserver kernel: BTRFS info (device loop2): new size for /dev/loop2 is 21474836480 Dec 6 19:51:37 Homeserver emhttpd: shcmd (180): /etc/rc.d/rc.docker start Dec 6 19:51:38 Homeserver root: starting dockerd ... Dec 6 19:51:41 Homeserver avahi-daemon[2450]: Joining mDNS multicast group on interface docker0.IPv4 with address 172.17.0.1. Dec 6 19:51:41 Homeserver avahi-daemon[2450]: New relevant interface docker0.IPv4 for mDNS. Dec 6 19:51:41 Homeserver avahi-daemon[2450]: Registering new address record for 172.17.0.1 on docker0.IPv4. Dec 6 19:51:41 Homeserver kernel: IPv6: ADDRCONF(NETDEV_UP): docker0: link is not readyDec 6 19:51:48 Homeserver emhttpd: shcmd (194): /usr/local/sbin/mount_image '/mnt/user/system/libvirt/libvirt.img' /etc/libvirt 1 Dec 6 19:51:48 Homeserver kernel: BTRFS: device fsid 8fdc5f38-45fc-41f3-b049-905869dce4e5 devid 1 transid 8 /dev/loop3 Dec 6 19:51:48 Homeserver kernel: BTRFS info (device loop3): disk space caching is enabled Dec 6 19:51:48 Homeserver kernel: BTRFS info (device loop3): has skinny extents Dec 6 19:51:48 Homeserver root: Resize '/etc/libvirt' of 'max' Dec 6 19:51:48 Homeserver kernel: BTRFS info (device loop3): new size for /dev/loop3 is 1073741824 Dec 6 19:51:48 Homeserver emhttpd: shcmd (196): /etc/rc.d/rc.libvirt start Dec 6 19:51:48 Homeserver root: Starting virtlockd... Dec 6 19:51:48 Homeserver root: Starting virtlogd... Dec 6 19:51:48 Homeserver root: Starting libvirtd... Dec 6 19:51:48 Homeserver kernel: tun: Universal TUN/TAP device driver, 1.6 Dec 6 19:51:48 Homeserver emhttpd: nothing to sync Dec 6 19:51:48 Homeserver kernel: virbr0: port 1(virbr0-nic) entered blocking state Dec 6 19:51:48 Homeserver kernel: virbr0: port 1(virbr0-nic) entered disabled state Dec 6 19:51:48 Homeserver kernel: device virbr0-nic entered promiscuous mode Dec 6 19:51:48 Homeserver avahi-daemon[2450]: Joining mDNS multicast group on interface virbr0.IPv4 with address 192.168.122.1. Dec 6 19:51:48 Homeserver avahi-daemon[2450]: New relevant interface virbr0.IPv4 for mDNS. Dec 6 19:51:48 Homeserver kernel: virbr0: port 1(virbr0-nic) entered blocking state Dec 6 19:51:48 Homeserver kernel: virbr0: port 1(virbr0-nic) entered listening state Dec 6 19:51:48 Homeserver avahi-daemon[2450]: Registering new address record for 192.168.122.1 on virbr0.IPv4. Dec 6 19:51:48 Homeserver dnsmasq[4014]: started, version 2.80 cachesize 150 Dec 6 19:51:48 Homeserver dnsmasq[4014]: compile time options: IPv6 GNU-getopt no-DBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify dumpfile Dec 6 19:51:48 Homeserver dnsmasq-dhcp[4014]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h Dec 6 19:51:48 Homeserver dnsmasq-dhcp[4014]: DHCP, sockets bound exclusively to interface virbr0 Dec 6 19:51:48 Homeserver dnsmasq[4014]: reading /etc/resolv.conf Dec 6 19:51:48 Homeserver dnsmasq[4014]: using nameserver 8.8.8.8#53 Dec 6 19:51:48 Homeserver dnsmasq[4014]: read /etc/hosts - 2 addresses Dec 6 19:51:48 Homeserver dnsmasq[4014]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses Dec 6 19:51:48 Homeserver dnsmasq-dhcp[4014]: read /var/lib/libvirt/dnsmasq/default.hostsfile Dec 6 19:51:48 Homeserver kernel: virbr0: port 1(virbr0-nic) entered disabled state Dec 6 19:54:26 Homeserver kernel: docker0: port 1(vethf317cba) entered blocking state Dec 6 19:54:26 Homeserver kernel: docker0: port 1(vethf317cba) entered disabled state Dec 6 19:54:26 Homeserver kernel: device vethf317cba entered promiscuous mode Dec 6 19:54:26 Homeserver kernel: IPv6: ADDRCONF(NETDEV_UP): vethf317cba: link is not ready Dec 6 19:54:26 Homeserver kernel: docker0: port 1(vethf317cba) entered blocking state Dec 6 19:54:26 Homeserver kernel: docker0: port 1(vethf317cba) entered forwarding stateDec 6 19:54:26 Homeserver kernel: docker0: port 1(vethf317cba) entered disabled state Dec 6 19:54:28 Homeserver kernel: eth0: renamed from veth95e856c Dec 6 19:54:28 Homeserver kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethf317cba: link becomes ready Dec 6 19:54:28 Homeserver kernel: docker0: port 1(vethf317cba) entered blocking state Dec 6 19:54:28 Homeserver kernel: docker0: port 1(vethf317cba) entered forwarding stateDec 6 19:54:28 Homeserver kernel: IPv6: ADDRCONF(NETDEV_CHANGE): docker0: link becomes ready Dec 6 19:54:30 Homeserver avahi-daemon[2450]: Joining mDNS multicast group on interface vethf317cba.IPv6 with address fe80::5c3c:3cff:fe10:67e7. Dec 6 19:54:30 Homeserver avahi-daemon[2450]: New relevant interface vethf317cba.IPv6 for mDNS. Dec 6 19:54:30 Homeserver avahi-daemon[2450]: Registering new address record for fe80::5c3c:3cff:fe10:67e7 on vethf317cba.*. Dec 6 19:54:30 Homeserver avahi-daemon[2450]: Joining mDNS multicast group on interface docker0.IPv6 with address fe80::42:afff:fe39:8666. Dec 6 19:54:30 Homeserver avahi-daemon[2450]: New relevant interface docker0.IPv6 for mDNS. Dec 6 19:54:30 Homeserver avahi-daemon[2450]: Registering new address record for fe80::42:afff:fe39:8666 on docker0.*. Dec 6 19:54:30 Homeserver kernel: docker0: port 1(vethf317cba) entered disabled state Dec 6 19:54:30 Homeserver kernel: veth95e856c: renamed from eth0 Dec 6 19:54:31 Homeserver avahi-daemon[2450]: Interface vethf317cba.IPv6 no longer relevant for mDNS. Dec 6 19:54:31 Homeserver avahi-daemon[2450]: Leaving mDNS multicast group on interface vethf317cba.IPv6 with address fe80::5c3c:3cff:fe10:67e7. Dec 6 19:54:31 Homeserver kernel: docker0: port 1(vethf317cba) entered disabled state Dec 6 19:54:31 Homeserver kernel: device vethf317cba left promiscuous mode Dec 6 19:54:31 Homeserver kernel: docker0: port 1(vethf317cba) entered disabled state Dec 6 19:54:31 Homeserver avahi-daemon[2450]: Withdrawing address record for fe80::5c3c:3cff:fe10:67e7 on vethf317cba. Dec 6 19:54:33 Homeserver ntpd[1819]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized Connection reset by 192.168.0.2 port 22 I tried the CA Fix Common Problems plugin, but it couldn't find any errors after a scan. I also ran a memtest on my RAM and it came back completely clean. 0 errors after 4 passes of every test. I originally was trying Unraid on a different CPU (Athlon 200GE) and hard drives (1x HGST 2TB and 1x Seagate 2TB) but after experiencing the freezes, I decided to try new hardware first. The hard drives especially were old and didn't have completely clean SMART reports (both had CRC errors) Edited December 8, 2019 by JamminTK Quote Link to comment
trurl Posted December 7, 2019 Share Posted December 7, 2019 Get a diagnostic before freeze. Also Settings - Syslog Server to mirror syslog to flash. Go ahead and disable docker service for now until you can get the system to stay up just to simplify things for troubleshooting. Quote Link to comment
JorgeB Posted December 7, 2019 Share Posted December 7, 2019 Ryzen on Linux can lock up due to issues with c-states, make sure bios is up to date, then look for "Power Supply Idle Control" (or similar) and set it to "typical current idle" (or similar), or completely disable C-sates. More info here: https://forums.unraid.net/bug-reports/prereleases/670-rc1-system-hard-lock-r354/ Quote Link to comment
JamminTK Posted December 8, 2019 Author Share Posted December 8, 2019 I'm sorry it took me an extra day to respond. Thanks for the advice. I updated my bios, turned off C-states, and set power supply idle control to typical current idle. So far, so good at 2 hours of uptime. Usually it crashes anywhere from 45 minutes to two hours in. I'll mark it as solved. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.