February 20, 20215 yr I'm having issues with my server locking up and becoming unresponsive. What I know so far: My server locks up consistently every 2- 3 days, just long enough to get through the Parity check When this happens I cannot ssh into the box and the UI/dockers/etc.. are unresponsive. I don't remember changing anything specific but it started happening a few weeks ago I've disabled all new dockers that were installed since then to try and isolate the issue with no luck I turned on the Method 1 recommend by @Squid in this forum post which I recently checked after the server rebooted and it didn’t show anything out of the ordinary other than a ton of "shfs: share cache full" msgs Does anyone have any guidance on how to troubleshoot this issue. Also attached my diagnostics. Thx unraider-diagnostics-20210219-1852.zip
February 20, 20215 yr Author Thx, I just enabled the Local syslog sever and hopefully that will give me more info after the next lockup.
February 21, 20215 yr Author Just had another lockup requiring a hard reboot and there doesn’t seem to be anything helpful in the logs as they also seemed to have locked up/stopping updating for a few hours? Feb 20 16:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 16:47:01 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 16:47:05 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 16:47:05 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 17:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 17:47:02 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 17:47:06 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 17:47:06 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 18:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 18:47:02 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 18:47:06 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 18:47:06 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 19:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 19:47:02 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 19:47:06 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 19:47:06 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 20:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 20:47:01 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 20:47:06 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 20:47:06 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 21:47:01 UnRaider root: Fix Common Problems Version 2021.02.18 Feb 20 21:47:02 UnRaider root: Fix Common Problems: Warning: Share backup set to not use the cache, but files / folders exist on the cache drive ** Ignored Feb 20 21:47:06 UnRaider root: Fix Common Problems: Warning: Docker application duckdns2 has moderator comments listed ** Ignored Feb 20 21:47:06 UnRaider root: Fix Common Problems: Warning: Docker application letsencrypt has moderator comments listed ** Ignored Feb 20 23:39:43 UnRaider cache_dirs: Arguments=-i backup -i domains -i downloads -i files -i isos -i multimedia -i nextcloud -i sync -i sync-downloads -i windows Feb 20 23:39:43 UnRaider cache_dirs: Max Scan Secs=10, Min Scan Secs=1 Feb 20 23:39:43 UnRaider cache_dirs: Scan Type=adaptive Feb 20 23:39:43 UnRaider cache_dirs: Min Scan Depth=4 Feb 20 23:39:43 UnRaider cache_dirs: Max Scan Depth=none Feb 20 23:39:43 UnRaider cache_dirs: Use Command='find -noleaf' Feb 20 23:39:43 UnRaider cache_dirs: ---------- Caching Directories --------------- Feb 20 23:39:43 UnRaider cache_dirs: backup Feb 20 23:39:43 UnRaider cache_dirs: domains Feb 20 23:39:43 UnRaider cache_dirs: downloads Feb 20 23:39:43 UnRaider cache_dirs: files Feb 20 23:39:43 UnRaider cache_dirs: isos Feb 20 23:39:43 UnRaider cache_dirs: multimedia Feb 20 23:39:43 UnRaider cache_dirs: nextcloud Feb 20 23:39:43 UnRaider cache_dirs: sync Feb 20 23:39:43 UnRaider cache_dirs: sync-downloads Feb 20 23:39:43 UnRaider cache_dirs: windows Feb 20 23:39:43 UnRaider cache_dirs: ---------------------------------------------- Feb 20 23:39:43 UnRaider cache_dirs: Setting Included dirs: backup,domains,downloads,files,isos,multimedia,nextcloud,sync,sync-downloads,windows Feb 20 23:39:43 UnRaider cache_dirs: Setting Excluded dirs: Feb 20 23:39:43 UnRaider cache_dirs: min_disk_idle_before_restarting_scan_sec=60 Feb 20 23:39:43 UnRaider cache_dirs: scan_timeout_sec_idle=150 Feb 20 23:39:43 UnRaider cache_dirs: scan_timeout_sec_busy=30 Feb 20 23:39:43 UnRaider cache_dirs: scan_timeout_sec_stable=30 Feb 20 23:39:43 UnRaider cache_dirs: frequency_of_full_depth_scan_sec=604800 Feb 20 23:39:43 UnRaider cache_dirs: cache_dirs service rc.cachedirs: Started: '/usr/local/emhttp/plugins/dynamix.cache.dirs/scripts/cache_dirs -i "backup" -i "domains" -i "downloads" -i "files" -i "isos" -i "multimedia" -i "nextcloud" -i "sync" -i "sync-downloads" -i "windows" 2>/dev/null' Feb 20 23:39:43 UnRaider root: Delaying execution of fix common problems scan for 10 minutes Feb 20 23:39:43 UnRaider unassigned.devices: Mounting 'Auto Mount' Devices... Feb 20 23:39:43 UnRaider unassigned.devices: Adding disk '/dev/sdb1'... Feb 20 23:39:43 UnRaider unassigned.devices: Mount drive command: /sbin/mount -t xfs -o rw,noatime,nodiratime,discard '/dev/sdb1' '/mnt/disks/ssd' Feb 20 23:39:43 UnRaider kernel: XFS (sdb1): Mounting V5 Filesystem Feb 20 23:39:43 UnRaider kernel: XFS (sdb1): Starting recovery (logdev: internal) Feb 20 23:39:43 UnRaider kernel: XFS (sdb1): Ending recovery (logdev: internal) Feb 20 23:39:43 UnRaider unassigned.devices: Successfully mounted '/dev/sdb1' on '/mnt/disks/ssd'. Feb 20 23:39:43 UnRaider unassigned.devices: Disk with serial 'WDC_WD140EDFZ-11A0VA0_9LG83JPA', mountpoint 'easystore' is not set to auto mount. Feb 20 23:39:43 UnRaider emhttpd: Starting services... Feb 20 23:39:43 UnRaider emhttpd: shcmd (65): /etc/rc.d/rc.samba restart Feb 20 23:39:43 UnRaider rsyslogd: [origin software="rsyslogd" swVersion="8.1908.0" x-pid="6884" x-info="https://www.rsyslog.com"] start Feb 20 23:39:45 UnRaider root: Starting Samba: /usr/sbin/smbd -D Feb 20 23:39:45 UnRaider root: /usr/sbin/nmbd -D Feb 20 23:39:45 UnRaider root: /usr/sbin/wsdd Feb 20 23:39:45 UnRaider root: /usr/sbin/winbindd -D Feb 20 23:39:45 UnRaider emhttpd: shcmd (79): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 50 Feb 20 23:39:45 UnRaider kernel: BTRFS: device fsid a8e1d964-f72d-42b6-a377-79a34367d3fd devid 1 transid 2223184 /dev/loop2 Feb 20 23:39:45 UnRaider kernel: BTRFS info (device loop2): disk space caching is enabled Feb 20 23:39:45 UnRaider kernel: BTRFS info (device loop2): has skinny extents Feb 20 23:39:45 UnRaider kernel: BTRFS info (device loop2): start tree-log replay Feb 20 23:39:45 UnRaider kernel: BTRFS info (device loop2): checking UUID tree Feb 20 23:39:45 UnRaider root: Resize '/var/lib/docker' of 'max' Feb 20 23:39:45 UnRaider kernel: BTRFS info (device loop2): new size for /dev/loop2 is 53687091200 Feb 20 23:39:45 UnRaider emhttpd: shcmd (81): /etc/rc.d/rc.docker start Feb 20 23:39:45 UnRaider root: starting dockerd ... Really at a loss on how to stabilize my server 😩 Also attached the diagnostic taken after the server booted back up. unraider-diagnostics-20210220-2347.zip Edited February 21, 20215 yr by unRaide
February 21, 20215 yr Community Expert 2 hours ago, unRaide said: and there doesn’t seem to be anything helpful in the logs That suggests a hardware issue, one other thing you can try it to boot the server in safe mode with all docker/VMs disable, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.
August 17, 20223 yr Author I didn’t unfortunately. Just had the server lock up on me again earlier today. Sad to say but I just hard reboot it every few days 😩
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.