March 11, 20251 yr Hi, since Version 7 it happened the second time now. That my server freeze. On the first time couldn't troubleshoot it, because I haven't set a syslogserver. This time I have, but I don't see anything suspicious. The first time I could login anymore. But with an external screen I could see a loginpromt, but the login won't get any feedback. After pressing ctrl alt del a process seems to hang. But also no response anymore. After I checked the grafana graphs I saw that after 02:00 wasn't anything logged anymore. This time the entire Server has not given any response. The display was just black, no loginpromt. The disk LED was always on?! Anyways it looks like grafana or prometheus could log at least some metrics. I checked the new syslogfiles and see that at 02:26 there are many docker network entries. Mar 10 22:32:35 cakestation Docker Auto Update: No updates will be installed Mar 11 02:25:07 cakestation kernel: veth9b5ce00: renamed from eth0 Mar 11 02:25:54 cakestation kernel: eth0: renamed from veth4fb02a8 Mar 11 02:26:01 cakestation kernel: vethf199325: renamed from eth0 Mar 11 02:26:01 cakestation kernel: docker0: port 18(veth717551c) entered disabled state Mar 11 02:26:01 cakestation kernel: docker0: port 18(veth717551c) entered disabled state Mar 11 02:26:01 cakestation kernel: veth717551c (unregistering): left allmulticast mode Mar 11 02:26:01 cakestation kernel: veth717551c (unregistering): left promiscuous mode Mar 11 02:26:01 cakestation kernel: docker0: port 18(veth717551c) entered disabled state Mar 11 02:26:01 cakestation kernel: docker0: port 18(vethf7fd347) entered blocking state Mar 11 02:26:01 cakestation kernel: docker0: port 18(vethf7fd347) entered disabled state Mar 11 02:26:01 cakestation kernel: vethf7fd347: entered allmulticast mode Mar 11 02:26:01 cakestation kernel: vethf7fd347: entered promiscuous mode Mar 11 02:26:01 cakestation kernel: docker0: port 18(vethf7fd347) entered blocking state Mar 11 02:26:01 cakestation kernel: docker0: port 18(vethf7fd347) entered forwarding state Mar 11 02:26:02 cakestation kernel: eth0: renamed from veth8867d93 This behavior went further till 04:05 since this time, there was trouble with usb devices Mar 11 04:05:15 cakestation kernel: usb 1-5: USB disconnect, device number 13 Mar 11 04:05:25 cakestation kernel: usb 1-5: new full-speed USB device number 14 using xhci_hcd Mar 11 04:05:25 cakestation kernel: cdc_acm 1-5:1.0: ttyACM0: USB ACM device Mar 11 04:05:25 cakestation kernel: usb 1-5: USB disconnect, device number 14 Mar 11 04:05:25 cakestation kernel: usb 1-5: new full-speed USB device number 15 using xhci_hcd Mar 11 04:05:25 cakestation kernel: usb 1-5: unable to read config index 0 descriptor/all Mar 11 04:05:25 cakestation kernel: usb 1-5: can't read configurations, error -110 Mar 11 04:05:25 cakestation kernel: usb 1-5: new full-speed USB device number 16 using xhci_hcd Mar 11 04:05:30 cakestation kernel: usb 1-5: unable to read config index 0 descriptor/all Mar 11 04:05:30 cakestation kernel: usb 1-5: can't read configurations, error -110 Mar 11 04:05:32 cakestation kernel: usb usb1-port5: attempt power cycle Mar 11 04:05:32 cakestation kernel: usb 1-5: new full-speed USB device number 17 using xhci_hcd Mar 11 04:05:32 cakestation kernel: cdc_acm 1-5:1.0: ttyACM0: USB ACM device I am not sure, but this could be my zigbee Stick. This went further til 06:06 which is also the last log entry. At 06:55 I had to hard reboot the server. Suspicious is, that both times it was something around 02:00 in the night. cakestation-diagnostics-20250311-1008.zip
March 11, 20251 yr Community Expert 3 hours ago, trashit said: syslogserver zip and attach that syslog 3 hours ago, trashit said: Suspicious is, that both times it was something around 02:00 in the night. What do you get from command line with this? cat /etc/cron.d/root
March 11, 20251 yr Author 3 hours ago, trurl said: cat /etc/cron.d/root # Generated system monitoring schedule: */1 * * * * /usr/local/emhttp/plugins/dynamix/scripts/monitor &> /dev/null # Generated mover schedule: 0 11 * * 2 /usr/local/sbin/mover start &> /dev/null # Generated Unraid OS update check schedule: 11 0 * * 1 /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/unraidcheck &> /dev/null # Appdata.Backup cron settings 25 02 * * 2 php /usr/local/emhttp/plugins/appdata.backup/scripts/backup.php > /dev/null 2>&1 # Generated cron settings for docker autoupdates 32 22 * * * /usr/local/emhttp/plugins/ca.update.applications/scripts/updateDocker.php >/dev/null 2>&1 # Generated cron settings for plugin autoupdates 0 0 * * 0 /usr/local/emhttp/plugins/ca.update.applications/scripts/updateApplications.php >/dev/null 2>&1 # CRON for CA background scanning of applications 46 * * * * php /usr/local/emhttp/plugins/community.applications/scripts/notices.php > /dev/null 2>&1 # Generated cron schedule for user.scripts 20 11 * * 3 /usr/local/emhttp/plugins/user.scripts/startCustom.php /boot/config/plugins/user.scripts/scripts/rsync-appdata/script > /dev/null 2>&1 30 11 * * 3 /usr/local/emhttp/plugins/user.scripts/startCustom.php /boot/config/plugins/user.scripts/scripts/rsync-syncthing/script > /dev/null 2>&1 02:25 could be the appdata-backup cause this problem? the backup is set to backup weekly... today syslog.zip Edited March 11, 20251 yr by trashit
March 25, 20251 yr Author it happened again. Two times. Once friday the 21.03 some when around 16:30 and this night some when around 02:30 this night the appdatabackup ran. On Friday it was something else. Seems it's really something with docker. On Friday I worked a bit with openwebui Container. This night it looks like immich container made a bit CPU load. But I still don't have a clue what breaks the entire system or where I can look for more informations. syslog-192.168.178.2-2.log cakestation-diagnostics-20250325-0753.zip
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.