January 23, 20251 yr Hi guys, documenting my investigation here but of course interested in anyone who can spot the smoking gun. A little more than a month ago (early Dec, this crash is Jan 22), I had a server crash (first time in about a year or so), with no syslog captured so I turned it on and piped it to another Unraid server on my home network. Today, while away from home, the server became unresponsive sometime in the AM and I came home to find it hanging--made sure that there was an up-to-date syslog on the other server, and then shut down (uncleanly, held power). Unraid Version: 6.11.1 Intel® Core™ i7-6700K CPU @ 4.00GHz 48 GB DDR4 Mobo details ASUSTeK COMPUTER INC. TUF Z270 MARK 1 , Version Rev 1.xx American Megatrends Inc., Version 1301 BIOS dated: Wed 14 Mar 2018 12:00:00 AM PDT Powered by UPS (but I don't think it was a power blip...) The system log has from Dec to Jan 22 pretty much just 60k lines of of "synthetic_pathref opening [path] failed {sensitive info}", so I've just snipped the important tail here from the last log from a few days ago.syslog-192.168.1.105.log.tail.txt Diagnostics file (after the server was restarted, maybe not super useful.) castle-diagnostics-20250122-2322.zip The server is back on and beginning a parity check. As for my next steps, I fed the tail end of the syslog into chatgpt to see what it thought, and got some good direction; 1) Check drives for SMART report 2) Memtest 3) "Check logs for docker containers?" 4) Update all containers, BIOS, Unraid itself Based on these particular log lines, I think it's gotta be docker related: Jan 22 09:05:56 Castle kernel: kernel BUG at fs/inode.c:1760! Jan 22 09:05:56 Castle kernel: invalid opcode: 0000 [#1] PREEMPT SMP PTI Jan 22 09:05:56 Castle kernel: CPU: 7 PID: 5205 Comm: containerd-shim Not tainted 5.19.14-Unraid #1 I went looking to see what was going on around 9:00AM today, and saw that a family member was watching Plex (and transcoding) as recently as 10:00AM... not sure how it could be working then based on the logs, but maybe the server wasn't truly hanging until later in the day? It was around noon/12:00 when I noticed it was unresponsive. By the evening, I had some family members asking me if Plex was broken. When I got home I hard-powered off the server, maybe causing an unclean shutdown when I didn't need to. Curious to hear what anyone else thinks. Parity check will probably run till tomorrow.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.