August 27Aug 27 Hello,Since a week I have frequent kernel panics, the uptime is < 24h.No hardware or software changes.Motherboard BIOS is up to date.My server is headless so I do not have a trace of the kernel messages.I do have a partial feedback from my kvm over ip :For now I tried :memtest86, OK.Disabling uneeded containers, no effectFollowing the latest kernel panic I disabled c-states and energy related features in the bios as the kernel messages seems to be energy efficiency related.unraid-diagnostics-20260826-1700.zip Edited August 27Aug 27 by uncured592
August 27Aug 27 Author I have an external syslog on another host.Nothing in the logs, the server just stop logging, here it is :I suspect the kernel panic happened after 01h34.There are some issues before that (a faulty disk connectivity with ata8) but it was 30 minutes before. Edited August 27Aug 27 by uncured592
August 27Aug 27 Community Expert Still post that log, there may be some clues before the crash, but if it's a hardware issue, typically there's nothing logged.
August 27Aug 27 Author Here's the unfiltered syslog from the external host.And the kernel panic time I remember : 27/08/2026 around 01h3426/08/2026 at 04h05 syslog.zip
August 27Aug 27 Community Expert The image does not contain the kernel panic itself, but it confirms another ata8 failure at 01:01:25. This is the parity disk, and the log shows a FLUSH CACHE EXT timeout, followed by a SATA link reset. The connection recovered, and the server continued logging for approximately another 33 minutes, so this does not prove that the disk-link problem caused the later crash. Nevertheless, it is a real hardware-path issue that should be addressed.The longer syslog also contains two earlier kernel faults: - On August 21, the first Oops occurred in the core memory-management function pagetable_dtor_free(), involving an invalid low-memory pointer. - On August 23, another Oops occurred in the ZFS function arc_buf_destroy(), involving the same invalid pointer. This was Oops [#2] without an intervening reboot, so it may have resulted from damage caused by the first fault rather than being an independent ZFS failure.The latest August 26 and 27 logs simply stop around the reported crash times without preserving the panic. I found no MCE/EDAC hardware-error report, PCIe AER error, critical temperature event, global out-of-memory condition, or other clear cause. The logged container OOM events were limited to individual container memory limits and do not indicate that the host ran out of memory.The parity-disk path has recorded repeated command timeouts and SATA link errors over an extended period. SMART does not show bad sectors or interface CRC errors, which makes an intermittent cable, power connection, port, backplane, controller, or link-power problem more likely than a disk-media problem.Taken together, faults in different kernel paths, the repeated impossible pointer value, and the absence of one consistent software trigger, this currently looks to me more like hardware/platform instability or memory corruption.I recommend proceeding as follows: 1. Replace or reseat both the SATA data and power connections for the parity disk. If possible, use a different controller port and temporarily avoid shared power splitters or backplanes. 2. Load BIOS defaults and disable XMP, overclocking, undervolting, or other performance tuning. 3. Boot Unraid in Safe Mode, ensuring the Powertop script and other third-party power-management changes are not applied. 4. Initially test with the array stopped. If stable for at least 24 hours, start the array while leaving Docker and VMs disabled. 5. If that remains stable, enable Docker and bring containers back in small groups. 6. Despite the previous successful Memtest result, consider another extended test. Testing one DIMM at a time may help identify an intermittent DIMM, slot, CPU memory-controller, or motherboard issue. 7. If another crash occurs, capture the beginning of the panic directly from the local console or NanoKVM. The initial BUG/Oops lines, CPU/task information, registers, and complete call trace are the most important parts.For now, I would prioritize correcting the parity-disk connection and testing the system with default firmware and Safe Mode settings. The SATA fault could be separate from the crashes, but it needs to be eliminated before we can interpret any further failures reliably.
August 27Aug 27 Author Thank you !I'll stay like this for a while to see if I have a kernel panic again (most likely tonight), then I'll follow your list.But I'll replace the sata cable for sure.Update : No kernel panic for the latest 24h and I was able to complete the parity check, as a side note there is also no more connectivity issue with my parity disk anymore. Edited August 28Aug 28 by uncured592
September 23Sep 23 Author Small update : I have been running for two weeks without any issue, then I had more and more kernel panic, the server could not even make it to 24h of uptime, sometime a couple of hours.Of course it happened when I was not at home, I could barely reboot the server with my nanokvm with tailscale help.Anyway, I unracked the 4U server and made some cleanup :Change of the thermal paste of the CPU and I moved from the default intel cpu cooler to a Thermalright Phantom Spirit 120 SE.Changing the faulty sata cable.I have some doubt on the intel provided cpu cooler I was using :While unscrewing it I noticed that one of its mounting pins wasn’t properly inserted into the motherboard, the pressure wasn’t evenly distributed across the CPU die.When I had those kernel panics, once I kept a nanokvm session opened on the unraid GUI and I'm pretty sure it shown a temp at 95°C at the bottom of the ui.I also have some doubt on the CPU itself, as the 14700 has some thermal issue, but I couldn't find anything conclusive with stress-ng.The memory has also been tested with memtest86 (the version embedded with unraid) but I couldn't make it crash.The server has been restarted and I'm now waiting for to see if it is crashing again.
September 25Sep 25 Author Two days later and still no issue despite some heavy load from a jellyfin stack I forgot to shut down.syslog/dmesg look clean. The parity check has finished with 3048 errors which is kind of expected considering all the kernel panic I had in the past week.Overall I think the culprit was the CPU cooler that was not correctly mounted which reduced its efficiency.The kernel panics were mostly during the night when the plex/jellyfin workload was happening (opening detection, ...) so precisely when the load was high on the p-cores.I'll have to find a way to send notifications in case of high temperature.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.