Frequent Crashes After Migrating to New Hardware

[Ryanoc3ros]

I recently upgraded my server to a ASrock Z690 Extreme MB and Intel i5-13500 CPU. Since upgrading I have had numerous crashes after anywhere from 30 minutes to 9 hours. I have been running "stress" as a way to tax the system and bring out any instability. Below are my notes on all of the actions I have taken.

 

Jun 21 19:02       Startup

Jun 21 19:39       Crash (uptime 37 minutes). Crashed around the time I unplugged Z-wave stick and restarted Home Assistant VM

 

Last log entries before crash:

Jun 21 19:39:33 Snorlax kernel: docker0: port 5(veth5aa8f3e) entered disabled state
Jun 21 19:39:33 Snorlax kernel: veth4dd9902: renamed from eth0
Jun 21 19:39:33 Snorlax kernel: docker0: port 5(veth5aa8f3e) entered disabled state
Jun 21 19:39:33 Snorlax kernel: device veth5aa8f3e left promiscuous mode
Jun 21 19:39:33 Snorlax kernel: docker0: port 5(veth5aa8f3e) entered disabled state
Jun 21 19:39:34 Snorlax kernel: docker0: port 5(veth8079aaf) entered blocking state
Jun 21 19:39:34 Snorlax kernel: docker0: port 5(veth8079aaf) entered disabled state
Jun 21 19:39:34 Snorlax kernel: device veth8079aaf entered promiscuous mode
Jun 21 19:39:34 Snorlax kernel: docker0: port 5(veth8079aaf) entered blocking state
Jun 21 19:39:34 Snorlax kernel: docker0: port 5(veth8079aaf) entered forwarding state
Jun 21 19:39:34 Snorlax kernel: eth0: renamed from vetha164b17
Jun 21 19:39:34 Snorlax kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth8079aaf: link becomes ready

 

Jun 21 19:45       Ran Memtest86, Passed no errors

 

Jun 21 20:20       Startup

Jun 21 20:30       Disk 1 had several SMART errors, it is an old hard drive, so as a precaution, I started moving files from disk 1 to other disks with Unbalance

Jun 22 05:42      Crash (uptime 9 hours)

  

Last log entries before crash:

Jun 22 05:42:30 Snorlax kernel: usb 2-8.4: Device not responding to setup address.
Jun 22 05:42:30 Snorlax kernel: usb 2-8.4: Device not responding to setup address.
Jun 22 05:42:31 Snorlax kernel: usb 2-8.4: device not accepting address 3, error -71
Jun 22 05:42:31 Snorlax kernel: usb 2-8.4: reset SuperSpeed USB device number 3 using xhci_hcd
Jun 22 05:42:31 Snorlax kernel: usb 2-8.4: LPM exit latency is zeroed, disabling LPM.

 

 

Below is my log of everything I have done today (June 22). There is something noteworthy at "Jun 22 13:13" in the syslogs.

 

*10:23 Startup
11:45  Manual Shutdown
11:50  BIOS Updates:
         Turned off CPU C-States
         Set "Restore on AC Power Loss" to "Power On"
         Disabled HD Audio
         Long Duration Power Limit changed from Auto to 125 (not sure why)
11:50  Removed USB Coral, Z-Wave dongle, Zigbee Dongle, left USB drive in 3.0 slot

*11:55 Startup
12:00  Set Frigate to use CPU detector
12:00  Ran CPU stress test starting at 12:00
13:13  Kernel Panic? Didn't lock up entire system
13:13  Home Assistant locks up
13:15  Stopped stress test
13:15  Restarted HA
13:15  Continued to run stress test
*16:06 Crash (uptime 4 hours)

16:35  Restart and quick crash
16:45  Move USB Flash from USB 3 port to USB 2 port
16:46  Startup and run stress test
21:15  Turned off Pi-Hole Docker
*21:52 Crash (uptime 5 hours)

*22:16 Startup in GUI mode so I can watch directly attached monitor
22:19  Shutdown all dockers
22:24  Disabled Docker
22:30  Removed several plugins
22:32  Started stress test
*23:30 Crash, monitor glitching and flashing rapidly

Any help would be greatly appreciated. Thank you.

syslog-10.0.0.13.log snorlax-diagnostics-20240622-2338.zip

Edited June 23 by Ryanoc3ros

[JorgeB]

If the issue started after a hardware changes it suggests a hardware issue, I do see some kvm related call traces, so you still have stability issues if you don't use VMs and/or disabled virtualization in the board BIOS?

[Ryanoc3ros]

I have disabled all VMs and will monitor the system for a day or so. Assuming it is a VM issue, is it recommended to recreate the VM from scratch? Or will that not affect much?

[JorgeB]

I think it's worth a try to recreate one.