System Hard Crashes [6.12.3]

[gloworm]

Upon upgrading to 6.12.2/6.12.3, I've begun having hard crashes. 'Nothing on the monitor but a few random characters, no signs of life, but the power is still on' crashes. I did enable logging, but the most recent crash didn't log the data for some reason, and previous crashes nothing is written to the log file, meaning the last lines are normal outputs. This led me down the road of possibly being a hardware issue. The only counter argument I have to that, is that every time it has happened between 1 and 4 am local time. Awfully convenient timing for a hardware issue.

 

At first I thought an IO error of my cache drive could have been causing a freak out. Wiped, stress tested the drives, and rebuilt the cache pool.

This morning, I woke up with the crash again. Tried to reboot and it couldn't find my flash drive. Investigated and found contents were fine, temporarily inserted into another port and booted into memtest86 (Going to go ahead and do a RAM check to cover my bases on hardware)

 

I thought about downgrading, but I had the same issue with 6.12.2 although I didn't have in on there for long. 6.11 was my last known 100% config.

 

So my questions are these,

 

1) With the exception of logging to Flash for syslog, does Unraid access the USB and could not being able to access the USB cause a hard crash? (If memory serves, Unraid loads all contents into RAM and runs via RAM; please confirm or deny)

2) Since I'm not getting software logs as the system crashes and effectively nothing is displayed on the monitor output, what ideas is there to try to resolve the issue?

 

Correct me if I'm wrong, but my next step if the RAM test comes back good would be to start in safe mode, start the array, and see how it performs. Just to rule out a plugin issue.

 

Please give some ideas!

 

P.S. As soon as I get a current log file (post cache rebuild), I'll post it.

Edited July 20, 2023 by gloworm
Added to Question

[itimpi]

The other thing you should do is enable the syslog server to get logs that survive the crash/reboot to see if they show anything useful.

[gloworm]

19 minutes ago, itimpi said:

The other thing you should do is enable the syslog server to get logs that survive the crash/reboot to see if they show anything useful.

 

43 minutes ago, gloworm said:

I did enable logging, but the most recent crash didn't log the data for some reason, and previous crashes nothing is written to the log file, meaning the last lines are normal outputs.

 The logs are being saved; but the last appendages to the log file are normal outputs. The exception was the most recent crash which the entire log file was lost.

[gloworm]

I have not experienced any crashes since I temporarily moved my Unraid USB Drive to a different port. Which doesn't make sense. I need to ask this question again,

 

"With the exception of logging to Flash for syslog, does Unraid access the USB and could not being able to access the USB cause a hard crash? (If memory serves, Unraid loads all contents into RAM and runs via RAM; [[[please confirm or deny]])"