Constant Crashes

[RyanServer711]

Joined the Unraid community and made my first server. I've been up and running for about 5 months with minimal issues.

 

Monday I noticed all my services were down and the machine was running. Complete lock up. Haven't been able to get it working since. Every time I log in it crashes within a minute or two. I've been able to boot in safe mode and it works enough to snoop around. 

Disabling docker seems to help but I still crash if I try to compute the drive size of my NDME drive. Starting docker results in a crash in about a minute.

 

I did notice in my logs that it crashed after this message.

Aug  2 18:32:27 Anton kernel: nvme nvme0: I/O 323 QID 4 timeout, aborting
Aug  2 18:32:57 Anton kernel: nvme nvme0: I/O 323 QID 4 timeout, reset controller
Aug  2 18:33:28 Anton kernel: nvme nvme0: I/O 24 QID 0 timeout, reset controller

 I have received that before crashing more than once. Bought another NVME drive in case that's the issue.

 

I updated my mobo bios and am running Unraid 6.10.3

 

Any insight on my issue would be greatly appreciated.

System Info.txt anton-diagnostics-20220802-1452.zip

Edited August 2, 2022 by RyanServer711
typo

[trurl]

setup syslog server

[RyanServer711]

8 hours ago, trurl said:

setup syslog server

Here is my current syslog file. Thanks

syslog.txt

[JorgeB]

There are some crashes/timeouts related to the NVMe device, please try running without it, keep the syslog server going and post a new one if it crashes again.

[RyanServer711]

I'll try that and grab the logs.

 

Thanks

[RyanServer711]

Removed the NVMe drive and everything seemed to be working fine, no crashes. Popped in the new drive and it was still running fine. Formated the drive and set as new cashe.

 

Unfortunately my Home Assistant VM and all my dockers are not there since they are on the old cashe. Tried loading a back up with the Appdata Backup/Restore plugin, which started a new round of instant boot crashes. I feel like I'm close to losing the hundreds of hours setting up all the dockers and home assistant. 

 

Attached are my syslogs.

syslog2.txt

[JorgeB]

Do you mean just restoring the backup to a different device made Unraid start to crash again?

[RyanServer711]

Yes, but after a couple power cycles it seems to be working. 

[JorgeB]

Something strange is going on there.

[RyanServer711]

Well I got all my dockers up and running. Hasn't crashed for some time now. I think I'm out of the woods. I'll contibute my problems to a badNVMe drive.

 

My issue now is restoring my Home Assistant VM.  I'm pretty sure I have all the VM files backed up just having trouble knowing where to plug everything in as far as the file structure. I'll make another post for that.

 

Thanks for the help

[RyanServer711]

Post realization....after I updated the bios, "Fast Boot" was enabled.

 

That was the reason I had trouble restarting.