Unraid server unstable, trying to figure out what hardware might be failing

[gowg]

Can I post the log file here? It seems to have LAN IP addresses, is it safe to post?

 

Symptoms: server works fine for a couple hours but then all dockers stop and the docker service becomes unavailable, then the server becomes unresponsive.

 

Here are some choice bits from the log:

 

Jul  1 23:18:45 quad-unraid kernel: Code: 73 76 8b 0e 48 8d 5c c6 10 31 d2 89 f8 f7 f1 44 8b 0c 93 45 85 c9 74 60 44 89 c8 2b 46 04 83 cf 01 48 01 c8 89 fd 48 8d 1c 83 <44> 8b 23 48 83 c3 04 44 89 e0 83 c8 01 39 c5 75 2f 49 8b 42 58 45

This error appears dozens of times in a row, a different core each time (this error is the end of the log file)

and also

Jul  1 22:59:52 quad-unraid kernel: BTRFS info (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 0, flush 0, corrupt 65, gen 0

and

Jul  1 23:18:20 quad-unraid kernel: docker_load[7974]: segfault at f9 ip 0000000000457780 sp 00007ffc3812d1f0 error 4 in bash[426000+c5000] likely on CPU 15 (core 3, socket 0)

 

Edited July 8, 2023 by gowg

[JorgeB]

Start by running memtest.

[gowg]

Solved. I ran memtest and encountered thousands of errors within seconds. I pulled the ram sticks one by one, and the very last one was the culprit.

 

My server is stable now, thanks.

 

Edit: I did also have to run a parity check and restore some backups since the bad RAM corrupted a bunch of stuff.

Edited July 22, 2023 by gowg

[DirtyDarrell]

28 minutes ago, gowg said:

Solved. I ran memtest and encountered thousands of errors within seconds. I pulled the ram sticks one by one, and the very last one was the culprit.

 

My server is stable now, thanks.

I had a similar issue as this using OC ram.
Glad you got it figured out!

 

-Darrell