Unraid server keeps freezing up randomly

[faroh]

Hello

 

My unRAID server seems to freeze up every once in awhile on it's own. I can't reach the webGUI or SSH into the server. I'm not sure what's going on but I have tried several things that have worked for other people such as changing docker network from macvlan to ipvlan. I also tried disabling c state in the bios (I have a Ryzen 9 processor). Any advice would be appreciated to resolve this issue.

 

tower-diagnostics-20230809-1417.zip

[JorgeB]

Enable the syslog server and post that after a crash.

 

[faroh]

So the crash just happened again. One thing I noticed is when the server starts up and I start the array it takes forever for the first SSH to load my profile. Also same thing happens when I run a basic "ls". The single core is pegged for almost a full minute (see screenshots). After that it operates smoothly.

 

I attached syslog as well as diagnostics.

 

Thanks,

Faroh

syslog-10.0.1.10.log tower-diagnostics-20230817-1353.zip

[JorgeB]

There's nothing relevant logged, CPU usage appears to be this:

 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
18904 root      20   0 3572604 100656  18424 S  25.0   0.2   0:44.39 frigate.p+

 

[faroh]

O that's interesting. I'll check it out and see what's causing it.

Any recommendations for the freezing up issue? 

[JorgeB]

Not really, there's nothing relevant logged that I can see.

[faroh]

Is there a way to enable a more verbose log level for unraid? Nothing coming up from a few quick searches.

Edited August 18, 2023 by faroh

[faroh]

I also saw some references to max memory speed and max memory. Any thoughts on if my memory speed is set too high (using default settings) or if I have to much memory on the server (64 gigs).

[faroh]

So it just happened again (been happening about once every other week at this point). Checking the syslogs I think this might be memory related?

Look at the timestamp `Sep 11 15:13:17`

syslog-10.0.1.10.log

[JorgeB]

Multiple call traces logged, look more hardware related, try using just two RAM sticks, if the same try the other two, that will basically rule out a RAM problem, also note that there have been various reports of stability issues with Ryzen 7000, possibly just too new for the kernel.

[faroh]

Ahh that's unfortunate. Alright I'll test out the RAM sticks and see what's going on.

[JorgeB]

Also look for a BIOS update.

[faroh]

Sounds good - I'll check for BIOS version update next.
So I ran memtest86 and no errors after 2 passes. I did notice my memory by default is running at almost 4700 MHz. Could that be causing issues? Should I consider lowering the speed below the default? Reading up on the Ryzen 9 specs it looks like if I have 4 sticks I shouldn't be running faster than 3600 MHz.

 

https://www.amd.com/en/products/cpu/amd-ryzen-9-7900x#:~:text=vary by manufacturer.-,Connectivity,-System Memory Type

I'll try reducing that and see things stabilize

[JorgeB]

DDR5 stock speed for that CPU is 4800MT/s IIRC.

[Squid]

3 minutes ago, JorgeB said:

DDR5 stock speed for that CPU is 4800MT/s IIRC.

SPD speed on the RAM is also 4800, and the diagnostics show you're running at 3600.

[faroh]

Cool, I should be good in terms of mem speed then. I did a bios update to latest today and I put a memory limit on my frigate containers in hopes that might help. I'll update here in a week or two if it doesn't happen again.

Edited September 12, 2023 by faroh

[faroh]

Update on this. I think my issue was my frigate container going crazy. Once I added a mem limit on it I haven't seen the crash since. So maybe that was the problem. I'll update again if I have another crash.