Unraid 6.7.2 - Crashing with new Hardware

[Steve Absolon]

Hello Community,

This is the first time I have had to post here since using Unraid for over 10yrs now without any major issues. I have one now which is why I am turning to the forum to see if I can get some more insight.

Due to a power surge which took out my PS + Motherboard I had to replace the kit and ever since doing so I the system is randomly crashing most times during a Parity Check. I have updated the BIOS on the motherboard to the latest available as well, I tried the latest Unraid 6.8.0 and the same thing occurred. I have setup Kiwi Syslog on my desktop pc to capture the logs from the server for analysis. I have also installed the "Fix Common Problems" plugin and corrected where possible still no change.

 

I have also started unraid in safe mode and this still issue still occurs.

 

Attached is an output from SysLog during what I believe was a crash event and the diags as well. 

 

MB: ASUSTeK COMPUTER INC. TUF B450M-PLUS GAMING Version Rev X.0x

BIOS: American Megatrends Inc. Version 2006. Dated: 11/13/2019

CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics @ 3600 MHz

HVM: Enabled

IOMMU: Enabled

Cache: 384 KiB, 2048 KiB, 4096 KiB

Memory: 8 GiB DDR4 (max. installable capacity 128 GiB)

Network: bond0: fault-tolerance (active-backup), mtu 1500
 eth0: 1000 Mbps, full duplex, mtu 1500

Kernel: Linux 4.19.56-Unraid x86_64

OpenSSL: 1.1.1c

 

Power Supply: CORSAIR VS550

Parity: Seagate 6TB BarraCuda 3.5" Hard Drive, SATA III, 256MB (ST6000DM003)
Disk 1: Seagate 4TB BarraCuda 3.5" SATA3 Desktop Hard Drive (ST4000DM005)
Disk 2: Seagate Barracuda 3TB 7200RPM SATA 3Gbps 32MB Cache 3.5-inch (ST3000DM001)
Disk 3: Seagate Barracuda 3TB 7200RPM SATA 3Gbps 32MB Cache 3.5-inch (ST3000DM001)
Disk 4: Seagate ST3000DM007 3TB BarraCuda 3.5" 5400RPM SATA3 Desktop Hard Drive - 256MB (ST3000DM007)

Flash: Samsung 32gig USB3

 

Thanks all in advance.

 

SysLog_Output.txt tower-diagnostics-20191229-2346.zip

Edited January 8, 2020 by Steve Absolon
Add in MB Details

[trurl]

On mobile now so can't look at Diagnostics yet. Have you done memtest? 

[Steve Absolon]

Sorry - Yeah done this as well for 24hrs no errors, I thought the same thing ie. Memory may be at fault. I left that bit out dam, I am a IT Sys Admin so know what it's like to try and provide as much info as I can to a job.

Thanks for responding.

 

[FreeMan]

You don't list your PSU or your drives. I was having issues at one point when I was drawing more power than a dying PSU could supply. List full machine specs, someone may spot an issue.

[Steve Absolon]

Updated first entry with the details

[Steve Absolon]

Here is another log when the system crashed this morning. Interestingly Kernel indicated out of memory condition. Don't know how this could be the case.

Another post similar to this error was fixed when the user upgraded to a new Unraid Version.

I wonder if there is actually a compatibility issue with this AMD CPU and the Kernel.

 

2019-12-31 07:26:02    Kernel.Error    10.0.0.254    Dec 31 07:26:01 Tower kernel: Out of memory: Kill process 10610 (monitor) score 1 or sacrifice child
2019-12-31 07:26:02    Kernel.Error    10.0.0.254    Dec 31 07:26:01 Tower kernel: Killed process 10610 (monitor) total-vm:75172kB, anon-rss:1108kB, file-rss:4kB, shmem-rss:9204kB
2019-12-31 07:26:02    Kernel.Info    10.0.0.254    Dec 31 07:26:01 Tower kernel: oom_reaper: reaped process 10610 (monitor), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
2019-12-31 07:26:02    Cron.Notice    10.0.0.254    Dec 31 07:26:01 Tower crond[1791]: exit status 137 from user root /usr/local/emhttp/plugins/dynamix/scripts/monitor &> /dev/null

SysLog_Output_Crash2.txt

[trurl]

Are you sure you don't have something filling RAM?

 

It is possible for dockers to fill RAM depending on their mappings. If you have a host path that isn't actual storage, then that path is in RAM since the Unraid OS is in RAM and only mounted storage is not RAM. Note that any Unassigned Device mapping that is not actually mounted is also a path in RAM.

[Steve Absolon]

18 hours ago, trurl said:

Are you sure you don't have something filling RAM?

 

It is possible for dockers to fill RAM depending on their mappings. If you have a host path that isn't actual storage, then that path is in RAM since the Unraid OS is in RAM and only mounted storage is not RAM. Note that any Unassigned Device mapping that is not actually mounted is also a path in RAM.

This is interesting you may be onto something now. Since I have more powerful hardware then before I am using Plex which has SMB mounts to another machine on my network for TV Recordings. What I will do tomorrow is go buy another 8 gig of RAM to remove this as a possibility. It does make sense that this could be the cause.

Will update this thread in 48hrs after I install the next stick of RAM.

 

[trurl]

Increasing RAM is not the solution to a mapping in RAM. Why would you want to allow anything to write to RAM where it will be lost?

[Steve Absolon]

Ok all,

I didn't add any additional RAM, I figured I would keep the Plex Docker shutdown and it's been two days running without an issue. I turned the Plex app back on this afternoon and sure enough the system crashed again.

I have added additional paths within the docker which uses a path created by the UDEV plugin to mount a SMB share from my media pc where my TV Recordings are held, not sure if this mapping is causing the problem as I would expect that if something was wrong with the Plex package others would be seeing issues as well.

Will keep testing.

 

[Steve Absolon]

Well I am still testing but it now I have identified that running MEMTest in multicore mode seems to also lock up the machine. 

With this knowledge I again re-ran MEMTest without multiCore mode enabled for 48hrs without lockups.

 

I came across the asus forum entry below which talks of others also having stability issues with this CPU and AMD's technology around CPU idle mode and voltage to the chips. I have now disabled a number of options in the BIOS and switched the bios to operating the CPU with two core leveling. This now has altered my CORE count in Unraid from 4 to 2. This in my mind should keep the CPU active enough to avoid any further idle power states.

http://forum.asrock.com/forum_posts.asp?TID=7890&PN=1&title=1800x-x370-taichi-freezing-issue

 

Will update you all again in coming days.