Server crashing almost daily, syslog mirrored to flash

[J05u]

Hello, my server crashing almost on daily basis 

Not sure if this is due to ram issue

After few ocasions i setup to mirror syslog to flash, no idea how to extract it

Here is my diagnostics, maybe anyone can advice what caused those crashes. I suspect RAM, but before RC4 ram was rock solid

fortress-diagnostics-20220408-0705.zip

[J05u]

Anyone..?

[J05u]

Still trying to figure out what happening with server

[trurl]

On 4/8/2022 at 2:08 AM, J05u said:

mirror syslog to flash, no idea how to extract it

You can access flash contents over the network if you share it on the network. Or you can put it in your PC after crash to get the logs.

 

Not related, but why do you have 100G docker.img? Have you had problems filling it? 20G is often more than enough, maybe a little more depending on which and how many dockers. I run 15-20 containers and they take about half of 20G.

[J05u]

 

31 minutes ago, trurl said:

You can access flash contents over the network if you share it on the network. Or you can put it in your PC after crash to get the logs.

 

Not related, but why do you have 100G docker.img? Have you had problems filling it? 20G is often more than enough, maybe a little more depending on which and how many dockers. I run 15-20 containers and they take about half of 20G.

To be honest can't remember, not bothering me. I think i messed up something, after i figure this out didn't change it back

I found syslog, can i add it here? Can't get why my server crashed/restarted on 08:38

syslog

Edited April 11, 2022 by J05u

[trurl]

Looks like it was rebooted here:

Apr 11 08:07:53 Fortress emhttpd: spinning down /dev/sde
Apr 11 08:39:05 Fortress kernel: microcode: microcode updated early to revision 0xec, date = 2021-04-28

Was that after crash?

[J05u]

Looks like it was rebooted here:

Apr 11 08:07:53 Fortress emhttpd: spinning down /dev/sdeApr 11 08:39:05 Fortress kernel: microcode: microcode updated early to revision 0xec, date = 2021-04-28

Was that after crash?

I wasn’t at home, I just noticed that parity checked started, this is why I am curious what happened.
I have ups, so I believe so shouldn’t be issues with power


Отправлено с моего iPhone используя Tapatalk

[J05u]

I just calculated by uptime from Unraid dashboard and seems like it’s correct. But still can’t get what happened why was that restart


Отправлено с моего iPhone используя Tapatalk

[J05u]

Again crashed, memtest didn't find any issues

Went back to RC3

[trurl]

On 4/11/2022 at 6:42 PM, J05u said:

I have ups, so I believe so shouldn’t be issues with power

Is the UPS monitored by Unraid so it can shutdown cleanly if power is lost? I even get email notifications from my server when my UPS detects a drop.

[J05u]

4 minutes ago, trurl said:

Is the UPS monitored by Unraid so it can shutdown cleanly if power is lost? I even get email notifications from my server when my UPS detects a drop.

I need to try to setup this notifications, but i am pretty sure that this not related with power cuts, as no one in house noticed any power drops. 

I am just trying all possible solutions to fix this problem, this is really annoying. I want to transfer files to bigger disks, but i can't risk due to this daily crashes...

[trurl]

Just now, J05u said:

need to try to setup this notifications

Do you not have any notifications setup? You must setup Notifications so Unraid can alert you by email or other agent as soon as a problem is detected. Don't let one unnoticed problem become multiple problems that are difficult to recover from.

[J05u]

On 4/14/2022 at 2:08 PM, trurl said:

Do you not have any notifications setup? You must setup Notifications so Unraid can alert you by email or other agent as soon as a problem is detected. Don't let one unnoticed problem become multiple problems that are difficult to recover from.

I reverted to RC3, again couple of crashes, and can't see nothing in syslog on flash.

I will try to disconnect UPS

 

syslog

[J05u]

On 4/16/2022 at 6:55 PM, J05u said:

I reverted to RC3, again couple of crashes, and can't see nothing in syslog on flash.

I will try to disconnect UPS

 

syslog 1.62 MB · 0 downloads

I disconnected ups, changed all Ram, after 1.5 day of running server hang and now is happening parity check 

can anybody please to check syslog or at least direct me somewhere, I don’t have a clue what is happening 

[J05u]

Found old thread about IGPU issue, i will try to disable also IGPU...

 

 

[J05u]

Yesterday i got this issue. I identified that server actually boots, but i don't have access to WEB UI, even IP is correct

Today i restored flash through my backup and will see how stable system will stay

[J05u]

I don’t know if update from r3 to rc4 corrupted system, but so far server running fine around 72 hours