Need help deciphering logs from a recent crash

[Rhuarc]

Hello, I had a recent crash that I am trying to troubleshoot. Server was up and running fine, then it was not accessible any longer. I had my fiance take a photo of what was on the screen and then reboot. After the reboot I downloaded the diagnostic logs. Hoping someone here can help me out! 

 

galaxy-diagnostics-20230810-1844.zip

 

Thank you all in advance! 

[JorgeB]

The syslog in  the diags starts over after every reboot, if it keeps happening enable the syslog server and post that after a crash.

 

[Rhuarc]

syslog

 

Here is a syslog that includes the crashes I think.

 

The recurring thing I am seeing that I think might be the problem is this line:

 

May 10 18:49:56 GALAXY kernel: caller _nv000651rm+0x1ad/0x200 [nvidia] mapping multiple BARs
May 10 18:49:57 GALAXY kernel: resource sanity check: requesting [mem 0x000c0000-0x000fffff], which spans more than PCI Bus 0000:00 [mem 0x000c0000-0x000dffff window]
May 10 18:49:57 GALAXY kernel: caller _nv000651rm+0x1ad/0x200 [nvidia] mapping multiple BARs
May 10 18:49:58 GALAXY kernel: resource sanity check: requesting [mem 0x000c0000-0x000fffff], which spans more than PCI Bus 0000:00 [mem 0x000c0000-0x000dffff window]

 

It keeps repeating like this over and over again I believe around the time that the server crashes.

 

Edit:  I found this thread that talks about it possibly being related to BIOS low power states?  Seems weird since I haven't changed anything with the server other than upgrading to 6.12.  Previously it was rock solid.

 

 

Any help is greatly appreciated!  THank you in advance!

 

Edited August 12, 2023 by Rhuarc

[JorgeB]

Aug 11 10:49:32 GALAXY kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Aug 11 10:49:32 GALAXY kernel: ? _raw_spin_unlock+0x14/0x29
Aug 11 10:49:32 GALAXY kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Try switching to ipvlan (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)).

[Rhuarc]

Would this be something that could cause the crashes that I'm seeing? I did a little research into that and it seemed that a lot of people ended up with more problems after switch that than before. 

 

But I'll give anything a shot at this point! 

[JorgeB]

19 hours ago, Rhuarc said:

Would this be something that could cause the crashes that I'm seeing?

Yes, it's known to end up crashing the server after 1 or 2 days usually.

[Rhuarc]

Well, 2 and a half days in and so far no crashes! 

 

It's crazy that something as simple as that can be make the difference in stability that I'm seeing!