[SOLVED] 6.9-RC2 Kernal panic and trace


Recommended Posts

So a quick update to close this out as fixed.

 

- Ran all three servers with issues in safe mode and had no kernel panics and were stable.

- Began trying to troubleshoot down which docker container or plugin was causing the instability

- Found out it was the Technitum DNS docker container in community apps.  With one wrinkle, on a 1G connection it was stable.  On a 10G connection it generated kernel panics and spin lock errors.  The stock install was on BR0 so wonder if it was in the dockers config for networking.

- Once I removed the Technitum DNS server docker from the three servers they have been completely stable

 

  • Like 2
Link to comment
  • JorgeB changed the title to [SOLVED] 6.9-RC2 Kernal panic and trace
  • 3 weeks later...

Hey! I'm seeing a similar issue on my server as well. Here's the relevant syslog:

 

Feb 17 19:46:21 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:25 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:29 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:33 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:37 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:41 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:45 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 21:09:50 thecloud vsftpd[24107]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:50 thecloud in.telnetd[24108]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:50 thecloud sshd[24109]: Connection from 10.0.10.1 port 36967 on 10.0.10.4 port 22 rdomain ""
Feb 17 21:09:50 thecloud sshd[24109]: error: kex_exchange_identification: Connection closed by remote host
Feb 17 21:09:50 thecloud sshd[24109]: Connection closed by 10.0.10.1 port 36967
Feb 17 21:09:50 thecloud vsftpd[24113]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:55 thecloud telnetd[24108]: ttloop: peer died: EOF
Feb 17 21:10:01 thecloud smbd[24110]: [2021/02/17 21:10:01.582271,  0] ../../source3/smbd/process.c:341(read_packet_remainder)
Feb 17 21:10:01 thecloud smbd[24110]:   read_fd_with_timeout failed for client 10.0.10.1 read error = NT_STATUS_END_OF_FILE.
Feb 17 21:53:56 thecloud kernel: Linux version 5.10.1-Unraid (root@Develop) (gcc (GCC) 9.3.0, GNU ld version 2.33.1-slack15) #1 SMP Thu Dec 17 11:41:39 PST 2020
Feb 17 21:53:56 thecloud kernel: Command line: BOOT_IMAGE=/bzimage initrd=/bzroot console=ttyS0,115200 console=tty0
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
Feb 17 21:53:56 thecloud kernel: x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format.
Feb 17 21:53:56 thecloud kernel: BIOS-provided physical RAM map:
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ebff] usable
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x000000000009ec00-0x000000000009ffff] reserved
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000000100000-0x000000003fffffff] usable
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000040000000-0x00000000403fffff] reserved

 

@gdeyoung do you happen to have a Ubiquiti device acting as your gateway/router? The reason I ask is because my USG Pro was actively trying to log into the server via ssh/telnet when it hung.

Link to comment
  • 7 months later...
On 2/17/2021 at 9:33 PM, juchong said:

Hey! I'm seeing a similar issue on my server as well. Here's the relevant syslog:

 

Feb 17 19:46:21 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:25 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:29 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:33 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:37 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:41 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 19:46:45 thecloud root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Feb 17 21:09:50 thecloud vsftpd[24107]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:50 thecloud in.telnetd[24108]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:50 thecloud sshd[24109]: Connection from 10.0.10.1 port 36967 on 10.0.10.4 port 22 rdomain ""
Feb 17 21:09:50 thecloud sshd[24109]: error: kex_exchange_identification: Connection closed by remote host
Feb 17 21:09:50 thecloud sshd[24109]: Connection closed by 10.0.10.1 port 36967
Feb 17 21:09:50 thecloud vsftpd[24113]: connect from 10.0.10.1 (10.0.10.1)
Feb 17 21:09:55 thecloud telnetd[24108]: ttloop: peer died: EOF
Feb 17 21:10:01 thecloud smbd[24110]: [2021/02/17 21:10:01.582271,  0] ../../source3/smbd/process.c:341(read_packet_remainder)
Feb 17 21:10:01 thecloud smbd[24110]:   read_fd_with_timeout failed for client 10.0.10.1 read error = NT_STATUS_END_OF_FILE.
Feb 17 21:53:56 thecloud kernel: Linux version 5.10.1-Unraid (root@Develop) (gcc (GCC) 9.3.0, GNU ld version 2.33.1-slack15) #1 SMP Thu Dec 17 11:41:39 PST 2020
Feb 17 21:53:56 thecloud kernel: Command line: BOOT_IMAGE=/bzimage initrd=/bzroot console=ttyS0,115200 console=tty0
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
Feb 17 21:53:56 thecloud kernel: x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
Feb 17 21:53:56 thecloud kernel: x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
Feb 17 21:53:56 thecloud kernel: x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format.
Feb 17 21:53:56 thecloud kernel: BIOS-provided physical RAM map:
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ebff] usable
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x000000000009ec00-0x000000000009ffff] reserved
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000000100000-0x000000003fffffff] usable
Feb 17 21:53:56 thecloud kernel: BIOS-e820: [mem 0x0000000040000000-0x00000000403fffff] reserved

 

@gdeyoung do you happen to have a Ubiquiti device acting as your gateway/router? The reason I ask is because my USG Pro was actively trying to log into the server via ssh/telnet when it hung.



Did you ever get your issue solved? I have a UDM pro that's also trying to log in via SSH. It always appears to be the last thing logged before crashing.

Edited by Hoopie
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.