Jump to content

  • My server is randomly locking up hard on RC4. IPMI reset not working so i have to power cycle.

    blender50

    By blender50

      • Closed

    This problem is causing parity checks which are taking a bunch of time and are returning all ok. So at lease that's some good news. I'm attaching diagnostics and here is the last piece of the syslog I collected prior to the crash... (I'm using a raspberry Pi as a syslog server)


    It looks like everything goes pear shaped at Mar 31 14:00:06

     

    Any help is much appreciated. My feeling is that this is RC4 related, since RC3 didn't pose any issues for me.....

    Cheers
    Steve

     

    From the raspiPi....
     

    Last login: Wed Mar 30 11:45:31 2022 from 23.120.16.38

    blender@raspberrypi3:~ $ sudo tail -f /var/log/syslog-unraid.log

    Mar 31 12:16:40 blender kernel: Code: 89 e7 e8 65 43 01 00 48 8d 54 1d ff 31 c9 48 89 ee 4c 89 ef e8 e2 00 01 00 4d 63 e6 5b 4c 89 e0 5d 41 5c 41 5d 41 5e 41 5f c3 <f0> 48 0f b3 3e c3 48 8b 07 48 8b 88 f0 01 00 00 81 b9 04 0d 00 00

    Mar 31 12:16:40 blender kernel: RSP: 0018:ffffc9000b6f7ba0 EFLAGS: 00010246

    Mar 31 12:16:40 blender kernel: RAX: ffff888153b2dbc8 RBX: ffff8898aa491c08 RCX: 00000000801c000d

    Mar 31 12:16:40 blender kernel: RDX: 00000000801c000e RSI: 0000000000000068 RDI: 0000000000000000

    Mar 31 12:16:40 blender kernel: RBP: ffff8898aa491df8 R08: ffff8881a0ba4bd0 R09: 00000000801c000d

    Mar 31 12:16:40 blender kernel: R10: ffff8881a0ba4bd0 R11: ffff8881a0ba4bd0 R12: ffff888107926800

    Mar 31 12:16:40 blender kernel: R13: ffff8898aa491c00 R14: ffff889371944000 R15: 0000000000000000

    Mar 31 12:16:40 blender kernel: FS:  000014803d2b5700(0000) GS:ffff88903fcc0000(0000) knlGS:0000000000000000

    Mar 31 12:16:40 blender kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

    Mar 31 12:16:40 blender kernel: CR2: 0000000000000068 CR3: 00000011d89f8006 CR4: 00000000000626e0

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sdd

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sde

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sdc

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sdn

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sdo

    Mar 31 12:22:15 blender emhttpd: spinning down /dev/sdl

    Mar 31 12:22:15 blender SAS Assist v0.86: Spinning down device /dev/sdo

    Mar 31 12:22:15 blender SAS Assist v0.86: Spinning down device /dev/sdn

    Mar 31 12:22:45 blender emhttpd: spinning down /dev/sdj

    Mar 31 12:22:45 blender emhttpd: spinning down /dev/sdk

    Mar 31 12:22:45 blender emhttpd: spinning down /dev/sdp

    Mar 31 12:22:45 blender SAS Assist v0.86: Spinning down device /dev/sdk

    Mar 31 12:22:45 blender SAS Assist v0.86: Spinning down device /dev/sdj

    Mar 31 12:22:45 blender SAS Assist v0.86: Spinning down device /dev/sdp

    Mar 31 12:25:02 blender kernel: mdcmd (45): set md_write_method 0

    Mar 31 12:25:02 blender kernel:

    Mar 31 12:48:20 blender emhttpd: read SMART /dev/sdj

    Mar 31 12:48:20 blender emhttpd: read SMART /dev/sdn

    Mar 31 13:11:54 blender kernel: mdcmd (46): set md_num_stripes 1280

    Mar 31 13:11:54 blender kernel: mdcmd (47): set md_queue_limit 80

    Mar 31 13:11:54 blender kernel: mdcmd (48): set md_sync_limit 5

    Mar 31 13:11:54 blender kernel: mdcmd (49): set md_write_method

    Mar 31 13:33:20 blender emhttpd: cmd: /usr/local/emhttp/plugins/DevPack/scripts/devmanager --download

    Mar 31 13:33:20 blender devpack: Processing Packages...

    Mar 31 13:33:20 blender devpack: Installing autoconf-2.69 package...

    Mar 31 13:33:21 blender devpack: Installing automake-1.16.1 package...

    Mar 31 13:33:22 blender devpack: Installing binutils-2.31.1 package...

    Mar 31 13:33:23 blender devpack: Installing flex-2.6.4 package...

    Mar 31 13:33:24 blender devpack: Installing gc-8.0.4 package...

    Mar 31 13:33:24 blender devpack: Installing gcc-9.2.0 package...

    Mar 31 13:33:29 blender devpack: Installing gcc-gplus-9.2.0 package...

    Mar 31 13:33:31 blender devpack: Installing glibc-2.30 package...

    Mar 31 13:33:31 blender devpack: Installing guile-2.2.3 package...

    Mar 31 13:33:33 blender devpack: Installing kernel-headers-4.14.37 package...

    Mar 31 13:33:34 blender devpack: Installing libmpc-1.1.0 package...

    Mar 31 13:33:34 blender devpack: Installing m4-1.4.18 package...

    Mar 31 13:33:34 blender devpack: Installing make-4.2.1 package...

    Mar 31 13:33:35 blender devpack: Cleaning up packages...

    Mar 31 13:33:35 blender devpack: All packages processed...

    Mar 31 14:00:06 blender kernel: BUG: kernel NULL pointer dereference, address: 0000000000000009

    Mar 31 14:00:06 blender kernel: #PF: supervisor write access in kernel mode

    Mar 31 14:00:06 blender kernel: #PF: error_code(0x0002) - not-present page

    Mar 31 14:00:06 blender kernel: PGD 800000126e9ae067 P4D 800000126e9ae067 PUD 126e9af067 PMD 0

    Mar 31 14:00:06 blender kernel: Oops: 0002 [#2] SMP PTI

    Mar 31 14:00:06 blender kernel: CPU: 14 PID: 4719 Comm: docker-proxy Tainted: G      D W         5.15.30-Unraid #1

    Mar 31 14:00:06 blender kernel: Hardware name: Supermicro X9DRD-7LN4F(-JBOD)/X9DRD-EF/X9DRD-7LN4F, BIOS 3.3 08/23/2018

    Mar 31 14:00:06 blender kernel: RIP: 0010:nf_nat_setup_info+0x70e/0x773 [nf_nat]

    Mar 31 14:00:06 blender kernel: Code: 8b 05 f7 64 00 00 48 8d 8b 98 00 00 00 4a 8d 14 e0 48 8b 02 48 89 93 a0 00 00 00 48 89 83 98 00 00 00 48 85 c0 48 89 0a 74 04 <48> 89 48 08 4c 89 ef e8 19 a5 6c e1 eb 19 41 83 fc 01 48 8b 83 80

    Mar 31 14:00:06 blender kernel: RSP: 0018:ffffc900093779f0 EFLAGS: 00010202

    Mar 31 14:00:06 blender kernel: RAX: 0000000000000001 RBX: ffff88839be5b340 RCX: ffff88839be5b3d8

    Mar 31 14:00:06 blender kernel: RDX: ffff888153b2dbc8 RSI: bd2a841968799d0f RDI: ffffffffa0110404

    Mar 31 14:00:06 blender kernel: RBP: ffffc90009377ac0 R08: ba7b8b957b421b02 R09: 6d816606d4b994be

    Mar 31 14:00:06 blender kernel: R10: ca5f36c0b3036a5c R11: 461b209f52c7bbe5 R12: 0000000000025b79

    Mar 31 14:00:06 blender kernel: R13: ffffffffa0110404 R14: 0000000000000000 R15: ffffc90009377a18

    Mar 31 14:00:06 blender kernel: FS:  000000c000100890(0000) GS:ffff88a03fb80000(0000) knlGS:0000000000000000

    Mar 31 14:00:06 blender kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

    Mar 31 14:00:06 blender kernel: CR2: 0000000000000009 CR3: 00000014467e6001 CR4: 00000000000626e0

    Mar 31 14:00:06 blender kernel: Call Trace:

    Mar 31 14:00:06 blender kernel: <TASK>

    Mar 31 14:00:06 blender kernel: __nf_nat_alloc_null_binding+0x68/0x7f [nf_nat]

    Mar 31 14:00:06 blender kernel: nf_nat_inet_fn+0xa6/0x187 [nf_nat]

    Mar 31 14:00:06 blender kernel: nf_nat_ipv4_out+0x15/0x8b [nf_nat]

    Mar 31 14:00:06 blender kernel: nf_hook_slow+0x3e/0x93

    Mar 31 14:00:06 blender kernel: nf_hook+0xaf/0xd6

    Mar 31 14:00:06 blender kernel: ? __ip_finish_output+0x14e/0x14e

    Mar 31 14:00:06 blender kernel: ip_output+0x82/0x8f

    Mar 31 14:00:06 blender kernel: ? __ip_finish_output+0x14e/0x14e

    Mar 31 14:00:06 blender kernel: __ip_queue_xmit+0x303/0x33d

    Mar 31 14:00:06 blender kernel: __tcp_transmit_skb+0x84e/0x8c0

    Mar 31 14:00:06 blender kernel: tcp_connect+0x768/0x7f7

    Mar 31 14:00:06 blender kernel: tcp_v4_connect+0x402/0x45b

    Mar 31 14:00:06 blender kernel: __inet_stream_connect+0xd8/0x2cf

    Mar 31 14:00:06 blender kernel: inet_stream_connect+0x39/0x4e

    Mar 31 14:00:06 blender kernel: __sys_connect+0x67/0xa2

    Mar 31 14:00:06 blender kernel: ? __sys_socket+0x88/0xb7

    Mar 31 14:00:06 blender kernel: __x64_sys_connect+0x14/0x17

    Mar 31 14:00:06 blender kernel: do_syscall_64+0x83/0xa5

    Mar 31 14:00:06 blender kernel: entry_SYSCALL_64_after_hwframe+0x44/0xae

    Mar 31 14:00:06 blender kernel: RIP: 0033:0x49c99b

    Mar 31 14:00:06 blender kernel: Code: fc ff eb bd e8 46 a8 fc ff e9 61 ff ff ff cc e8 9b 81 fc ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30

    Mar 31 14:00:06 blender kernel: RSP: 002b:000000c0003758b0 EFLAGS: 00000202 ORIG_RAX: 000000000000002a

    Mar 31 14:00:06 blender kernel: RAX: ffffffffffffffda RBX: 000000c00001e000 RCX: 000000000049c99b

    Mar 31 14:00:06 blender kernel: RDX: 0000000000000010 RSI: 000000c0003ca0ec RDI: 0000000000000005

    Mar 31 14:00:06 blender kernel: RBP: 000000c0003758f8 R08: 00000000005067c0 R09: 0000000000000000

    Mar 31 14:00:06 blender kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001

    Mar 31 14:00:06 blender kernel: R13: 0000000000000001 R14: 0000000000000008 R15: ffffffffffffffff

    Mar 31 14:00:06 blender kernel: </TASK>

    Mar 31 14:00:06 blender kernel: Modules linked in: vhost_net vhost vhost_iotlb tap kvm_intel kvm md_mod xt_connmark xt_comment iptable_raw xt_mark cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle tun xt_nat xt_tcpudp veth macvlan xt_conntrack nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables igb i2c_algo_bit x86_pkg_temp_thermal intel_powerclamp coretemp ipmi_ssif crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd rapl mpt3sas intel_cstate intel_uncore i2c_i801 ahci sr_mod i2c_smbus raid_class input_leds cdrom libahci led_class i2c_core scsi_transport_sas

    Mar 31 14:00:06 blender kernel: acpi_ipmi wmi ipmi_si acpi_pad button [last unloaded: md_mod]

    Mar 31 14:00:06 blender kernel: CR2: 0000000000000009

    Mar 31 14:00:06 blender kernel: ---[ end trace 0be523834a4c13bc ]---

    Mar 31 14:00:06 blender kernel: RIP: 0010:clear_bit+0x0/0x6

    Mar 31 14:00:06 blender kernel: Code: 89 e7 e8 65 43 01 00 48 8d 54 1d ff 31 c9 48 89 ee 4c 89 ef e8 e2 00 01 00 4d 63 e6 5b 4c 89 e0 5d 41 5c 41 5d 41 5e 41 5f c3 <f0> 48 0f b3 3e c3 48 8b 07 48 8b 88 f0 01 00 00 81 b9 04 0d 00 00

    Mar 31 14:00:06 blender kernel: RSP: 0018:ffffc9000b6f7ba0 EFLAGS: 00010246

    Mar 31 14:00:06 blender kernel: RAX: ffff888153b2dbc8 RBX: ffff8898aa491c08 RCX: 00000000801c000d

    Mar 31 14:00:06 blender kernel: RDX: 00000000801c000e RSI: 0000000000000068 RDI: 0000000000000000

    blender-diagnostics-20220331-1726.zip

    Go to reports Prereleases

    User Feedback

    Recommended Comments

    ChatNoir

    I see MACVLAN in the log. If you are using docker containers with unique IPs, you should switch your Docker custom network type to IPVLAN that was introduced in 6.10 for that reason.

    The setting is in Settings / Docker Settings. You may have to switch to advanced view.

    Link to comment
    itimpi

    I also notice a lot of items appear to be loaded via Nerdpack and it is possible one of them is not compatible with this release.   It might be worth running in Safe Mode for a while as this would suppress any plugins runningand if that works it will give you a starting point for further troubleshooting.

    Link to comment
    blender50

    I've changed from MACVLAN to IPVLAN in docker/settings. We'll see how this goes. After searching MACVLAN in the forums I see that others have had the same issue, so hopefully this will do the trick. I'll also update any of my nerd pack installs, as a best practice ;-)

    Quote

     

     

    Link to comment
    blender50

    Well it's been close to 5 days and everything is running smoothly! It appears that switching from MACVLAN to IPVLAN did the trick. (although if MACVLAN remains an option it needs looking into....)

    Thanks again for the help ChatNoir & itimpi you are champions!

    • Like 1
    • Thanks 1
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×

  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.

    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.

×
×
  • Create New...