September 16, 20241 yr Hi All, My system is hanging randomly, It seems I can trigger it faster when I am transferring files via ftp, but it does it other times without warning. When it hangs, the max usage I see is around ~29%, Array is up but none of the apps are accessible and I am unable to collect a diagnostic report when it is in this state. All temps on drives is normal. It will be working, then not working. I have checked the logs unsure of the cause of the dilemma. Server is ~1 month old or so. [Top] only shows the following as utilizing any CPU [8146 root 20 0 0 0 0 S 16.3 0.0 130:28.73 unraidd0 ] at this time 9 of the CPUs are showing 100% From Syslog: Sep 16 08:15:43 Plex kernel: CPU: 12 PID: 11882 Comm: zfs Tainted: P O 6.1.106-Unraid #1 Sep 16 08:15:43 Plex kernel: Hardware name: Gigabyte Technology Co., Ltd. Z790 AORUS ELITE AX/Z790 AORUS ELITE AX, BIOS FG 09/12/2023 Sep 16 08:15:43 Plex kernel: RIP: 0010:percpu_counter_add_batch+0x2b/0xa2 Sep 16 08:15:43 Plex kernel: Code: 56 41 55 49 89 f5 41 54 4c 63 e2 55 53 48 89 fb bf 01 00 00 00 e8 ff e7 c0 ff 48 c7 c7 aa f6 14 82 e8 82 10 3f 00 48 8b 43 20 <65> 44 8b 30 49 63 ee 4c 01 ed 48 89 ea 48 f7 da 48 0f 48 d5 49 39 Sep 16 08:15:43 Plex kernel: RSP: 0018:ffffc900291e7730 EFLAGS: 00010202 Sep 16 08:15:43 Plex kernel: RAX: 7fd43949d5480f48 RBX: ffffffff8148b088 RCX: 0000000000000000 Sep 16 08:15:43 Plex kernel: RDX: 000000003fffffff RSI: ffffffff8214f6aa RDI: ffffffff82103f00 Sep 16 08:15:43 Plex kernel: RBP: ffff8881714fd400 R08: 9ae16a3b2f90408f R09: 9ae16a3b2f90404f Sep 16 08:15:43 Plex kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 000000003fffffff Sep 16 08:15:43 Plex kernel: R13: 0000000000000001 R14: 000000000000b340 R15: ffffffffa0e8caa0 Sep 16 08:15:43 Plex kernel: FS: 000015362d5a6800(0000) GS:ffff88a03f700000(0000) knlGS:0000000000000000 Sep 16 08:15:43 Plex kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Sep 16 08:15:43 Plex kernel: CR2: 000015362d563000 CR3: 00000001e6256006 CR4: 0000000000770ee0 Sep 16 08:15:43 Plex kernel: PKRU: 55555554 Sep 16 08:15:43 Plex kernel: Call Trace: Sep 16 08:15:43 Plex kernel: <TASK> Sep 16 08:15:43 Plex kernel: ? __die_body+0x1a/0x5c Sep 16 08:15:43 Plex kernel: ? die_addr+0x38/0x51 Sep 16 08:15:43 Plex kernel: ? exc_general_protection+0x30f/0x345 Sep 16 08:15:43 Plex kernel: ? asm_exc_general_protection+0x22/0x30 Sep 16 08:15:43 Plex kernel: ? percpu_counter_add_batch+0x1b/0xa2 Sep 16 08:15:43 Plex kernel: ? percpu_counter_add_batch+0x2b/0xa2 Sep 16 08:15:43 Plex kernel: ? percpu_counter_add_batch+0x1b/0xa2 Sep 16 08:15:43 Plex kernel: arc_buf_access+0x1b0/0x1f6 [zfs] Sep 16 08:15:43 Plex kernel: dbuf_hold_impl+0x3e6/0x558 [zfs] Sep 16 08:15:43 Plex kernel: dbuf_hold_level+0x2b/0x57 [zfs] Sep 16 08:15:43 Plex kernel: dnode_hold_impl+0x159/0x9a9 [zfs] Sep 16 08:15:43 Plex kernel: ? dbuf_read+0x24b/0x4da [zfs] Sep 16 08:15:43 Plex kernel: dmu_buf_hold_noread+0x37/0xf9 [zfs] Sep 16 08:15:43 Plex kernel: ? string+0x42/0x4b Sep 16 08:15:43 Plex kernel: dmu_buf_hold+0x38/0x75 [zfs] Sep 16 08:15:43 Plex kernel: zap_lockdir+0x4e/0xaf [zfs] Sep 16 08:15:43 Plex kernel: zap_lookup_norm+0x5a/0xcb [zfs] Sep 16 08:15:43 Plex kernel: zap_contains+0x1a/0x2f [zfs] Sep 16 08:15:43 Plex kernel: dsl_prop_get_all_impl+0xb7/0x466 [zfs] Sep 16 08:15:43 Plex kernel: ? preempt_latency_start+0x2b/0x46 Sep 16 08:15:43 Plex kernel: ? _raw_spin_lock+0x13/0x1c Sep 16 08:15:43 Plex kernel: ? dbuf_rele_and_unlock+0x469/0x4ef [zfs] Sep 16 08:15:43 Plex kernel: ? percpu_counter_add_batch+0x85/0xa2 Sep 16 08:15:43 Plex kernel: ? percpu_counter_add_batch+0x85/0xa2 Sep 16 08:15:43 Plex kernel: ? slab_post_alloc_hook+0x4d/0x15e Sep 16 08:15:43 Plex kernel: ? slab_post_alloc_hook+0x4d/0x15e Sep 16 08:15:43 Plex kernel: ? spl_kvmalloc+0x55/0x8a [spl] Sep 16 08:15:43 Plex kernel: ? __kmem_cache_alloc_node+0x118/0x147 Sep 16 08:15:43 Plex kernel: ? spl_kvmalloc+0x55/0x8a [spl] Sep 16 08:15:43 Plex kernel: ? __kmalloc_node+0x9f/0xb1 Sep 16 08:15:43 Plex kernel: ? spl_kvmalloc+0x55/0x8a [spl] Sep 16 08:15:43 Plex kernel: ? spl_kmem_alloc_impl+0xb2/0xf2 [spl] Sep 16 08:15:43 Plex kernel: ? preempt_latency_start+0x2b/0x46 Sep 16 08:15:43 Plex kernel: ? _raw_spin_lock+0x13/0x1c Sep 16 08:15:43 Plex kernel: dsl_prop_get_all_ds+0x114/0x177 [zfs] Sep 16 08:15:43 Plex kernel: ? _raw_spin_unlock+0x14/0x29 Sep 16 08:15:43 Plex kernel: ? dsl_dataset_hold_flags+0x1fb/0x225 [zfs] Sep 16 08:15:43 Plex kernel: ? preempt_latency_start+0x2b/0x46 Sep 16 08:15:43 Plex kernel: ? _raw_spin_lock+0x13/0x1c Sep 16 08:15:43 Plex kernel: ? _raw_spin_unlock+0x14/0x29 Sep 16 08:15:43 Plex kernel: ? dmu_objset_from_ds+0x11e/0x148 [zfs] Sep 16 08:15:43 Plex kernel: zfs_ioc_objset_stats_impl+0x4e/0xef [zfs] Sep 16 08:15:43 Plex kernel: zfs_ioc_objset_stats+0x40/0x75 [zfs] Sep 16 08:15:43 Plex kernel: zfsdev_ioctl_common+0x68f/0x726 [zfs] Sep 16 08:15:43 Plex kernel: ? mod_lruvec_page_state.constprop.0+0x1c/0x2e Sep 16 08:15:43 Plex kernel: ? __kmalloc_large_node+0xd6/0xfb Sep 16 08:15:43 Plex kernel: ? __kmalloc_node+0x5e/0xb1 Sep 16 08:15:43 Plex kernel: zfsdev_ioctl+0x5b/0xb4 [zfs] Sep 16 08:15:43 Plex kernel: vfs_ioctl+0x1b/0x2f Sep 16 08:15:43 Plex kernel: __do_sys_ioctl+0x52/0x78 Sep 16 08:15:43 Plex kernel: do_syscall_64+0x65/0x7b Sep 16 08:15:43 Plex kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Sep 16 08:15:43 Plex kernel: RIP: 0033:0x15362d8494e8 Sep 16 08:15:43 Plex kernel: Code: 00 00 48 8d 44 24 08 48 89 54 24 e0 48 89 44 24 c0 48 8d 44 24 d0 48 89 44 24 c8 b8 10 00 00 00 c7 44 24 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 07 89 d0 c3 0f 1f 40 00 48 8b 15 f9 e8 0d Sep 16 08:15:43 Plex kernel: RSP: 002b:00007ffd0a7c1318 EFLAGS: 00000206 ORIG_RAX: 0000000000000010 Sep 16 08:15:43 Plex kernel: RAX: ffffffffffffffda RBX: 00007ffd0a7c1340 RCX: 000015362d8494e8 Sep 16 08:15:43 Plex kernel: RDX: 00007ffd0a7c1340 RSI: 0000000000005a12 RDI: 0000000000000004 Sep 16 08:15:43 Plex kernel: RBP: 00007ffd0a7c1330 R08: 00000000ffffffff R09: 0000000000000000 Sep 16 08:15:43 Plex kernel: R10: 000015362d753370 R11: 0000000000000206 R12: 0000000000425320 Sep 16 08:15:43 Plex kernel: R13: 0000000000425320 R14: 0000000000427e10 R15: 000015362d5a46d8 Sep 16 08:15:43 Plex kernel: </TASK> Full diagnostics unavailable as it will not let me collect when in this state, will run indefinitely.
September 17, 20241 yr Community Expert Solution Crash is zfs related, so possibly and issue with a zfs filesystem or some hardware issues, like bad RAM, causing it.
September 17, 20241 yr Author Thanks, will take a look and try moving off zfs to see if that helps -- update you in a month when my data moves 😭 it's moving very slow using [unbalanced]
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.