skler

you know what you do. thanks!

When diagnostic was done there were: - The mover running, moving from cache to media share in disks 1,2,4,5,6 (media in with high-water conf, screenshot attached) - GUI Moving files, from disk3 to cache (with acceptable performance of 100mb/s) - The TimeMachine backup on disk 7, now stopped (sorry I didn't saw it) I can't turn off the mover. But the problem is that if I do 2 simple mv, when the mover doesn't runs es: mv /mnt/disk1/mybigfile.iso /mnt/disk3/ mv /mnt/disk2/mybigfile.iso /mnt/disk4/ The parity disk (sdc) has a maximum of write speed of 25-30mb/s. I don't have any special io-wait, maybe this can help: littleboy-diagnostics-20231213-1219.zip

Installed all disks, activated parity again and is all slow again 😕 I did: - a preclear of disks with an average speed of 180/200mb/s - writing to cache is really fast (ZFS raid0) - writing on a single disk (excluded from the array) seems good (ST8000VN0002 (sdo) from diagnostic) When I write into the array (with or without turbo mode) I'm on average speed of 25mb/s dd test is not accurate, I've activated a lot of cache on ZFS cat /boot/config/zfs.conf # limetech - Default ARC size is MemTotal/8 (in bytes) # I set 128GB options zfs zfs_arc_max=128000000000 littleboy-diagnostics-20231213-1005.zip

new log: ==> /var/log/syslog <== Nov 29 18:50:39 littleboy kernel: eth0: renamed from veth43bcc6d Nov 29 18:50:39 littleboy kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth4f5471a: link becomes ready Nov 29 18:50:39 littleboy kernel: docker0: port 3(veth4f5471a) entered blocking state Nov 29 18:50:39 littleboy kernel: docker0: port 3(veth4f5471a) entered forwarding state Nov 29 18:50:40 littleboy kernel: eth0: renamed from vethc561baa Nov 29 18:50:40 littleboy kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth5002242: link becomes ready Nov 29 18:50:40 littleboy kernel: docker0: port 4(veth5002242) entered blocking state Nov 29 18:50:40 littleboy kernel: docker0: port 4(veth5002242) entered forwarding state Nov 29 18:53:06 littleboy kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 21143 jiffies s: 105445 root: 0x1/. Nov 29 18:53:06 littleboy kernel: rcu: blocking rcu_node structures (internal RCU debug): l=1:0-14:0x4/. Nov 29 18:53:06 littleboy kernel: Sending NMI from CPU 84 to CPUs 2: Nov 29 18:53:06 littleboy kernel: NMI backtrace for cpu 2 Nov 29 18:53:06 littleboy kernel: CPU: 2 PID: 850 Comm: npm install Tainted: P W O 6.1.49-Unraid #1 Nov 29 18:53:06 littleboy kernel: Hardware name: VxRack AS PowerEdge R740xd/0RR8YK, BIOS 2.19.1 06/04/2023 Nov 29 18:53:06 littleboy kernel: RIP: 0010:nf_nat_setup_info+0x146/0x7d1 [nf_nat] Nov 29 18:53:06 littleboy kernel: Code: 18 4c 89 f2 4c 89 ff e8 1e f8 ff ff 48 8b 15 4a 6a 00 00 89 c0 48 8d 04 c2 4c 8b 30 4d 85 f6 74 2a 49 81 ee 90 00 00 00 eb 21 <8a> 44 24 46 41 38 46 46 74 21 49 8b 96 90 00 00 00 48 85 d2 0f 84 Nov 29 18:53:06 littleboy kernel: RSP: 0018:ffffc900089fc738 EFLAGS: 00000286 Nov 29 18:53:06 littleboy kernel: RAX: ffff88a106958e06 RBX: ffff888e5d6a1000 RCX: a706ed18065b4908 Nov 29 18:53:06 littleboy kernel: RDX: ffff8881eda97890 RSI: 26cbe6897865622e RDI: 2a87b77939a0cc9d Nov 29 18:53:06 littleboy kernel: RBP: ffffc900089fc800 R08: 6f84c2f0878e2f7a R09: dcf3e2ac25213978 Nov 29 18:53:06 littleboy kernel: R10: a3df108aaba40bc7 R11: ffffc900089fc710 R12: ffffc900089fc814 Nov 29 18:53:06 littleboy kernel: R13: 0000000000000000 R14: ffff8881eda97800 R15: ffffffff82a11d00 Nov 29 18:53:06 littleboy kernel: FS: 00001457b2f33800(0000) GS:ffff889fff640000(0000) knlGS:0000000000000000 Nov 29 18:53:06 littleboy kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 29 18:53:06 littleboy kernel: CR2: 0000000008b3e000 CR3: 000000267aee6003 CR4: 00000000007706e0 Nov 29 18:53:06 littleboy kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 29 18:53:06 littleboy kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Nov 29 18:53:06 littleboy kernel: PKRU: 55555554 Nov 29 18:53:06 littleboy kernel: Call Trace: Nov 29 18:53:06 littleboy kernel: <NMI> Nov 29 18:53:06 littleboy kernel: ? nmi_cpu_backtrace+0xd3/0x104 Nov 29 18:53:06 littleboy kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Nov 29 18:53:06 littleboy kernel: ? nmi_handle+0x54/0x131 Nov 29 18:53:06 littleboy kernel: ? nf_nat_setup_info+0x146/0x7d1 [nf_nat] Nov 29 18:53:06 littleboy kernel: ? default_do_nmi+0x66/0x15b Nov 29 18:53:06 littleboy kernel: ? exc_nmi+0xbf/0x130 Nov 29 18:53:06 littleboy kernel: ? end_repeat_nmi+0x16/0x67 Nov 29 18:53:06 littleboy kernel: ? nf_nat_setup_info+0x146/0x7d1 [nf_nat] Nov 29 18:53:06 littleboy kernel: ? nf_nat_setup_info+0x146/0x7d1 [nf_nat] Nov 29 18:53:06 littleboy kernel: ? nf_nat_setup_info+0x146/0x7d1 [nf_nat] Nov 29 18:53:06 littleboy kernel: </NMI> Nov 29 18:53:06 littleboy kernel: <IRQ> Nov 29 18:53:06 littleboy kernel: ? krealloc+0x82/0x93 Nov 29 18:53:06 littleboy kernel: nf_nat_masquerade_ipv4+0x114/0x13c [nf_nat] Nov 29 18:53:06 littleboy kernel: masquerade_tg+0x48/0x66 [xt_MASQUERADE] Nov 29 18:53:06 littleboy kernel: ipt_do_table+0x51b/0x5bf [ip_tables] Nov 29 18:53:06 littleboy kernel: ? fib_validate_source+0xce/0xdf Nov 29 18:53:06 littleboy kernel: nf_nat_inet_fn+0x123/0x1a8 [nf_nat] Nov 29 18:53:06 littleboy kernel: nf_nat_ipv4_out+0x15/0x91 [nf_nat] Nov 29 18:53:06 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:53:06 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:06 littleboy kernel: nf_hook+0xdf/0x110 Nov 29 18:53:06 littleboy kernel: ? dev_ethtool+0x7c0/0x2349 Nov 29 18:53:06 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:06 littleboy kernel: ip_output+0x78/0x88 Nov 29 18:53:06 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:06 littleboy kernel: ip_sabotage_in+0x4f/0x60 [br_netfilter] Nov 29 18:53:06 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:53:06 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:53:06 littleboy kernel: NF_HOOK.constprop.0+0x79/0xd9 Nov 29 18:53:06 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:53:06 littleboy kernel: __netif_receive_skb_one_core+0x77/0x9c Nov 29 18:53:06 littleboy kernel: netif_receive_skb+0xbf/0x127 Nov 29 18:53:06 littleboy kernel: br_handle_frame_finish+0x438/0x472 [bridge] Nov 29 18:53:06 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:06 littleboy kernel: br_nf_hook_thresh+0xe2/0x109 [br_netfilter] Nov 29 18:53:06 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:06 littleboy kernel: br_nf_pre_routing_finish+0x2c1/0x2ec [br_netfilter] Nov 29 18:53:06 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:06 littleboy kernel: ? NF_HOOK.isra.0+0xe4/0x140 [br_netfilter] Nov 29 18:53:06 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:53:06 littleboy kernel: br_nf_pre_routing+0x236/0x24a [br_netfilter] Nov 29 18:53:06 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:53:06 littleboy kernel: br_handle_frame+0x277/0x2e0 [bridge] Nov 29 18:53:06 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:06 littleboy kernel: __netif_receive_skb_core.constprop.0+0x4fa/0x6e9 Nov 29 18:53:06 littleboy kernel: ? se_is_idle+0x16/0x34 Nov 29 18:53:06 littleboy kernel: ? place_entity+0x6e/0xae Nov 29 18:53:06 littleboy kernel: __netif_receive_skb_one_core+0x40/0x9c Nov 29 18:53:06 littleboy kernel: process_backlog+0x8c/0x116 Nov 29 18:53:06 littleboy kernel: __napi_poll.constprop.0+0x28/0x124 Nov 29 18:53:06 littleboy kernel: net_rx_action+0x159/0x24f Nov 29 18:53:06 littleboy kernel: __do_softirq+0x126/0x288 Nov 29 18:53:06 littleboy kernel: do_softirq+0x7f/0xab Nov 29 18:53:06 littleboy kernel: </IRQ> Nov 29 18:53:06 littleboy kernel: <TASK> Nov 29 18:53:06 littleboy kernel: __local_bh_enable_ip+0x4c/0x6b Nov 29 18:53:06 littleboy kernel: ip_finish_output2+0x36a/0x39d Nov 29 18:53:06 littleboy kernel: __ip_queue_xmit+0x2d8/0x31f Nov 29 18:53:06 littleboy kernel: __tcp_transmit_skb+0x841/0x8b1 Nov 29 18:53:06 littleboy kernel: tcp_connect+0x7e4/0x86f Nov 29 18:53:06 littleboy kernel: tcp_v4_connect+0x41a/0x47d Nov 29 18:53:06 littleboy kernel: __inet_stream_connect+0xdd/0x314 Nov 29 18:53:06 littleboy kernel: ? percpu_counter_add_batch+0x85/0xa2 Nov 29 18:53:06 littleboy kernel: inet_stream_connect+0x39/0x52 Nov 29 18:53:06 littleboy kernel: __sys_connect+0x68/0xa7 Nov 29 18:53:06 littleboy kernel: __x64_sys_connect+0x14/0x1b Nov 29 18:53:06 littleboy kernel: do_syscall_64+0x68/0x81 Nov 29 18:53:06 littleboy kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Nov 29 18:53:06 littleboy kernel: RIP: 0033:0x1457b313d59b Nov 29 18:53:06 littleboy kernel: Code: 83 ec 18 89 54 24 0c 48 89 34 24 89 7c 24 08 e8 bb fa ff ff 8b 54 24 0c 48 8b 34 24 41 89 c0 8b 7c 24 08 b8 2a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 08 e8 f1 fa ff ff 8b 44 Nov 29 18:53:06 littleboy kernel: RSP: 002b:00007ffe60b9dec0 EFLAGS: 00000293 ORIG_RAX: 000000000000002a Nov 29 18:53:06 littleboy kernel: RAX: ffffffffffffffda RBX: 0000000005b380a0 RCX: 00001457b313d59b Nov 29 18:53:06 littleboy kernel: RDX: 0000000000000010 RSI: 00007ffe60b9df50 RDI: 00000000000002d0 Nov 29 18:53:06 littleboy kernel: RBP: 00007ffe60b9df20 R08: 0000000000000000 R09: 000000000714c230 Nov 29 18:53:06 littleboy kernel: R10: 0000000000000f84 R11: 0000000000000293 R12: 000000000714c288 Nov 29 18:53:06 littleboy kernel: R13: 00007ffe60b9df50 R14: 0000000000000010 R15: 00001457b2f336a0 Nov 29 18:53:06 littleboy kernel: </TASK> Nov 29 18:53:44 littleboy kernel: rcu: INFO: rcu_preempt self-detected stall on CPU Nov 29 18:53:44 littleboy kernel: rcu: 2-....: (1 GPs behind) idle=14dc/1/0x4000000000000000 softirq=9300215/9300216 fqs=21807 Nov 29 18:53:44 littleboy kernel: (t=60001 jiffies g=36305665 q=1727860 ncpus=88) Nov 29 18:53:44 littleboy kernel: CPU: 2 PID: 850 Comm: npm install Tainted: P W O 6.1.49-Unraid #1 Nov 29 18:53:44 littleboy kernel: Hardware name: VxRack AS PowerEdge R740xd/0RR8YK, BIOS 2.19.1 06/04/2023 Nov 29 18:53:44 littleboy kernel: RIP: 0010:nf_nat_setup_info+0x157/0x7d1 [nf_nat] Nov 29 18:53:44 littleboy kernel: Code: 00 00 89 c0 48 8d 04 c2 4c 8b 30 4d 85 f6 74 2a 49 81 ee 90 00 00 00 eb 21 8a 44 24 46 41 38 46 46 74 21 49 8b 96 90 00 00 00 <48> 85 d2 0f 84 56 ff ff ff 4c 8d b2 70 ff ff ff 4d 85 f6 75 da e9 Nov 29 18:53:44 littleboy kernel: RSP: 0018:ffffc900089fc738 EFLAGS: 00000212 Nov 29 18:53:44 littleboy kernel: RAX: ffff88a106958e06 RBX: ffff888e5d6a1000 RCX: a706ed18065b4908 Nov 29 18:53:44 littleboy kernel: RDX: ffff8881eda97890 RSI: 26cbe6897865622e RDI: 2a87b77939a0cc9d Nov 29 18:53:44 littleboy kernel: RBP: ffffc900089fc800 R08: 6f84c2f0878e2f7a R09: dcf3e2ac25213978 Nov 29 18:53:44 littleboy kernel: R10: a3df108aaba40bc7 R11: ffffc900089fc710 R12: ffffc900089fc814 Nov 29 18:53:44 littleboy kernel: R13: 0000000000000000 R14: ffff8881eda97800 R15: ffffffff82a11d00 Nov 29 18:53:44 littleboy kernel: FS: 00001457b2f33800(0000) GS:ffff889fff640000(0000) knlGS:0000000000000000 Nov 29 18:53:44 littleboy kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 29 18:53:44 littleboy kernel: CR2: 0000000008b3e000 CR3: 000000267aee6003 CR4: 00000000007706e0 Nov 29 18:53:44 littleboy kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 29 18:53:44 littleboy kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Nov 29 18:53:44 littleboy kernel: PKRU: 55555554 Nov 29 18:53:44 littleboy kernel: Call Trace: Nov 29 18:53:44 littleboy kernel: <IRQ> Nov 29 18:53:44 littleboy kernel: ? rcu_dump_cpu_stacks+0x95/0xb9 Nov 29 18:53:44 littleboy kernel: ? rcu_sched_clock_irq+0x337/0xa37 Nov 29 18:53:44 littleboy kernel: ? cgroup_rstat_updated+0x21/0xa5 Nov 29 18:53:44 littleboy kernel: ? tick_init_jiffy_update+0x7c/0x7c Nov 29 18:53:44 littleboy kernel: ? update_process_times+0x62/0x81 Nov 29 18:53:44 littleboy kernel: ? tick_sched_timer+0x43/0x71 Nov 29 18:53:44 littleboy kernel: ? __hrtimer_run_queues+0xe8/0x190 Nov 29 18:53:44 littleboy kernel: ? hrtimer_interrupt+0x9c/0x16e Nov 29 18:53:44 littleboy kernel: ? __sysvec_apic_timer_interrupt+0xc2/0x12f Nov 29 18:53:44 littleboy kernel: ? sysvec_apic_timer_interrupt+0x27/0xa6 Nov 29 18:53:44 littleboy kernel: ? asm_sysvec_apic_timer_interrupt+0x16/0x20 Nov 29 18:53:44 littleboy kernel: ? nf_nat_setup_info+0x157/0x7d1 [nf_nat] Nov 29 18:53:44 littleboy kernel: ? nf_nat_setup_info+0x128/0x7d1 [nf_nat] Nov 29 18:53:44 littleboy kernel: ? krealloc+0x82/0x93 Nov 29 18:53:44 littleboy kernel: nf_nat_masquerade_ipv4+0x114/0x13c [nf_nat] Nov 29 18:53:44 littleboy kernel: masquerade_tg+0x48/0x66 [xt_MASQUERADE] Nov 29 18:53:44 littleboy kernel: ipt_do_table+0x51b/0x5bf [ip_tables] Nov 29 18:53:44 littleboy kernel: ? fib_validate_source+0xce/0xdf Nov 29 18:53:44 littleboy kernel: nf_nat_inet_fn+0x123/0x1a8 [nf_nat] Nov 29 18:53:44 littleboy kernel: nf_nat_ipv4_out+0x15/0x91 [nf_nat] Nov 29 18:53:44 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:53:44 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:44 littleboy kernel: nf_hook+0xdf/0x110 Nov 29 18:53:44 littleboy kernel: ? dev_ethtool+0x7c0/0x2349 Nov 29 18:53:44 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:44 littleboy kernel: ip_output+0x78/0x88 Nov 29 18:53:44 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:53:44 littleboy kernel: ip_sabotage_in+0x4f/0x60 [br_netfilter] Nov 29 18:53:44 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:53:44 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:53:44 littleboy kernel: NF_HOOK.constprop.0+0x79/0xd9 Nov 29 18:53:44 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:53:44 littleboy kernel: __netif_receive_skb_one_core+0x77/0x9c Nov 29 18:53:44 littleboy kernel: netif_receive_skb+0xbf/0x127 Nov 29 18:53:44 littleboy kernel: br_handle_frame_finish+0x438/0x472 [bridge] Nov 29 18:53:44 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:44 littleboy kernel: br_nf_hook_thresh+0xe2/0x109 [br_netfilter] Nov 29 18:53:44 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:44 littleboy kernel: br_nf_pre_routing_finish+0x2c1/0x2ec [br_netfilter] Nov 29 18:53:44 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:44 littleboy kernel: ? NF_HOOK.isra.0+0xe4/0x140 [br_netfilter] Nov 29 18:53:44 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:53:44 littleboy kernel: br_nf_pre_routing+0x236/0x24a [br_netfilter] Nov 29 18:53:44 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:53:44 littleboy kernel: br_handle_frame+0x277/0x2e0 [bridge] Nov 29 18:53:44 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:53:44 littleboy kernel: __netif_receive_skb_core.constprop.0+0x4fa/0x6e9 Nov 29 18:53:44 littleboy kernel: ? se_is_idle+0x16/0x34 Nov 29 18:53:44 littleboy kernel: ? place_entity+0x6e/0xae Nov 29 18:53:44 littleboy kernel: __netif_receive_skb_one_core+0x40/0x9c Nov 29 18:53:44 littleboy kernel: process_backlog+0x8c/0x116 Nov 29 18:53:44 littleboy kernel: __napi_poll.constprop.0+0x28/0x124 Nov 29 18:53:44 littleboy kernel: net_rx_action+0x159/0x24f Nov 29 18:53:44 littleboy kernel: __do_softirq+0x126/0x288 Nov 29 18:53:44 littleboy kernel: do_softirq+0x7f/0xab Nov 29 18:53:44 littleboy kernel: </IRQ> Nov 29 18:53:44 littleboy kernel: <TASK> Nov 29 18:53:44 littleboy kernel: __local_bh_enable_ip+0x4c/0x6b Nov 29 18:53:44 littleboy kernel: ip_finish_output2+0x36a/0x39d Nov 29 18:53:44 littleboy kernel: __ip_queue_xmit+0x2d8/0x31f Nov 29 18:53:44 littleboy kernel: __tcp_transmit_skb+0x841/0x8b1 Nov 29 18:53:44 littleboy kernel: tcp_connect+0x7e4/0x86f Nov 29 18:53:44 littleboy kernel: tcp_v4_connect+0x41a/0x47d Nov 29 18:53:44 littleboy kernel: __inet_stream_connect+0xdd/0x314 Nov 29 18:53:44 littleboy kernel: ? percpu_counter_add_batch+0x85/0xa2 Nov 29 18:53:44 littleboy kernel: inet_stream_connect+0x39/0x52 Nov 29 18:53:44 littleboy kernel: __sys_connect+0x68/0xa7 Nov 29 18:53:44 littleboy kernel: __x64_sys_connect+0x14/0x1b Nov 29 18:53:44 littleboy kernel: do_syscall_64+0x68/0x81 Nov 29 18:53:44 littleboy kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Nov 29 18:53:44 littleboy kernel: RIP: 0033:0x1457b313d59b Nov 29 18:53:44 littleboy kernel: Code: 83 ec 18 89 54 24 0c 48 89 34 24 89 7c 24 08 e8 bb fa ff ff 8b 54 24 0c 48 8b 34 24 41 89 c0 8b 7c 24 08 b8 2a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 08 e8 f1 fa ff ff 8b 44 Nov 29 18:53:44 littleboy kernel: RSP: 002b:00007ffe60b9dec0 EFLAGS: 00000293 ORIG_RAX: 000000000000002a Nov 29 18:53:44 littleboy kernel: RAX: ffffffffffffffda RBX: 0000000005b380a0 RCX: 00001457b313d59b Nov 29 18:53:44 littleboy kernel: RDX: 0000000000000010 RSI: 00007ffe60b9df50 RDI: 00000000000002d0 Nov 29 18:53:44 littleboy kernel: RBP: 00007ffe60b9df20 R08: 0000000000000000 R09: 000000000714c230 Nov 29 18:53:44 littleboy kernel: R10: 0000000000000f84 R11: 0000000000000293 R12: 000000000714c288 Nov 29 18:53:44 littleboy kernel: R13: 00007ffe60b9df50 R14: 0000000000000010 R15: 00001457b2f336a0 Nov 29 18:53:44 littleboy kernel: </TASK> Nov 29 18:54:11 littleboy kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 85655 jiffies s: 105445 root: 0x1/. Nov 29 18:54:11 littleboy kernel: rcu: blocking rcu_node structures (internal RCU debug): l=1:0-14:0x4/. Nov 29 18:54:11 littleboy kernel: Sending NMI from CPU 84 to CPUs 2: Nov 29 18:54:11 littleboy kernel: NMI backtrace for cpu 2 Nov 29 18:54:11 littleboy kernel: CPU: 2 PID: 850 Comm: npm install Tainted: P W O 6.1.49-Unraid #1 Nov 29 18:54:11 littleboy kernel: Hardware name: VxRack AS PowerEdge R740xd/0RR8YK, BIOS 2.19.1 06/04/2023 Nov 29 18:54:11 littleboy kernel: RIP: 0010:nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:54:11 littleboy kernel: Code: ff ff 48 8b 15 4a 6a 00 00 89 c0 48 8d 04 c2 4c 8b 30 4d 85 f6 74 2a 49 81 ee 90 00 00 00 eb 21 8a 44 24 46 41 38 46 46 74 21 <49> 8b 96 90 00 00 00 48 85 d2 0f 84 56 ff ff ff 4c 8d b2 70 ff ff Nov 29 18:54:11 littleboy kernel: RSP: 0018:ffffc900089fc738 EFLAGS: 00000212 Nov 29 18:54:11 littleboy kernel: RAX: ffff88a106958e06 RBX: ffff888e5d6a1000 RCX: a706ed18065b4908 Nov 29 18:54:11 littleboy kernel: RDX: ffff8881eda97890 RSI: 26cbe6897865622e RDI: 2a87b77939a0cc9d Nov 29 18:54:11 littleboy kernel: RBP: ffffc900089fc800 R08: 6f84c2f0878e2f7a R09: dcf3e2ac25213978 Nov 29 18:54:11 littleboy kernel: R10: a3df108aaba40bc7 R11: ffffc900089fc710 R12: ffffc900089fc814 Nov 29 18:54:11 littleboy kernel: R13: 0000000000000000 R14: ffff8881eda97800 R15: ffffffff82a11d00 Nov 29 18:54:11 littleboy kernel: FS: 00001457b2f33800(0000) GS:ffff889fff640000(0000) knlGS:0000000000000000 Nov 29 18:54:11 littleboy kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 29 18:54:11 littleboy kernel: CR2: 0000000008b3e000 CR3: 000000267aee6003 CR4: 00000000007706e0 Nov 29 18:54:11 littleboy kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 29 18:54:11 littleboy kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Nov 29 18:54:11 littleboy kernel: PKRU: 55555554 Nov 29 18:54:11 littleboy kernel: Call Trace: Nov 29 18:54:11 littleboy kernel: <NMI> Nov 29 18:54:11 littleboy kernel: ? nmi_cpu_backtrace+0xd3/0x104 Nov 29 18:54:11 littleboy kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Nov 29 18:54:11 littleboy kernel: ? nmi_handle+0x54/0x131 Nov 29 18:54:11 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:54:11 littleboy kernel: ? default_do_nmi+0x66/0x15b Nov 29 18:54:11 littleboy kernel: ? exc_nmi+0xbf/0x130 Nov 29 18:54:11 littleboy kernel: ? end_repeat_nmi+0x16/0x67 Nov 29 18:54:11 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:54:11 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:54:11 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:54:11 littleboy kernel: </NMI> Nov 29 18:54:11 littleboy kernel: <IRQ> Nov 29 18:54:11 littleboy kernel: ? krealloc+0x82/0x93 Nov 29 18:54:11 littleboy kernel: nf_nat_masquerade_ipv4+0x114/0x13c [nf_nat] Nov 29 18:54:11 littleboy kernel: masquerade_tg+0x48/0x66 [xt_MASQUERADE] Nov 29 18:54:11 littleboy kernel: ipt_do_table+0x51b/0x5bf [ip_tables] Nov 29 18:54:11 littleboy kernel: ? fib_validate_source+0xce/0xdf Nov 29 18:54:11 littleboy kernel: nf_nat_inet_fn+0x123/0x1a8 [nf_nat] Nov 29 18:54:11 littleboy kernel: nf_nat_ipv4_out+0x15/0x91 [nf_nat] Nov 29 18:54:11 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:54:11 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:54:11 littleboy kernel: nf_hook+0xdf/0x110 Nov 29 18:54:11 littleboy kernel: ? dev_ethtool+0x7c0/0x2349 Nov 29 18:54:11 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:54:11 littleboy kernel: ip_output+0x78/0x88 Nov 29 18:54:11 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:54:11 littleboy kernel: ip_sabotage_in+0x4f/0x60 [br_netfilter] Nov 29 18:54:11 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:54:11 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:54:11 littleboy kernel: NF_HOOK.constprop.0+0x79/0xd9 Nov 29 18:54:11 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:54:11 littleboy kernel: __netif_receive_skb_one_core+0x77/0x9c Nov 29 18:54:11 littleboy kernel: netif_receive_skb+0xbf/0x127 Nov 29 18:54:11 littleboy kernel: br_handle_frame_finish+0x438/0x472 [bridge] Nov 29 18:54:11 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:54:11 littleboy kernel: br_nf_hook_thresh+0xe2/0x109 [br_netfilter] Nov 29 18:54:11 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:54:11 littleboy kernel: br_nf_pre_routing_finish+0x2c1/0x2ec [br_netfilter] Nov 29 18:54:11 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:54:11 littleboy kernel: ? NF_HOOK.isra.0+0xe4/0x140 [br_netfilter] Nov 29 18:54:11 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:54:11 littleboy kernel: br_nf_pre_routing+0x236/0x24a [br_netfilter] Nov 29 18:54:11 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:54:11 littleboy kernel: br_handle_frame+0x277/0x2e0 [bridge] Nov 29 18:54:11 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:54:11 littleboy kernel: __netif_receive_skb_core.constprop.0+0x4fa/0x6e9 Nov 29 18:54:11 littleboy kernel: ? se_is_idle+0x16/0x34 Nov 29 18:54:11 littleboy kernel: ? place_entity+0x6e/0xae Nov 29 18:54:11 littleboy kernel: __netif_receive_skb_one_core+0x40/0x9c Nov 29 18:54:11 littleboy kernel: process_backlog+0x8c/0x116 Nov 29 18:54:11 littleboy kernel: __napi_poll.constprop.0+0x28/0x124 Nov 29 18:54:11 littleboy kernel: net_rx_action+0x159/0x24f Nov 29 18:54:11 littleboy kernel: __do_softirq+0x126/0x288 Nov 29 18:54:11 littleboy kernel: do_softirq+0x7f/0xab Nov 29 18:54:11 littleboy kernel: </IRQ> Nov 29 18:54:11 littleboy kernel: <TASK> Nov 29 18:54:11 littleboy kernel: __local_bh_enable_ip+0x4c/0x6b Nov 29 18:54:11 littleboy kernel: ip_finish_output2+0x36a/0x39d Nov 29 18:54:11 littleboy kernel: __ip_queue_xmit+0x2d8/0x31f Nov 29 18:54:11 littleboy kernel: __tcp_transmit_skb+0x841/0x8b1 Nov 29 18:54:11 littleboy kernel: tcp_connect+0x7e4/0x86f Nov 29 18:54:11 littleboy kernel: tcp_v4_connect+0x41a/0x47d Nov 29 18:54:11 littleboy kernel: __inet_stream_connect+0xdd/0x314 Nov 29 18:54:11 littleboy kernel: ? percpu_counter_add_batch+0x85/0xa2 Nov 29 18:54:11 littleboy kernel: inet_stream_connect+0x39/0x52 Nov 29 18:54:11 littleboy kernel: __sys_connect+0x68/0xa7 Nov 29 18:54:11 littleboy kernel: __x64_sys_connect+0x14/0x1b Nov 29 18:54:11 littleboy kernel: do_syscall_64+0x68/0x81 Nov 29 18:54:11 littleboy kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Nov 29 18:54:11 littleboy kernel: RIP: 0033:0x1457b313d59b Nov 29 18:54:11 littleboy kernel: Code: 83 ec 18 89 54 24 0c 48 89 34 24 89 7c 24 08 e8 bb fa ff ff 8b 54 24 0c 48 8b 34 24 41 89 c0 8b 7c 24 08 b8 2a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 08 e8 f1 fa ff ff 8b 44 Nov 29 18:54:11 littleboy kernel: RSP: 002b:00007ffe60b9dec0 EFLAGS: 00000293 ORIG_RAX: 000000000000002a Nov 29 18:54:11 littleboy kernel: RAX: ffffffffffffffda RBX: 0000000005b380a0 RCX: 00001457b313d59b Nov 29 18:54:11 littleboy kernel: RDX: 0000000000000010 RSI: 00007ffe60b9df50 RDI: 00000000000002d0 Nov 29 18:54:11 littleboy kernel: RBP: 00007ffe60b9df20 R08: 0000000000000000 R09: 000000000714c230 Nov 29 18:54:11 littleboy kernel: R10: 0000000000000f84 R11: 0000000000000293 R12: 000000000714c288 Nov 29 18:54:11 littleboy kernel: R13: 00007ffe60b9df50 R14: 0000000000000010 R15: 00001457b2f336a0 Nov 29 18:54:11 littleboy kernel: </TASK> Nov 29 18:55:16 littleboy kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 151192 jiffies s: 105445 root: 0x1/. Nov 29 18:55:16 littleboy kernel: rcu: blocking rcu_node structures (internal RCU debug): l=1:0-14:0x4/. Nov 29 18:55:16 littleboy kernel: Sending NMI from CPU 84 to CPUs 2: Nov 29 18:55:16 littleboy kernel: NMI backtrace for cpu 2 Nov 29 18:55:16 littleboy kernel: CPU: 2 PID: 850 Comm: npm install Tainted: P W O 6.1.49-Unraid #1 Nov 29 18:55:16 littleboy kernel: Hardware name: VxRack AS PowerEdge R740xd/0RR8YK, BIOS 2.19.1 06/04/2023 Nov 29 18:55:16 littleboy kernel: RIP: 0010:nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:55:16 littleboy kernel: Code: ff ff 48 8b 15 4a 6a 00 00 89 c0 48 8d 04 c2 4c 8b 30 4d 85 f6 74 2a 49 81 ee 90 00 00 00 eb 21 8a 44 24 46 41 38 46 46 74 21 <49> 8b 96 90 00 00 00 48 85 d2 0f 84 56 ff ff ff 4c 8d b2 70 ff ff Nov 29 18:55:16 littleboy kernel: RSP: 0018:ffffc900089fc738 EFLAGS: 00000212 Nov 29 18:55:16 littleboy kernel: RAX: ffff88a106958e06 RBX: ffff888e5d6a1000 RCX: a706ed18065b4908 Nov 29 18:55:16 littleboy kernel: RDX: ffff8881eda97890 RSI: 26cbe6897865622e RDI: 2a87b77939a0cc9d Nov 29 18:55:16 littleboy kernel: RBP: ffffc900089fc800 R08: 6f84c2f0878e2f7a R09: dcf3e2ac25213978 Nov 29 18:55:16 littleboy kernel: R10: a3df108aaba40bc7 R11: ffffc900089fc710 R12: ffffc900089fc814 Nov 29 18:55:16 littleboy kernel: R13: 0000000000000000 R14: ffff8881eda97800 R15: ffffffff82a11d00 Nov 29 18:55:16 littleboy kernel: FS: 00001457b2f33800(0000) GS:ffff889fff640000(0000) knlGS:0000000000000000 Nov 29 18:55:16 littleboy kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 29 18:55:16 littleboy kernel: CR2: 0000000008b3e000 CR3: 000000267aee6003 CR4: 00000000007706e0 Nov 29 18:55:16 littleboy kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 29 18:55:16 littleboy kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Nov 29 18:55:16 littleboy kernel: PKRU: 55555554 Nov 29 18:55:16 littleboy kernel: Call Trace: Nov 29 18:55:16 littleboy kernel: <NMI> Nov 29 18:55:16 littleboy kernel: ? nmi_cpu_backtrace+0xd3/0x104 Nov 29 18:55:16 littleboy kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Nov 29 18:55:16 littleboy kernel: ? nmi_handle+0x54/0x131 Nov 29 18:55:16 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:55:16 littleboy kernel: ? default_do_nmi+0x66/0x15b Nov 29 18:55:16 littleboy kernel: ? exc_nmi+0xbf/0x130 Nov 29 18:55:16 littleboy kernel: ? end_repeat_nmi+0x16/0x67 Nov 29 18:55:16 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:55:16 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:55:16 littleboy kernel: ? nf_nat_setup_info+0x150/0x7d1 [nf_nat] Nov 29 18:55:16 littleboy kernel: </NMI> Nov 29 18:55:16 littleboy kernel: <IRQ> Nov 29 18:55:16 littleboy kernel: ? krealloc+0x82/0x93 Nov 29 18:55:16 littleboy kernel: nf_nat_masquerade_ipv4+0x114/0x13c [nf_nat] Nov 29 18:55:16 littleboy kernel: masquerade_tg+0x48/0x66 [xt_MASQUERADE] Nov 29 18:55:16 littleboy kernel: ipt_do_table+0x51b/0x5bf [ip_tables] Nov 29 18:55:16 littleboy kernel: ? fib_validate_source+0xce/0xdf Nov 29 18:55:16 littleboy kernel: nf_nat_inet_fn+0x123/0x1a8 [nf_nat] Nov 29 18:55:16 littleboy kernel: nf_nat_ipv4_out+0x15/0x91 [nf_nat] Nov 29 18:55:16 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:55:16 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:55:16 littleboy kernel: nf_hook+0xdf/0x110 Nov 29 18:55:16 littleboy kernel: ? dev_ethtool+0x7c0/0x2349 Nov 29 18:55:16 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:55:16 littleboy kernel: ip_output+0x78/0x88 Nov 29 18:55:16 littleboy kernel: ? __ip_finish_output+0x144/0x144 Nov 29 18:55:16 littleboy kernel: ip_sabotage_in+0x4f/0x60 [br_netfilter] Nov 29 18:55:16 littleboy kernel: nf_hook_slow+0x3a/0x96 Nov 29 18:55:16 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:55:16 littleboy kernel: NF_HOOK.constprop.0+0x79/0xd9 Nov 29 18:55:16 littleboy kernel: ? ip_rcv_finish_core.constprop.0+0x3e8/0x3e8 Nov 29 18:55:16 littleboy kernel: __netif_receive_skb_one_core+0x77/0x9c Nov 29 18:55:16 littleboy kernel: netif_receive_skb+0xbf/0x127 Nov 29 18:55:16 littleboy kernel: br_handle_frame_finish+0x438/0x472 [bridge] Nov 29 18:55:16 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:55:16 littleboy kernel: br_nf_hook_thresh+0xe2/0x109 [br_netfilter] Nov 29 18:55:16 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:55:16 littleboy kernel: br_nf_pre_routing_finish+0x2c1/0x2ec [br_netfilter] Nov 29 18:55:16 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:55:16 littleboy kernel: ? NF_HOOK.isra.0+0xe4/0x140 [br_netfilter] Nov 29 18:55:16 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:55:16 littleboy kernel: br_nf_pre_routing+0x236/0x24a [br_netfilter] Nov 29 18:55:16 littleboy kernel: ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] Nov 29 18:55:16 littleboy kernel: br_handle_frame+0x277/0x2e0 [bridge] Nov 29 18:55:16 littleboy kernel: ? br_pass_frame_up+0xdd/0xdd [bridge] Nov 29 18:55:16 littleboy kernel: __netif_receive_skb_core.constprop.0+0x4fa/0x6e9 Nov 29 18:55:16 littleboy kernel: ? se_is_idle+0x16/0x34 Nov 29 18:55:16 littleboy kernel: ? place_entity+0x6e/0xae Nov 29 18:55:16 littleboy kernel: __netif_receive_skb_one_core+0x40/0x9c Nov 29 18:55:16 littleboy kernel: process_backlog+0x8c/0x116 Nov 29 18:55:16 littleboy kernel: __napi_poll.constprop.0+0x28/0x124 Nov 29 18:55:16 littleboy kernel: net_rx_action+0x159/0x24f Nov 29 18:55:16 littleboy kernel: __do_softirq+0x126/0x288 Nov 29 18:55:16 littleboy kernel: do_softirq+0x7f/0xab Nov 29 18:55:16 littleboy kernel: </IRQ> Nov 29 18:55:16 littleboy kernel: <TASK> Nov 29 18:55:16 littleboy kernel: __local_bh_enable_ip+0x4c/0x6b Nov 29 18:55:16 littleboy kernel: ip_finish_output2+0x36a/0x39d Nov 29 18:55:16 littleboy kernel: __ip_queue_xmit+0x2d8/0x31f Nov 29 18:55:16 littleboy kernel: __tcp_transmit_skb+0x841/0x8b1 Nov 29 18:55:16 littleboy kernel: tcp_connect+0x7e4/0x86f Nov 29 18:55:16 littleboy kernel: tcp_v4_connect+0x41a/0x47d Nov 29 18:55:16 littleboy kernel: __inet_stream_connect+0xdd/0x314 Nov 29 18:55:16 littleboy kernel: ? percpu_counter_add_batch+0x85/0xa2 Nov 29 18:55:16 littleboy kernel: inet_stream_connect+0x39/0x52 Nov 29 18:55:16 littleboy kernel: __sys_connect+0x68/0xa7 Nov 29 18:55:16 littleboy kernel: __x64_sys_connect+0x14/0x1b Nov 29 18:55:16 littleboy kernel: do_syscall_64+0x68/0x81 Nov 29 18:55:16 littleboy kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Nov 29 18:55:16 littleboy kernel: RIP: 0033:0x1457b313d59b Nov 29 18:55:16 littleboy kernel: Code: 83 ec 18 89 54 24 0c 48 89 34 24 89 7c 24 08 e8 bb fa ff ff 8b 54 24 0c 48 8b 34 24 41 89 c0 8b 7c 24 08 b8 2a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2f 44 89 c7 89 44 24 08 e8 f1 fa ff ff 8b 44 Nov 29 18:55:16 littleboy kernel: RSP: 002b:00007ffe60b9dec0 EFLAGS: 00000293 ORIG_RAX: 000000000000002a Nov 29 18:55:16 littleboy kernel: RAX: ffffffffffffffda RBX: 0000000005b380a0 RCX: 00001457b313d59b Nov 29 18:55:16 littleboy kernel: RDX: 0000000000000010 RSI: 00007ffe60b9df50 RDI: 00000000000002d0 Nov 29 18:55:16 littleboy kernel: RBP: 00007ffe60b9df20 R08: 0000000000000000 R09: 000000000714c230 Nov 29 18:55:16 littleboy kernel: R10: 0000000000000f84 R11: 0000000000000293 R12: 000000000714c288 Nov 29 18:55:16 littleboy kernel: R13: 00007ffe60b9df50 R14: 0000000000000010 R15: 00001457b2f336a0 Nov 29 18:55:16 littleboy kernel: </TASK>

same error this morning, I had a ssh connection in the server interrupted with this: Message from syslogd@littleboy at Nov 28 06:52:58 ... kernel:Kernel panic - not syncing: Fatal exception in interrupt

Hi all, this morning the server was offline. I did a reboot (more than one tbh) and now all seems to works well. Is it possible to understand where was the crash? I gave a look to the diagnostics but I didn't find anything littleboy-diagnostics-20231127-1053.zip

works well without parity

the parity sync requires 20 days, it's going at 10mb/s 😞 I've paused it

Hi all, I'm experiencing a really slow writing speed on my Seagate IRONWOLF PRO 20TB (ST20000NT001) disks: # dd if=/dev/zero of=/mnt/disk1/test bs=128k count=10k 10240+0 records in 10240+0 records out 1342177280 bytes (1.3 GB, 1.2 GiB) copied, 54.2975 s, 24.7 MB/s with old western digital red 8tb disk (WDC_WD80EFZX) and 8tb seagate (ST8000VN0002) all seems fine: dd if=/dev/zero of=/mnt/disk4/test bs=128k count=50k conv=fdatasync 51200+0 records in 51200+0 records out 6710886400 bytes (6.7 GB, 6.2 GiB) copied, 4.56653 s, 1.5 GB/s all disks have the same FS (ZFS), and the two old (but well working) disks have some SMART errors too. littleboy-diagnostics-20231113-1124.zip