Abstract7227

Ah damn you're right, I apologize. I probably missed it because I went from 2 rank, 2/4 3200Mhz, which was fine, to 2 rank, 4/4 3200Mhz. I configured it to 2666Mhz now. Thx for the feedback

My current diagnostics (after reboot, as I cannot get my diagnostics during the crash) trantor-diagnostics-20230309-1712.zip

cause I think this is the 3rd time that the whole server crashes while only spitting out a btrfs error. Anything else I can investigate?

and that can crash the entire server?

This might be a noob question, but why is it that a btrfs error result in my server not being accessible anymore via the webgui (500 error)?

so I have to replace that one? cause that drive is pretty new

I tried that, rebooted my server, got another crash today (logs attached) with btrfs corruption warning. At this point I: replaced my RAM (twice) replaced my motherboard replaced my CPU (twice) replaced my PSU replaced my cache drives The only original parts now are the boot flash drive, the spinning drives (which are xfs formated) and the case. Assuming I can rule a hardware issue out at this point, I am not sure how I should proceed now. I only run a nextcloud instance with a swag reverse proxy and a home assistant instance on there, so it's not like I am doing a lot of crazy things with this. Any recommendations? issue.txt

So I did everything. Replaced RAM, CPU, motherboard, PSU, even installed new power cables to make sure they are not the cause. Even used a different power outlet. The system is still crashing, found one warning in the logs (attached), after googling I found this issue so if I understand this correctly, using the Custom : br0 with a static IP address causes issues? Currently, all of my docker containers have a custom static IP address assigned to them via br0. Am I to understand I have to bridge them all? I don't quite understand why though, is this a bug in unraid? T issue.txt

swapped my motherboard for a ASRock X570M Pro4 a few days back, already got a crash however. CPU is next, PSU after that I was wondering if a faulty OS flash drive could also throw an error like that?

So I tested my RAM for 48-ish hours with no error got another crash today though, relevant logs in attachment. can't reach the server or even ping it at the moment. I googled the error, found some guy that fixed this issue by replacing his motherboard, CPU and RAM. issue.txt

So theres little chance this can solve the problem? Is there a rule of thumb to how many memtest passes I need to be pretty confident about the RAM?

For the record, I am talking about that multiplier setting

Reread your post and something I also did not do is check my RAM speeds, it is set to auto in the BIOS, do I have to be conserned about this or is this only a problem if I made a manual effort in the past to overclock it?

The memtest passed. I am unsure how to proceed next though. To summarize: Given that I already replaced my RAM and ran memtests on both module kits, can I rule out that it is a memory issue? Or is there maybe a BIOS setting for memory that I still need to look at? I also set the "Global C-State Control" setting of the BIOS to "disabled", double checked today that it is indeed off. Something I noticed today that might have an impact however, there is a settings called "Power Supply Idle Control" that was on auto, I now put it on "Typical" (thought I already changed that setting from your first suggestion, but it seems I didn't hit save maybe)

I'll try that, but given that I already replaced the RAM and the previous RAM already passed 3 memtests, I doubt the actual RAM is the issue. Can it be something else?

I reformatted the cache pool and recreated the docker image just to be safe, I did notice an error when logging off this evening though Jan 8 23:18:40 Trantor kernel: ------------[ cut here ]------------ Jan 8 23:18:40 Trantor kernel: WARNING: CPU: 3 PID: 27295 at net/netfilter/nf_conntrack_core.c:1208 __nf_conntrack_confirm+0xa5/0x2cb [nf_conntrack] Jan 8 23:18:40 Trantor kernel: Modules linked in: af_packet xt_nat macvlan nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter vhost_net tun vhost tap kvm_amd ccp kvm md_mod cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 vhost_iotlb xfs dm_crypt dm_mod dax ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge stp llc bonding tls atlantic r8169 realtek amdgpu edac_mce_amd edac_core gpu_sched i2c_algo_bit drm_ttm_helper ttm drm_display_helper drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd agpgart i2c_piix4 cryptd syscopyarea gigabyte_wmi wmi_bmof nvme rapl sysfillrect sysimgblt i2c_core fb_sys_fops k10temp nvme_core ahci libahci wmi thermal button video backlight acpi_cpufreq unix [last unloaded: md_mod] Jan 8 23:18:40 Trantor kernel: CPU: 3 PID: 27295 Comm: kworker/3:0 Not tainted 5.19.17-Unraid #2 Jan 8 23:18:40 Trantor kernel: Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS ELITE V2/B550 AORUS ELITE V2, BIOS F12 01/18/2021 Jan 8 23:18:40 Trantor kernel: Workqueue: events macvlan_process_broadcast [macvlan] Jan 8 23:18:40 Trantor kernel: RIP: 0010:__nf_conntrack_confirm+0xa5/0x2cb [nf_conntrack] Jan 8 23:18:40 Trantor kernel: Code: c6 48 89 44 24 10 e8 dd e2 ff ff 8b 7c 24 04 89 da 89 c6 89 04 24 e8 56 e6 ff ff 84 c0 75 a2 48 8b 85 80 00 00 00 a8 08 74 18 <0f> 0b 8b 34 24 8b 7c 24 04 e8 16 de ff ff e8 2c e3 ff ff e9 7e 01 Jan 8 23:18:40 Trantor kernel: RSP: 0018:ffffc900002fcda0 EFLAGS: 00010202 Jan 8 23:18:40 Trantor kernel: RAX: 0000000000000188 RBX: 0000000000000000 RCX: 68bb66f7d2b120cc Jan 8 23:18:40 Trantor kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffffa04fda94 Jan 8 23:18:40 Trantor kernel: RBP: ffff88825ae58300 R08: a7e9887c0c492187 R09: 785c58fafb367072 Jan 8 23:18:40 Trantor kernel: R10: 38393c2e2c7d7d9b R11: 42de7f8ec750db87 R12: ffffffff82909480 Jan 8 23:18:40 Trantor kernel: R13: 0000000000017a15 R14: ffff888287957400 R15: 0000000000000000 Jan 8 23:18:40 Trantor kernel: FS: 0000000000000000(0000) GS:ffff888ffe0c0000(0000) knlGS:0000000000000000 Jan 8 23:18:40 Trantor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 8 23:18:40 Trantor kernel: CR2: 000000c000249000 CR3: 000000000220a000 CR4: 0000000000750ee0 Jan 8 23:18:40 Trantor kernel: PKRU: 55555554 Jan 8 23:18:40 Trantor kernel: Call Trace: Jan 8 23:18:40 Trantor kernel: <IRQ> Jan 8 23:18:40 Trantor kernel: nf_conntrack_confirm+0x25/0x54 [nf_conntrack] Jan 8 23:18:40 Trantor kernel: nf_hook_slow+0x3d/0x96 Jan 8 23:18:40 Trantor kernel: ? ip_protocol_deliver_rcu+0x164/0x164 Jan 8 23:18:40 Trantor kernel: NF_HOOK.constprop.0+0x79/0xd9 Jan 8 23:18:40 Trantor kernel: ? ip_protocol_deliver_rcu+0x164/0x164 Jan 8 23:18:40 Trantor kernel: __netif_receive_skb_one_core+0x77/0x9c Jan 8 23:18:40 Trantor kernel: process_backlog+0x8c/0x116 Jan 8 23:18:40 Trantor kernel: __napi_poll.constprop.0+0x2b/0x124 Jan 8 23:18:40 Trantor kernel: net_rx_action+0x159/0x24f Jan 8 23:18:40 Trantor kernel: __do_softirq+0x129/0x288 Jan 8 23:18:40 Trantor kernel: do_softirq+0x7f/0xab Jan 8 23:18:40 Trantor kernel: </IRQ> Jan 8 23:18:40 Trantor kernel: <TASK> Jan 8 23:18:40 Trantor kernel: __local_bh_enable_ip+0x4c/0x6b Jan 8 23:18:40 Trantor kernel: netif_rx+0x52/0x5a Jan 8 23:18:40 Trantor kernel: macvlan_broadcast+0x10a/0x150 [macvlan] Jan 8 23:18:40 Trantor kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan] Jan 8 23:18:40 Trantor kernel: process_one_work+0x1ab/0x295 Jan 8 23:18:40 Trantor kernel: worker_thread+0x18b/0x244 Jan 8 23:18:40 Trantor kernel: ? rescuer_thread+0x281/0x281 Jan 8 23:18:40 Trantor kernel: kthread+0xe7/0xef Jan 8 23:18:40 Trantor kernel: ? kthread_complete_and_exit+0x1b/0x1b Jan 8 23:18:40 Trantor kernel: ret_from_fork+0x22/0x30 Jan 8 23:18:40 Trantor kernel: </TASK> Jan 8 23:18:40 Trantor kernel: ---[ end trace 0000000000000000 ]--- should I be worried?

Jan 8 06:15:45 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 06:15:45 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 06:15:45 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 08:48:27 Trantor kernel: usb 1-8: reset high-speed USB device number 4 using xhci_hcd Jan 8 08:48:28 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 08:48:28 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 08:48:28 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Jan 8 19:10:39 Trantor kernel: BTRFS critical (device dm-4): corrupt leaf: block=1773830144 slot=120 extent bytenr=1368440832 len=16384 invalid extent flag, have 0xffff88816be3c410 expect 1 bit set in 0x3 Jan 8 19:10:39 Trantor kernel: BTRFS info (device dm-4): leaf 1773830144 gen 67607 total ptrs 171 free space 6365 owner 2 Jan 8 19:10:39 Trantor kernel: #011item 0 key (1359380480 169 0) itemoff 16250 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 1 key (1359413248 169 0) itemoff 16217 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 2 key (1359495168 169 0) itemoff 16184 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 3 key (1359577088 169 0) itemoff 16151 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 4 key (1359609856 169 0) itemoff 16118 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 5 key (1359626240 169 0) itemoff 16085 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 6 key (1359675392 169 0) itemoff 16052 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 7 key (1359708160 169 0) itemoff 16019 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 8 key (1359757312 169 0) itemoff 15986 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 9 key (1359773696 169 0) itemoff 15953 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 10 key (1359790080 169 0) itemoff 15920 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 11 key (1359855616 169 0) itemoff 15887 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 12 key (1359872000 169 0) itemoff 15854 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 13 key (1359904768 169 0) itemoff 15821 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 14 key (1359921152 169 0) itemoff 15788 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 15 key (1359937536 169 0) itemoff 15755 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 16 key (1359953920 169 0) itemoff 15722 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 17 key (1359986688 169 0) itemoff 15689 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 18 key (1360003072 169 0) itemoff 15656 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 19 key (1360084992 169 0) itemoff 15623 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 20 key (1360117760 169 0) itemoff 15590 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 21 key (1360134144 169 0) itemoff 15557 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 22 key (1360150528 169 0) itemoff 15524 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 23 key (1360166912 169 0) itemoff 15491 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 24 key (1360199680 169 0) itemoff 15458 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 25 key (1360232448 169 0) itemoff 15425 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 26 key (1360248832 169 0) itemoff 15392 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 27 key (1360265216 169 0) itemoff 15359 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 28 key (1360314368 169 0) itemoff 15326 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 66897 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 7 Jan 8 19:10:39 Trantor kernel: #011item 29 key (1360412672 169 0) itemoff 15293 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 30 key (1360445440 169 0) itemoff 15260 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 31 key (1360510976 169 0) itemoff 15227 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 67607 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 7 Jan 8 19:10:39 Trantor kernel: #011item 32 key (1360543744 169 0) itemoff 15194 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 33 key (1360560128 169 0) itemoff 15161 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 34 key (1360576512 169 0) itemoff 15128 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 35 key (1360592896 169 0) itemoff 15095 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 36 key (1360609280 169 0) itemoff 15062 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 37 key (1360658432 169 0) itemoff 15029 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 38 key (1360674816 169 0) itemoff 14996 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 39 key (1360723968 169 0) itemoff 14963 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 40 key (1360756736 169 0) itemoff 14930 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 41 key (1360773120 169 0) itemoff 14897 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 42 key (1360805888 169 0) itemoff 14864 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 43 key (1360936960 169 0) itemoff 14831 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 44 key (1360953344 169 0) itemoff 14798 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 45 key (1360969728 169 0) itemoff 14765 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 46 key (1360986112 169 0) itemoff 14732 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 7 Jan 8 19:10:39 Trantor kernel: #011item 47 key (1361035264 169 0) itemoff 14699 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 48 key (1361100800 169 0) itemoff 14666 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 49 key (1361117184 169 0) itemoff 14633 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 50 key (1361166336 169 0) itemoff 14600 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 51 key (1361182720 169 0) itemoff 14567 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 52 key (1361199104 169 0) itemoff 14534 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 53 key (1361215488 169 0) itemoff 14501 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 54 key (1361248256 169 0) itemoff 14468 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 55 key (1361313792 169 0) itemoff 14435 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 56 key (1361330176 169 0) itemoff 14402 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 57 key (1361510400 169 0) itemoff 14369 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 58 key (1361592320 169 0) itemoff 14336 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 59 key (1361625088 169 0) itemoff 14303 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 60 key (1361723392 169 0) itemoff 14270 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 61 key (1361772544 169 0) itemoff 14237 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 62 key (1361821696 169 0) itemoff 14204 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 63 key (1361854464 169 0) itemoff 14171 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 64 key (1361870848 169 0) itemoff 14138 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 65 key (1361903616 169 0) itemoff 14105 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 66 key (1361952768 169 0) itemoff 14072 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 67 key (1361985536 169 0) itemoff 14039 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 68 key (1362018304 169 0) itemoff 14006 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 69 key (1362083840 169 0) itemoff 13973 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 70 key (1362100224 169 0) itemoff 13940 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 71 key (1362116608 169 0) itemoff 13907 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 72 key (1362231296 169 0) itemoff 13874 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 73 key (1362296832 169 0) itemoff 13841 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 74 key (1362329600 169 0) itemoff 13808 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 75 key (1362345984 169 0) itemoff 13775 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 76 key (1362378752 169 0) itemoff 13742 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 77 key (1362444288 169 0) itemoff 13709 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 78 key (1362477056 169 0) itemoff 13676 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 79 key (1362526208 169 0) itemoff 13643 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 80 key (1362542592 169 0) itemoff 13610 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 81 key (1362558976 169 0) itemoff 13577 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 82 key (1362575360 169 0) itemoff 13544 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 83 key (1362673664 169 0) itemoff 13511 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 84 key (1362706432 169 0) itemoff 13478 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 85 key (1362722816 169 0) itemoff 13445 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 86 key (1362739200 169 0) itemoff 13412 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 87 key (1362804736 169 0) itemoff 13379 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 88 key (1362870272 169 0) itemoff 13346 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 89 key (1363066880 169 0) itemoff 13313 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 90 key (1363083264 169 0) itemoff 13280 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 53897 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 91 key (1363230720 169 0) itemoff 13247 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 92 key (1363329024 169 0) itemoff 13214 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 93 key (1363476480 169 0) itemoff 13181 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 43689 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 94 key (1363804160 169 0) itemoff 13148 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 95 key (1363951616 169 0) itemoff 13115 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 96 key (1364475904 169 0) itemoff 13082 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 43689 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 97 key (1364590592 169 0) itemoff 13049 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 48600 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 98 key (1364951040 169 0) itemoff 13016 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 43689 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 99 key (1365655552 169 0) itemoff 12983 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51182 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 100 key (1365803008 169 0) itemoff 12950 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51182 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 101 key (1365934080 169 0) itemoff 12917 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 102 key (1365983232 169 0) itemoff 12884 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 103 key (1365999616 169 0) itemoff 12851 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 104 key (1366016000 169 0) itemoff 12818 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 105 key (1366048768 169 0) itemoff 12785 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 106 key (1366065152 169 0) itemoff 12752 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 107 key (1366097920 169 0) itemoff 12719 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 108 key (1366130688 169 0) itemoff 12686 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 109 key (1366147072 169 0) itemoff 12653 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 110 key (1366196224 169 0) itemoff 12620 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 111 key (1366228992 169 0) itemoff 12587 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 112 key (1366294528 169 0) itemoff 12554 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 43689 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 113 key (1366573056 169 0) itemoff 12521 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 114 key (1366589440 169 0) itemoff 12488 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 7 Jan 8 19:10:39 Trantor kernel: #011item 115 key (1367375872 169 0) itemoff 12455 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 63156 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 116 key (1367719936 169 0) itemoff 12422 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 117 key (1368064000 169 0) itemoff 12389 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 118 key (1368276992 169 0) itemoff 12356 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51182 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 119 key (1368424448 169 0) itemoff 12323 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 120 key (1368440832 169 0) itemoff 12290 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 18446612688175088656 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 121 key (1368489984 169 0) itemoff 12257 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 122 key (1368506368 169 0) itemoff 12224 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 123 key (1368539136 169 0) itemoff 12191 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 124 key (1368588288 169 0) itemoff 12158 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 46247 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 125 key (1368686592 169 0) itemoff 12125 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 126 key (1368702976 169 0) itemoff 12092 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 127 key (1368735744 169 0) itemoff 12059 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 128 key (1368768512 169 0) itemoff 12026 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 46247 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 129 key (1368784896 169 0) itemoff 11993 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 130 key (1368817664 169 0) itemoff 11960 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 131 key (1368834048 169 0) itemoff 11927 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 132 key (1368850432 169 0) itemoff 11894 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 133 key (1368866816 169 0) itemoff 11861 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 134 key (1368883200 169 0) itemoff 11828 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 135 key (1368915968 169 0) itemoff 11795 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 136 key (1368932352 169 0) itemoff 11762 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 137 key (1368965120 169 0) itemoff 11729 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 43689 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 5 Jan 8 19:10:39 Trantor kernel: #011item 138 key (1369194496 169 0) itemoff 11696 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 139 key (1369243648 169 0) itemoff 11663 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 140 key (1369260032 169 0) itemoff 11630 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 141 key (1369276416 169 0) itemoff 11597 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 142 key (1369358336 169 0) itemoff 11564 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 143 key (1369374720 169 0) itemoff 11531 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 144 key (1369456640 169 0) itemoff 11498 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 145 key (1369489408 169 0) itemoff 11465 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 146 key (1369522176 169 0) itemoff 11432 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 147 key (1369538560 169 0) itemoff 11399 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 148 key (1369587712 169 0) itemoff 11366 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 149 key (1369604096 169 0) itemoff 11333 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 150 key (1369620480 169 0) itemoff 11300 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 151 key (1369636864 169 0) itemoff 11267 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 152 key (1369653248 169 0) itemoff 11234 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 153 key (1369669632 169 0) itemoff 11201 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 154 key (1369686016 169 0) itemoff 11168 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 7 Jan 8 19:10:39 Trantor kernel: #011item 155 key (1369702400 169 0) itemoff 11135 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 156 key (1369718784 169 0) itemoff 11102 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 157 key (1369735168 169 0) itemoff 11069 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 158 key (1369784320 169 0) itemoff 11036 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 159 key (1369800704 169 0) itemoff 11003 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 160 key (1369817088 169 0) itemoff 10970 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 161 key (1369899008 169 0) itemoff 10937 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 162 key (1369915392 169 0) itemoff 10904 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 61535 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 163 key (1370177536 169 0) itemoff 10871 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 164 key (1370259456 169 0) itemoff 10838 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 165 key (1370275840 169 0) itemoff 10805 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 166 key (1370292224 169 0) itemoff 10772 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 167 key (1370324992 169 0) itemoff 10739 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 168 key (1370374144 169 0) itemoff 10706 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 169 key (1370406912 169 0) itemoff 10673 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: #011item 170 key (1370439680 169 0) itemoff 10640 itemsize 33 Jan 8 19:10:39 Trantor kernel: #011#011extent refs 1 gen 51331 flags 2 Jan 8 19:10:39 Trantor kernel: #011#011ref#0: tree block backref root 2 Jan 8 19:10:39 Trantor kernel: BTRFS error (device dm-4): block=1773830144 write time tree block corruption detected Jan 8 19:10:39 Trantor kernel: BTRFS: error (device dm-4) in btrfs_commit_transaction:2418: errno=-5 IO failure (Error while writing out transaction) Jan 8 19:10:39 Trantor kernel: BTRFS info (device dm-4: state E): forced readonly Jan 8 19:10:39 Trantor kernel: BTRFS warning (device dm-4: state E): Skipping commit of aborted transaction. Jan 8 19:10:39 Trantor kernel: BTRFS: error (device dm-4: state EA) in cleanup_transaction:1982: errno=-5 IO failure Jan 8 19:10:40 Trantor kernel: traps: postgres[9654] general protection fault ip:145788c64602 sp:7fff44a95fb0 error:0 in libc-2.31.so[145788c64000+15a000] Jan 8 19:10:40 Trantor kernel: veth4684853: renamed from eth0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 9119488 op 0x1:(WRITE) flags 0x800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 9119488 op 0x1:(WRITE) flags 0x800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 9119488 op 0x1:(WRITE) flags 0x800 phys_seg 3 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 9119408 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 4902448 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 5, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 1101824 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 6, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 577536 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 7, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 1101856 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 8, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 577568 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 9, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: I/O error, dev loop2, sector 577600 op 0x1:(WRITE) flags 0x1800 phys_seg 44 prio class 0 Jan 8 19:10:40 Trantor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 10, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:10:40 Trantor kernel: BTRFS: error (device loop2) in btrfs_commit_transaction:2418: errno=-5 IO failure (Error while writing out transaction) Jan 8 19:10:40 Trantor kernel: BTRFS info (device loop2: state E): forced readonly Jan 8 19:10:40 Trantor kernel: BTRFS warning (device loop2: state E): Skipping commit of aborted transaction. Jan 8 19:10:40 Trantor kernel: BTRFS: error (device loop2: state EA) in cleanup_transaction:1982: errno=-5 IO failure Jan 8 19:10:46 Trantor kernel: blk_print_req_error: 4 callbacks suppressed Jan 8 19:10:46 Trantor kernel: I/O error, dev loop2, sector 9119408 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0 Jan 8 19:10:46 Trantor kernel: btrfs_dev_stat_print_on_error: 4 callbacks suppressed Jan 8 19:10:46 Trantor kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 15, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:11:12 Trantor kernel: I/O error, dev loop2, sector 4902448 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0 Jan 8 19:11:12 Trantor kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 16, rd 0, flush 0, corrupt 0, gen 0 Jan 8 19:11:12 Trantor kernel: I/O error, dev loop2, sector 5015672 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 0 Jan 8 19:11:12 Trantor kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 17, rd 0, flush 0, corrupt 0, gen 0 Let me guess, reformat the cache pool?

Hi, While updating a docker image today, I got the following log entry Jan 6 08:17:26 Trantor kernel: BUG: unable to handle page fault for address: ffffffff0422dfc6 Jan 6 08:17:26 Trantor kernel: #PF: supervisor write access in kernel mode Jan 6 08:17:26 Trantor kernel: #PF: error_code(0x0002) - not-present page Jan 6 08:17:26 Trantor kernel: PGD 220e067 P4D 220e067 PUD 0 Jan 6 08:17:26 Trantor kernel: Oops: 0002 [#1] PREEMPT SMP NOPTI Jan 6 08:17:26 Trantor kernel: CPU: 1 PID: 8603 Comm: dockerd Tainted: G W 5.19.17-Unraid #2 Jan 6 08:17:26 Trantor kernel: Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS ELITE V2/B550 AORUS ELITE V2, BIOS F12 01/18/2021 Jan 6 08:17:26 Trantor kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x153/0x1d0 Jan 6 08:17:26 Trantor kernel: Code: b9 01 00 00 00 f0 0f b1 0b 74 76 eb cc c1 ee 12 83 e0 03 ff ce 48 c1 e0 05 48 63 f6 48 05 00 ce 02 00 48 03 04 f5 e0 6a 16 82 <48> 89 10 8b 42 08 85 c0 75 04 f3 90 eb f5 48 8b 32 48 85 f6 74 bc Jan 6 08:17:26 Trantor kernel: RSP: 0018:ffffc900020b7a20 EFLAGS: 00010086 Jan 6 08:17:26 Trantor kernel: RAX: ffffffff0422dfc6 RBX: ffff888ae6d70398 RCX: 0000000000080000 Jan 6 08:17:26 Trantor kernel: RDX: ffff888ffe06ce00 RSI: 0000000000002b00 RDI: ffff888ae6d70398 Jan 6 08:17:26 Trantor kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000 Jan 6 08:17:26 Trantor kernel: R10: ffff888b301e5730 R11: ffffc900020b79e8 R12: ffff888ffe06ce00 Jan 6 08:17:26 Trantor kernel: R13: 0000000000000000 R14: 0000000000000246 R15: 0000000000000000 Jan 6 08:17:26 Trantor kernel: FS: 000015213f7db700(0000) GS:ffff888ffe040000(0000) knlGS:0000000000000000 Jan 6 08:17:26 Trantor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 6 08:17:26 Trantor kernel: CR2: ffffffff0422dfc6 CR3: 000000016927c000 CR4: 0000000000750ee0 Jan 6 08:17:26 Trantor kernel: PKRU: 55555554 Jan 6 08:17:26 Trantor kernel: Call Trace: Jan 6 08:17:26 Trantor kernel: <TASK> Jan 6 08:17:26 Trantor kernel: do_raw_spin_lock+0x14/0x1a Jan 6 08:17:26 Trantor kernel: _raw_spin_lock_irqsave+0x2c/0x37 Jan 6 08:17:26 Trantor kernel: __wake_up_common_lock+0x59/0xad Jan 6 08:17:26 Trantor kernel: clear_state_bit+0xe0/0x14b Jan 6 08:17:26 Trantor kernel: __clear_extent_bit+0x29f/0x320 Jan 6 08:17:26 Trantor kernel: clear_extent_bit+0x15/0x1e Jan 6 08:17:26 Trantor kernel: btrfs_evict_inode+0x1c3/0x3cb Jan 6 08:17:26 Trantor kernel: ? autoremove_wake_function+0x33/0x33 Jan 6 08:17:26 Trantor kernel: evict+0xbc/0x150 Jan 6 08:17:26 Trantor kernel: __dentry_kill+0xcb/0x131 Jan 6 08:17:26 Trantor kernel: shrink_dentry_list+0xaa/0xba Jan 6 08:17:26 Trantor kernel: shrink_dcache_parent+0xf3/0x118 Jan 6 08:17:26 Trantor kernel: d_invalidate+0x74/0xdd Jan 6 08:17:26 Trantor kernel: ? btrfs_subvolume_release_metadata+0x3b/0x5a Jan 6 08:17:26 Trantor kernel: btrfs_delete_subvolume+0x3cc/0x505 Jan 6 08:17:26 Trantor kernel: btrfs_ioctl_snap_destroy+0x434/0x516 Jan 6 08:17:26 Trantor kernel: btrfs_ioctl+0x243/0x2843 Jan 6 08:17:26 Trantor kernel: ? __do_sys_newfstatat+0x35/0x5c Jan 6 08:17:26 Trantor kernel: vfs_ioctl+0x1e/0x2f Jan 6 08:17:26 Trantor kernel: __do_sys_ioctl+0x52/0x78 Jan 6 08:17:26 Trantor kernel: do_syscall_64+0x6b/0x81 Jan 6 08:17:26 Trantor kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd Jan 6 08:17:26 Trantor kernel: RIP: 0033:0x4baa7b Jan 6 08:17:26 Trantor kernel: Code: e8 2a e5 fa ff eb 88 cc cc cc cc cc cc cc cc e8 db 2c fb ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30 Jan 6 08:17:26 Trantor kernel: RSP: 002b:000000c002aefac0 EFLAGS: 00000206 ORIG_RAX: 0000000000000010 Jan 6 08:17:26 Trantor kernel: RAX: ffffffffffffffda RBX: 000000c00005e500 RCX: 00000000004baa7b Jan 6 08:17:26 Trantor kernel: RDX: 000000c002aefb60 RSI: 000000005000940f RDI: 000000000000001b Jan 6 08:17:26 Trantor kernel: RBP: 000000c002af0ca8 R08: 000000c001328001 R09: 0000000000000000 Jan 6 08:17:26 Trantor kernel: R10: 0000000000000008 R11: 0000000000000206 R12: 000000c000bc5a90 Jan 6 08:17:26 Trantor kernel: R13: 0000000000000000 R14: 000000c000a77860 R15: 0000000000000000 Jan 6 08:17:26 Trantor kernel: </TASK> Jan 6 08:17:26 Trantor kernel: Modules linked in: af_packet cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost xt_nat vhost_iotlb tap xt_tcpudp macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs dm_crypt dm_mod dax md_mod ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge stp llc bonding tls atlantic r8169 realtek amdgpu edac_mce_amd edac_core gpu_sched i2c_algo_bit drm_ttm_helper kvm_amd ttm drm_display_helper kvm drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd i2c_piix4 cryptd gigabyte_wmi wmi_bmof agpgart nvme rapl i2c_core k10temp syscopyarea nvme_core sysfillrect ccp sysimgblt fb_sys_fops ahci libahci wmi thermal button video backlight acpi_cpufreq unix [last unloaded: atlantic] Jan 6 08:17:26 Trantor kernel: CR2: ffffffff0422dfc6 Jan 6 08:17:26 Trantor kernel: ---[ end trace 0000000000000000 ]--- Jan 6 08:17:26 Trantor kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x153/0x1d0 Jan 6 08:17:26 Trantor kernel: Code: b9 01 00 00 00 f0 0f b1 0b 74 76 eb cc c1 ee 12 83 e0 03 ff ce 48 c1 e0 05 48 63 f6 48 05 00 ce 02 00 48 03 04 f5 e0 6a 16 82 <48> 89 10 8b 42 08 85 c0 75 04 f3 90 eb f5 48 8b 32 48 85 f6 74 bc Jan 6 08:17:26 Trantor kernel: RSP: 0018:ffffc900020b7a20 EFLAGS: 00010086 Jan 6 08:17:26 Trantor kernel: RAX: ffffffff0422dfc6 RBX: ffff888ae6d70398 RCX: 0000000000080000 Jan 6 08:17:26 Trantor kernel: RDX: ffff888ffe06ce00 RSI: 0000000000002b00 RDI: ffff888ae6d70398 Jan 6 08:17:26 Trantor kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000 Jan 6 08:17:26 Trantor kernel: R10: ffff888b301e5730 R11: ffffc900020b79e8 R12: ffff888ffe06ce00 Jan 6 08:17:26 Trantor kernel: R13: 0000000000000000 R14: 0000000000000246 R15: 0000000000000000 Jan 6 08:17:26 Trantor kernel: FS: 000015213f7db700(0000) GS:ffff888ffe040000(0000) knlGS:0000000000000000 Jan 6 08:17:26 Trantor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 6 08:17:26 Trantor kernel: CR2: ffffffff0422dfc6 CR3: 000000016927c000 CR4: 0000000000750ee0 Jan 6 08:17:26 Trantor kernel: PKRU: 55555554 Jan 6 08:17:26 Trantor kernel: note: dockerd[8603] exited with preempt_count 2 Can this be some aftermath of the c-state issue that is now fixed or should I worry about this? I assume I have to recreate the docker.img again?

I went through the bios again and found the "Global C-states" setting, which I disabled now, not sure how I missed it previously. So do I still have to worry about my cache pool at this point? Also, can I assume that the RAM that was in there was probably fine? Every memtest I ran seemed to succeed

So I have this device: Cache Samsung_SSD_980_PRO_1TB - 1 TB (nvme0n1) I'm guessing the p1 refers to a partition on this drive? So given that I replaced those drives and reformatted that cache pool, the issue is the motherboard? I recently had another crash with the following log entry: Dec 24 09:29:20 Trantor kernel: usb 1-8: reset high-speed USB device number 5 using xhci_hcd Dec 24 09:29:20 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Dec 24 09:29:20 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Dec 24 09:29:20 Trantor kernel: program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Dec 25 02:00:00 Trantor kernel: BUG: kernel NULL pointer dereference, address: 0000000000000000 Dec 25 02:00:00 Trantor kernel: #PF: supervisor write access in kernel mode Dec 25 02:00:00 Trantor kernel: #PF: error_code(0x0002) - not-present page Dec 25 02:00:00 Trantor kernel: PGD 0 P4D 0 Dec 25 02:00:00 Trantor kernel: Oops: 0002 [#1] PREEMPT SMP NOPTI Dec 25 02:00:00 Trantor kernel: CPU: 4 PID: 27426 Comm: php-fpm8 Tainted: G W 5.19.17-Unraid #2 Dec 25 02:00:00 Trantor kernel: Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS ELITE V2/B550 AORUS ELITE V2, BIOS F12 01/18/2021 Dec 25 02:00:00 Trantor kernel: RIP: 0010:__rb_erase_color+0xe7/0x1ca Dec 25 02:00:00 Trantor kernel: Code: 8b 6b 10 f6 45 00 01 75 2f 4c 8b 75 08 48 89 d8 48 89 ee 31 c9 48 83 c8 01 4c 89 ea 48 89 df 4c 89 73 10 48 89 5d 08 4c 89 f5 <49> 89 06 e8 d8 fe ff ff 2e e8 9a be 79 00 48 8b 45 10 48 85 c0 74 Dec 25 02:00:00 Trantor kernel: RSP: 0018:ffffc900026dfc20 EFLAGS: 00010282 Dec 25 02:00:00 Trantor kernel: RAX: ffff888b8aee8d19 RBX: ffff888b8aee8d18 RCX: 0000000000000000 Dec 25 02:00:00 Trantor kernel: RDX: ffff888880dee9b0 RSI: ffff888b8aee8b98 RDI: ffff888b8aee8d18 Dec 25 02:00:00 Trantor kernel: RBP: 0000000000000000 R08: ffff888aff800140 R09: 0000000080400006 Dec 25 02:00:00 Trantor kernel: R10: ffff888aff800140 R11: 0000000000007cf5 R12: ffffffff811c69da Dec 25 02:00:00 Trantor kernel: R13: ffff888880dee9b0 R14: 0000000000000000 R15: ffff8888997df600 Dec 25 02:00:00 Trantor kernel: FS: 0000000000000000(0000) GS:ffff888ffe100000(0000) knlGS:0000000000000000 Dec 25 02:00:00 Trantor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Dec 25 02:00:00 Trantor kernel: CR2: 0000000000000000 CR3: 00000001a798a000 CR4: 0000000000750ee0 Dec 25 02:00:00 Trantor kernel: PKRU: 55555554 Dec 25 02:00:00 Trantor kernel: Call Trace: Dec 25 02:00:00 Trantor kernel: <TASK> Dec 25 02:00:00 Trantor kernel: unlink_file_vma+0x45/0x56 Dec 25 02:00:00 Trantor kernel: free_pgtables+0x88/0xc4 Dec 25 02:00:00 Trantor kernel: exit_mmap+0xeb/0x15a Dec 25 02:00:00 Trantor kernel: __mmput+0x43/0xdb Dec 25 02:00:00 Trantor kernel: do_exit+0x30b/0x8e5 Dec 25 02:00:00 Trantor kernel: ? try_to_wake_up+0x20e/0x248 Dec 25 02:00:00 Trantor kernel: do_group_exit+0x8f/0x8f Dec 25 02:00:00 Trantor kernel: get_signal+0x606/0x63e Dec 25 02:00:00 Trantor kernel: ? complete_signal+0x126/0x15e Dec 25 02:00:00 Trantor kernel: arch_do_signal_or_restart+0x36/0x607 Dec 25 02:00:00 Trantor kernel: ? _raw_spin_unlock_irqrestore+0x24/0x3a Dec 25 02:00:00 Trantor kernel: ? do_send_sig_info+0x68/0x8e Dec 25 02:00:00 Trantor kernel: ? kill_pid_info+0x4e/0x62 Dec 25 02:00:00 Trantor kernel: ? __do_sys_kill+0x53/0x15a Dec 25 02:00:00 Trantor kernel: exit_to_user_mode_prepare+0x58/0x10d Dec 25 02:00:00 Trantor kernel: syscall_exit_to_user_mode+0x18/0x27 Dec 25 02:00:00 Trantor kernel: do_syscall_64+0x77/0x81 Dec 25 02:00:00 Trantor kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd Dec 25 02:00:00 Trantor kernel: RIP: 0033:0x1548c72f9415 Dec 25 02:00:00 Trantor kernel: Code: Unable to access opcode bytes at RIP 0x1548c72f93eb. Dec 25 02:00:00 Trantor kernel: RSP: 002b:00007ffe43563ba0 EFLAGS: 00000246 ORIG_RAX: 000000000000003e Dec 25 02:00:00 Trantor kernel: RAX: 0000000000000000 RBX: 00001548c7349b7c RCX: 00001548c72f9415 Dec 25 02:00:00 Trantor kernel: RDX: 0000000000000000 RSI: 000000000000000f RDI: 0000000000000120 Dec 25 02:00:00 Trantor kernel: RBP: 00007ffe43563c30 R08: 0000000000000000 R09: 0000000000000000 Dec 25 02:00:00 Trantor kernel: R10: 0000000000000008 R11: 0000000000000246 R12: 00007ffe43563bb0 Dec 25 02:00:00 Trantor kernel: R13: 000000000000000f R14: 0000000000000000 R15: 0000000000000000 Dec 25 02:00:00 Trantor kernel: </TASK> Dec 25 02:00:00 Trantor kernel: Modules linked in: af_packet xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle xt_nat vhost_net tun vhost vhost_iotlb tap xt_tcpudp macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs dm_crypt dm_mod dax cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver md_mod ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge stp llc bonding tls atlantic r8169 realtek amdgpu gpu_sched i2c_algo_bit drm_ttm_helper ttm drm_display_helper drm_kms_helper drm edac_mce_amd edac_core agpgart kvm_amd gigabyte_wmi wmi_bmof kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel i2c_piix4 crypto_simd cryptd rapl nvme k10temp i2c_core nvme_core ahci syscopyarea sysfillrect ccp libahci sysimgblt fb_sys_fops wmi thermal button video backlight acpi_cpufreq unix [last unloaded: atlantic] Dec 25 02:00:00 Trantor kernel: CR2: 0000000000000000 Dec 25 02:00:00 Trantor kernel: ---[ end trace 0000000000000000 ]--- Dec 25 02:00:00 Trantor kernel: RIP: 0010:__rb_erase_color+0xe7/0x1ca Dec 25 02:00:00 Trantor kernel: Code: 8b 6b 10 f6 45 00 01 75 2f 4c 8b 75 08 48 89 d8 48 89 ee 31 c9 48 83 c8 01 4c 89 ea 48 89 df 4c 89 73 10 48 89 5d 08 4c 89 f5 <49> 89 06 e8 d8 fe ff ff 2e e8 9a be 79 00 48 8b 45 10 48 85 c0 74 Dec 25 02:00:00 Trantor kernel: RSP: 0018:ffffc900026dfc20 EFLAGS: 00010282 Dec 25 02:00:00 Trantor kernel: RAX: ffff888b8aee8d19 RBX: ffff888b8aee8d18 RCX: 0000000000000000 Dec 25 02:00:00 Trantor kernel: RDX: ffff888880dee9b0 RSI: ffff888b8aee8b98 RDI: ffff888b8aee8d18 Dec 25 02:00:00 Trantor kernel: RBP: 0000000000000000 R08: ffff888aff800140 R09: 0000000080400006 Dec 25 02:00:00 Trantor kernel: R10: ffff888aff800140 R11: 0000000000007cf5 R12: ffffffff811c69da Dec 25 02:00:00 Trantor kernel: R13: ffff888880dee9b0 R14: 0000000000000000 R15: ffff8888997df600 Dec 25 02:00:00 Trantor kernel: FS: 0000000000000000(0000) GS:ffff888ffe100000(0000) knlGS:0000000000000000 Dec 25 02:00:00 Trantor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Dec 25 02:00:00 Trantor kernel: CR2: 0000000000000000 CR3: 00000001a798a000 CR4: 0000000000750ee0 Dec 25 02:00:00 Trantor kernel: PKRU: 55555554 Dec 25 02:00:00 Trantor kernel: Fixing recursive fault but reboot is needed! Dec 25 02:00:00 Trantor kernel: BUG: scheduling while atomic: php-fpm8/27426/0x00000000 Dec 25 02:00:00 Trantor kernel: Modules linked in: af_packet xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle xt_nat vhost_net tun vhost vhost_iotlb tap xt_tcpudp macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs dm_crypt dm_mod dax cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver md_mod ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge stp llc bonding tls atlantic r8169 realtek amdgpu gpu_sched i2c_algo_bit drm_ttm_helper ttm drm_display_helper drm_kms_helper drm edac_mce_amd edac_core agpgart kvm_amd gigabyte_wmi wmi_bmof kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel i2c_piix4 crypto_simd cryptd rapl nvme k10temp i2c_core nvme_core ahci syscopyarea sysfillrect ccp libahci sysimgblt fb_sys_fops wmi thermal button video backlight acpi_cpufreq unix [last unloaded: atlantic] Dec 25 02:00:00 Trantor kernel: Preemption disabled at: Dec 25 02:00:00 Trantor kernel: [<0000000000000000>] 0x0 Dec 25 02:00:00 Trantor kernel: CPU: 4 PID: 27426 Comm: php-fpm8 Tainted: G D W 5.19.17-Unraid #2 Dec 25 02:00:00 Trantor kernel: Hardware name: Gigabyte Technology Co., Ltd. B550 AORUS ELITE V2/B550 AORUS ELITE V2, BIOS F12 01/18/2021 Dec 25 02:00:00 Trantor kernel: Call Trace: Dec 25 02:00:00 Trantor kernel: <TASK> Dec 25 02:00:00 Trantor kernel: dump_stack_lvl+0x44/0x5c Dec 25 02:00:00 Trantor kernel: __schedule_bug+0xa3/0xb6 Dec 25 02:00:00 Trantor kernel: __schedule+0x59/0x5f6 Dec 25 02:00:00 Trantor kernel: ? _printk+0x58/0x73 Dec 25 02:00:00 Trantor kernel: do_task_dead+0x3f/0x41 Dec 25 02:00:00 Trantor kernel: make_task_dead+0xb2/0xba Dec 25 02:00:00 Trantor kernel: rewind_stack_and_make_dead+0x17/0x17 Dec 25 02:00:00 Trantor kernel: RIP: 0033:0x1548c72f9415 Dec 25 02:00:00 Trantor kernel: Code: Unable to access opcode bytes at RIP 0x1548c72f93eb. Dec 25 02:00:00 Trantor kernel: RSP: 002b:00007ffe43563ba0 EFLAGS: 00000246 ORIG_RAX: 000000000000003e Dec 25 02:00:00 Trantor kernel: RAX: 0000000000000000 RBX: 00001548c7349b7c RCX: 00001548c72f9415 Dec 25 02:00:00 Trantor kernel: RDX: 0000000000000000 RSI: 000000000000000f RDI: 0000000000000120 Dec 25 02:00:00 Trantor kernel: RBP: 00007ffe43563c30 R08: 0000000000000000 R09: 0000000000000000 Dec 25 02:00:00 Trantor kernel: R10: 0000000000000008 R11: 0000000000000246 R12: 00007ffe43563bb0 Dec 25 02:00:00 Trantor kernel: R13: 000000000000000f R14: 0000000000000000 R15: 0000000000000000 Dec 25 02:00:00 Trantor kernel: </TASK>

How would I identify that? I took a look at the logs after reboot and searched for dm-5 and found this Dec 20 22:04:02 Trantor emhttpd: shcmd (83): /usr/sbin/cryptsetup luksOpen /dev/nvme0n1p1 nvme0n1p1 --allow-discards Dec 20 22:04:04 Trantor kernel: BTRFS: device fsid ba3dc23d-8bbb-4684-81ad-65242a2c1ca0 devid 1 transid 23179 /dev/dm-5 scanned by udevd (3753) Dec 20 22:04:04 Trantor emhttpd: shcmd (86): /usr/sbin/cryptsetup luksOpen /dev/nvme1n1p1 nvme1n1p1 --allow-discards Dec 20 22:04:06 Trantor kernel: BTRFS: device fsid ba3dc23d-8bbb-4684-81ad-65242a2c1ca0 devid 2 transid 23179 /dev/dm-6 scanned by udevd (3753) makes me think dm-5 is one of my replaced nvme drives

so I swapped out both my cache disks about 2 weeks ago and reformatted my cache pool. Had a crash today, logs don't show any error at the time of crash, but a few days ago did report a btrfs warning (see attached) maybe important, currently my syslog destination is another unraid server, but at the point of crash I could not get into the webgui or even sent pings to the crashing server, maybe the relevant logs are never sent to the destination, should I enable mirror syslog to flash? Also maybe important, the NIC used is an AQN-107 10Gbit controller, I don't see it on the supported network controllers list. It seems to work fine, but maybe its not 100% fine, causing this issue? trantor crash.txt

So this might be a dumb question, but why would btrfs crash because of one drive acting funny? from what I understand, my cache pool is configured in RAID1

only the one device?

I got a crash again today, included the logs from the syslog server of the relevant time period error trantor.txt