September 14, 20232 yr I just replaced my old cache (about 3 years old) drive with a new one this week because I was getting write errors. Now the brand new drive is getting write errors. The log is below and diagnostics are attached. Any help would be appreciated. Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1474997 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 200 key (562445156352 168 4096) itemoff 5630 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1474998 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 201 key (562445164544 168 4096) itemoff 5577 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475000 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 202 key (562445168640 168 4096) itemoff 5524 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475001 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 203 key (562445172736 168 4096) itemoff 5471 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475002 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 204 key (562445176832 168 4096) itemoff 5418 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475003 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 205 key (562445180928 168 4096) itemoff 5365 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475004 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 206 key (562445185024 168 4096) itemoff 5312 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475005 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: item 207 key (562445189120 168 4096) itemoff 5259 itemsize 53 Sep 14 11:16:28 Mediaserv kernel: extent refs 1 gen 719 flags 1 Sep 14 11:16:28 Mediaserv kernel: ref#0: extent data backref root 5 objectid 1475006 offset 0 count 1 Sep 14 11:16:28 Mediaserv kernel: BTRFS error (device nvme0n1p1): block=452270407680 write time tree block corruption detected Sep 14 11:16:28 Mediaserv kernel: BTRFS: error (device nvme0n1p1) in btrfs_commit_transaction:2460: errno=-5 IO failure (Error while writing out transaction) Sep 14 11:16:28 Mediaserv kernel: BTRFS info (device nvme0n1p1: state E): forced readonly Sep 14 11:16:28 Mediaserv kernel: BTRFS warning (device nvme0n1p1: state E): Skipping commit of aborted transaction. Sep 14 11:16:28 Mediaserv kernel: BTRFS: error (device nvme0n1p1: state EA) in cleanup_transaction:1958: errno=-5 IO failure Sep 14 11:16:30 Mediaserv kernel: loop: Write error at byte offset 19011211264, length 4096. Sep 14 11:16:30 Mediaserv kernel: loop: Write error at byte offset 19011297280, length 4096. Sep 14 11:16:30 Mediaserv kernel: I/O error, dev loop2, sector 37131272 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:30 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:30 Mediaserv kernel: I/O error, dev loop2, sector 37131440 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2 Sep 14 11:16:30 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:30 Mediaserv kernel: loop: Write error at byte offset 14657961984, length 4096. Sep 14 11:16:30 Mediaserv kernel: I/O error, dev loop2, sector 28628832 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:30 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:35 Mediaserv kernel: loop: Write error at byte offset 14629277696, length 4096. Sep 14 11:16:35 Mediaserv kernel: loop: Write error at byte offset 19011616768, length 4096. Sep 14 11:16:35 Mediaserv kernel: I/O error, dev loop2, sector 28572808 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:35 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:35 Mediaserv kernel: I/O error, dev loop2, sector 37132064 op 0x1:(WRITE) flags 0x100000 phys_seg 4 prio class 2 Sep 14 11:16:35 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 5, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15072919552, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15341355008, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15072952320, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15341387776, length 4096. Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29439296 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 2 Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15073034240, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15341469696, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15073247232, length 4096. Sep 14 11:16:39 Mediaserv kernel: loop: Write error at byte offset 15341682688, length 4096. Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 6, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29963584 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 7, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29439360 op 0x1:(WRITE) flags 0x1800 phys_seg 16 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 8, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29963648 op 0x1:(WRITE) flags 0x1800 phys_seg 16 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 9, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29439520 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 10, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29963808 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 11, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29439936 op 0x1:(WRITE) flags 0x1800 phys_seg 32 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 12, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: I/O error, dev loop2, sector 29964224 op 0x1:(WRITE) flags 0x1800 phys_seg 32 prio class 2 Sep 14 11:16:39 Mediaserv kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 13, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:39 Mediaserv kernel: BTRFS: error (device loop2) in btrfs_commit_transaction:2460: errno=-5 IO failure (Error while writing out transaction) Sep 14 11:16:39 Mediaserv kernel: BTRFS info (device loop2: state E): forced readonly Sep 14 11:16:39 Mediaserv kernel: BTRFS warning (device loop2: state E): Skipping commit of aborted transaction. Sep 14 11:16:39 Mediaserv kernel: BTRFS: error (device loop2: state EA) in cleanup_transaction:1958: errno=-5 IO failure Sep 14 11:16:40 Mediaserv kernel: lo_write_bvec: 30 callbacks suppressed Sep 14 11:16:40 Mediaserv kernel: loop: Write error at byte offset 19011211264, length 4096. Sep 14 11:16:40 Mediaserv kernel: loop: Write error at byte offset 19011297280, length 4096. Sep 14 11:16:40 Mediaserv kernel: blk_print_req_error: 30 callbacks suppressed Sep 14 11:16:40 Mediaserv kernel: I/O error, dev loop2, sector 37131272 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:40 Mediaserv kernel: btrfs_dev_stat_inc_and_print: 30 callbacks suppressed Sep 14 11:16:40 Mediaserv kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 44, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:40 Mediaserv kernel: I/O error, dev loop2, sector 37131440 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2 Sep 14 11:16:40 Mediaserv kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 45, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:42 Mediaserv kernel: docker0: port 11(veth78f1a95) entered disabled state Sep 14 11:16:42 Mediaserv kernel: veth560cc44: renamed from eth0 Sep 14 11:16:45 Mediaserv kernel: loop: Write error at byte offset 15070703616, length 4096. Sep 14 11:16:45 Mediaserv kernel: loop: Write error at byte offset 14629277696, length 4096. Sep 14 11:16:45 Mediaserv kernel: I/O error, dev loop2, sector 29434968 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:45 Mediaserv kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 46, rd 0, flush 0, corrupt 0, gen 0 Sep 14 11:16:45 Mediaserv kernel: I/O error, dev loop2, sector 28572808 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2 Sep 14 11:16:45 Mediaserv kernel: BTRFS error (device loop2: state EA): bdev /dev/loop2 errs: wr 47, rd 0, flush 0, corrupt 0, gen 0 mediaserv-diagnostics-20230914-1132.zip
September 14, 20232 yr Community Expert 34 minutes ago, krh1009 said: write time tree block corruption detected This usually means bad RAM or other kernel memory corruption, recommend running memtest.
September 14, 20232 yr Author 25 minutes ago, JorgeB said: This usually means bad RAM or other kernel memory corruption, recommend running memtest. OK thanks. I'm also near the end of 3 day parity disk rebuild, that should be done in 3 hours. I'll try the memtest when it's done.
September 14, 20232 yr Does this seem similar? A bunch of folks have been seeing these errors with BTRFS corruption on a cache drive. One fix is to use a different file system. I have switched my cache drive to XFS, but I may go to ZFS and mirror the cache drive.
September 15, 20232 yr Author 3 hours ago, wayner said: Does this seem similar? A bunch of folks have been seeing these errors with BTRFS corruption on a cache drive. One fix is to use a different file system. I have switched my cache drive to XFS, but I may go to ZFS and mirror the cache drive. YES!! Sounds like the same problem. I'm running 6.12.1. I'll try switching file systems to xfs. Thanks
September 15, 20232 yr Author @JorgeB I'm trying to copy my cache drive to XFS formatted NVMe drive in unassigned devices. After I format the drive I get this screen, do you know why?
September 15, 20232 yr Community Expert Looks like it's a bug with UD: https://forums.unraid.net/topic/92462-unassigned-devices-managing-disk-drives-and-remote-shares-outside-of-the-unraid-array/?do=findComment&comment=1306697
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.