Skip to content
View in the app

A better way to browse. Learn more.
Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
Unraid Summer Sale in live: 20% off Starter and Unleashed + buy one, get one 50% off

Cache disk zfs and btrfs error (device loop2)

emmenem
in General Support

Featured Replies

Hi folks,

 

I am a new Unraid user who's experiencing the following problem.

A couple of days ago my Unraid NAS went unresponsive. I suspect there was a hw failure connected to my nvme0n1 configured as mirror drive of a zfs cache:

1825806409_Screenshot2023-10-03at20_32_00.thumb.png.1cd585ff16088feaf050563b77e45f29.png

When the error occured, I found in the system log the suggestion to add nvme_core.default_ps_max_latency_us=0 as a boot parameter. After updating the Syslinux Configuration accordingly, I rebooted the NAS. After a successful parity check, the NAS was ready and responsive as before.

 

Unfortunately inside the system log there are now some worrisome lines regarding BTRFS and device loop2 still my cache pool is zfs and my array is xfs!. These lines are repeated in a loop:

  

ct  2 11:01:54 Valbotower emhttpd: mounting /mnt/cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (34): mkdir -p /mnt/cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (35): /usr/sbin/zpool import -N -o autoexpand=on  -d /dev/nvme0n1p1 -d /dev/nvme1n1p1 6308235940963637281 cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (36): /usr/sbin/zfs set mountpoint=/mnt/cache cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (37): /usr/sbin/zfs set atime=off cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (38): /usr/sbin/zfs mount cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (39): /usr/sbin/zpool set autotrim=on cache
Oct  2 11:01:54 Valbotower emhttpd: shcmd (40): /usr/sbin/zfs set compression=on cache
Oct  2 11:01:54 Valbotower emhttpd: /mnt/cache root profile: mirror
Oct  2 11:01:54 Valbotower emhttpd: /mnt/cache root groups: 1
Oct  2 11:01:54 Valbotower emhttpd: /mnt/cache root width: 2
Oct  2 11:01:54 Valbotower emhttpd: /usr/sbin/zpool status -PL cache 2>&1
Oct  2 11:01:54 Valbotower emhttpd:   pool: cache
Oct  2 11:01:54 Valbotower emhttpd:  state: ONLINE
Oct  2 11:01:54 Valbotower emhttpd: status: One or more devices has experienced an unrecoverable error.  An
Oct  2 11:01:54 Valbotower emhttpd:  attempt was made to correct the error.  Applications are unaffected.
Oct  2 11:01:54 Valbotower emhttpd: action: Determine if the device needs to be replaced, and clear the errors
Oct  2 11:01:54 Valbotower emhttpd:  using 'zpool clear' or replace the device with 'zpool replace'.
Oct  2 11:01:54 Valbotower emhttpd:    see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
Oct  2 11:01:54 Valbotower emhttpd:   scan: resilvered 7.20M in 00:00:00 with 0 errors on Mon Oct  2 11:01:54 2023
Oct  2 11:01:54 Valbotower emhttpd: config:
Oct  2 11:01:54 Valbotower emhttpd:  NAME                STATE     READ WRITE CKSUM
Oct  2 11:01:54 Valbotower emhttpd:  cache               ONLINE       0     0     0
Oct  2 11:01:54 Valbotower emhttpd:    mirror-0          ONLINE       0     0     0
Oct  2 11:01:54 Valbotower emhttpd:      /dev/nvme0n1p1  ONLINE       0     0     0
Oct  2 11:01:54 Valbotower emhttpd:      /dev/nvme1n1p1  ONLINE       0     0     2
Oct  2 11:01:54 Valbotower emhttpd: errors: No known data errors
Oct  2 11:01:54 Valbotower emhttpd: shcmd (41): /usr/sbin/zfs mount -a

[...]

Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered blocking state
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered disabled state
Oct  3 20:00:55 Valbotower kernel: device vethdd5a7e5 entered promiscuous mode
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered blocking state
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered forwarding state
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered disabled state
Oct  3 20:00:55 Valbotower kernel: eth0: renamed from veth1829f70
Oct  3 20:00:55 Valbotower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethdd5a7e5: link becomes ready
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered blocking state
Oct  3 20:00:55 Valbotower kernel: br-ead4be2b819d: port 1(vethdd5a7e5) entered forwarding state
Oct  3 20:00:55 Valbotower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): br-ead4be2b819d: link becomes ready
Oct  3 20:02:28 Valbotower kernel: loop: Write error at byte offset 930349056, length 4096.
Oct  3 20:02:28 Valbotower kernel: I/O error, dev loop2, sector 1816952 op 0x1:(WRITE) flags 0x0 phys_seg 4 prio class 2
Oct  3 20:02:28 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931659776, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819648 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931672064, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819672 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931680256, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819688 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931684352, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819696 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 5, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931688448, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819704 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 6, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931692544, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819712 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 7, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931725312, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819776 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 8, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931729408, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819784 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 9, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:29 Valbotower kernel: loop: Write error at byte offset 931733504, length 4096.
Oct  3 20:02:29 Valbotower kernel: I/O error, dev loop2, sector 1819792 op 0x1:(WRITE) flags 0x0 phys_seg 3 prio class 2
Oct  3 20:02:29 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 10, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: lo_write_bvec: 221 callbacks suppressed
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 933494784, length 4096.
Oct  3 20:02:33 Valbotower kernel: blk_print_req_error: 221 callbacks suppressed
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1823232 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:33 Valbotower kernel: btrfs_dev_stat_inc_and_print: 221 callbacks suppressed
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 232, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 931659776, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1819648 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 233, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 930349056, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1817088 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 234, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 932839424, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1821952 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 235, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 933511168, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1823264 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 236, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 932859904, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1821992 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 237, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 932880384, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1822032 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 238, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 933580800, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1823400 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 239, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 932921344, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1822112 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 240, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:33 Valbotower kernel: loop: Write error at byte offset 933597184, length 4096.
Oct  3 20:02:33 Valbotower kernel: I/O error, dev loop2, sector 1823432 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:02:33 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 241, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:02:40 Valbotower kernel: veth6e110ce: renamed from eth0
Oct  3 20:02:40 Valbotower kernel: br-77e8d6079f94: port 2(vethb98866e) entered disabled state
Oct  3 20:02:40 Valbotower kernel: br-77e8d6079f94: port 2(vethb98866e) entered disabled state
Oct  3 20:02:40 Valbotower kernel: device vethb98866e left promiscuous mode
Oct  3 20:02:40 Valbotower kernel: br-77e8d6079f94: port 2(vethb98866e) entered disabled state
Oct  3 20:02:41 Valbotower kernel: br-77e8d6079f94: port 2(veth78b13e2) entered blocking state
Oct  3 20:02:41 Valbotower kernel: br-77e8d6079f94: port 2(veth78b13e2) entered disabled state
Oct  3 20:02:41 Valbotower kernel: device veth78b13e2 entered promiscuous mode
Oct  3 20:02:41 Valbotower kernel: br-77e8d6079f94: port 2(veth78b13e2) entered blocking state
Oct  3 20:02:41 Valbotower kernel: br-77e8d6079f94: port 2(veth78b13e2) entered forwarding state
Oct  3 20:02:41 Valbotower kernel: eth0: renamed from veth013e42e
Oct  3 20:02:41 Valbotower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth78b13e2: link becomes ready
Oct  3 20:03:07 Valbotower kernel: lo_write_bvec: 352 callbacks suppressed
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 933494784, length 4096.
Oct  3 20:03:07 Valbotower kernel: blk_print_req_error: 352 callbacks suppressed
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1823232 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:03:07 Valbotower kernel: btrfs_dev_stat_inc_and_print: 352 callbacks suppressed
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 594, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 930349056, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1817088 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 595, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 932839424, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1821952 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 596, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 931659776, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1819648 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 597, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 932847616, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1821968 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 598, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 930365440, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1817120 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 599, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 932864000, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1822000 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 600, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 931704832, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1819736 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 601, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 933548032, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1823336 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 602, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:07 Valbotower kernel: loop: Write error at byte offset 930398208, length 4096.
Oct  3 20:03:07 Valbotower kernel: I/O error, dev loop2, sector 1817184 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 2
Oct  3 20:03:07 Valbotower kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 603, rd 0, flush 0, corrupt 0, gen 0
Oct  3 20:03:15 Valbotower kernel: veth013e42e: renamed from eth0

 

Please help me understand the cause of this error and if I need to replace the nvme disk.

I have also attached the diagnostics for further reference.

 

Thanks in advance!

 

unraid-diagnostics-20231003-2021.zip

  • Quote
    • Community Expert

    loop2 is the docker image, you need to recreate it.

     

    Also do a zpool clear and then scrub.

  • Quote

    • Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Reply to this topic...
    Go to topic listing

    Account

    Navigation

    Search

    Search

    Configure browser push notifications
    Chrome (Android)
    1. Tap the lock icon next to the address bar.
    2. Tap Permissions → Notifications.
    3. Adjust your preference.
    Chrome (Desktop)
    1. Click the padlock icon in the address bar.
    2. Select Site settings.
    3. Find Notifications and adjust your preference.