CSUM errors on BTRFS device loop3


Addy

Recommended Posts

 

Mar 31 18:43:38 unRAID kernel: BTRFS warning (device loop3): csum failed root 9866 ino 10569 off 8192 csum 0x9e9cefb8 expected csum 0xc39c48ba mirror 1

 

Hey guys, 

 

I'm trying to figure out what is going on above, seems to spam my logs with it when I start my Plex Docker (which is currently crashing on start). The only BTRFS drives I have is my cache pool.. could one of those be f'd? Smart seems all good on them 

 

That said however, if I run the following

 

root@unRAID:~# btrfs dev stats /mnt/cache/
[/dev/sdh1].write_io_errs    0
[/dev/sdh1].read_io_errs     0
[/dev/sdh1].flush_io_errs    0
[/dev/sdh1].corruption_errs  0
[/dev/sdh1].generation_errs  0
[/dev/sdi1].write_io_errs    3376200
[/dev/sdi1].read_io_errs     1425037
[/dev/sdi1].flush_io_errs    150448
[/dev/sdi1].corruption_errs  0
[/dev/sdi1].generation_errs  0

 

It shows a tone of write, read and flush io errors

 

I do have two cache drives, but I'm not sure how to run dev stats on sdi(2?)

 

Is one of my cache drives dying? I have the unraid diag attached to this post, notable I am getting GPU errors on my GTX 1050 aswell.. but I don't think its related and I assume will fix when I re-install the upgrade unraid and re-install the drivers. 

 

I am going to re-install Plex, maybe this will fix it, but unsure why this has happened, maybe my docker.img is boned?

unraid-diagnostics-20210331-1847.zip

Link to comment
42 minutes ago, JorgeB said:

This is usually the result of a device dropping offline, see below for more info:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=700582

 

That's what likely caused the issues with loop3, which is the docker image and should now be recreated.

 

Thanks for the info!

 

I am running a scrub on one of the SSD's now. It has come back with a couple un-correctable errors

 

What is my best coarse of action from here? Replace that SSD and rebuild? 

Link to comment
5 minutes ago, JorgeB said:

This won't help, if there are uncorrectable errors the corrupt file(s) will be listed in the syslog, you'll need to delete them or restore from backups.

 

yay

 

Okay, syslog attached incase,

 

I see this 

Mar 31 20:26:37 unRAID kernel: BTRFS warning (device sdh1): checksum error at logical 382557040640 on dev /dev/sdi1, physical 59465216000, root 5, inode 8972582, offset 216076288, length 4096, links 1 (path: appdata/binhex-minidlna/files.db)
Mar 31 20:26:37 unRAID kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
Mar 31 20:26:38 unRAID kernel: BTRFS error (device sdh1): unable to fixup (regular) error at logical 382557040640 on dev /dev/sdi1

 

Can I assume the appdata/binhex-minidlna/files.db is corrupt?

 

It says Corrupt 2, but says that file twice

One being on device sdh1 and sdi1

syslog

Link to comment

Seem to still be spamming this into syslog when starting plex however

Mar 31 20:45:58 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:58 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 268, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 269, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 270, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 271, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 272, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 273, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 274, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 275, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 276, gen 0
Mar 31 20:45:59 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:45:59 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 277, gen 0
Mar 31 20:46:04 unRAID kernel: btrfs_print_data_csum_error: 147 callbacks suppressed
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: btrfs_dev_stat_print_on_error: 147 callbacks suppressed
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 425, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 426, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 427, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 428, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 429, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 430, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 431, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 432, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 433, gen 0
Mar 31 20:46:04 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:04 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 434, gen 0
Mar 31 20:46:09 unRAID kernel: btrfs_print_data_csum_error: 214 callbacks suppressed
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: btrfs_dev_stat_print_on_error: 214 callbacks suppressed
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 649, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 650, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 651, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 652, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 653, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 654, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 655, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 656, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 657, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1

 

Link to comment
26 minutes ago, JorgeB said:

Yes.

 

okay have fix that, think there is something corrupt inside the docker.img? Is that loop3? as you stated. I will try re-create it.. hope my plex files are gucci

 

I get a tone of errors of this nature when starting plex 

 

Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 654, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 655, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 656, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 657, gen 0
Mar 31 20:46:09 unRAID kernel: BTRFS warning (device loop3): csum failed root 9874 ino 10569 off 8192 csum 0xb8ef9c9e expected csum 0xba489cc3 mirror 1
Mar 31 20:46:09 unRAID kernel: BTRFS error (device loop3): bdev /dev/loop3 errs: wr 0, rd 0, flush 0, corrupt 658, gen 0

 

 

After docker.img rebuild, plex is gucci again... thanks so much for you help @JorgeB

Edited by Addy
  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.