BTRFS CSUM ERROR

[LoyalScotsman]

logs attached, daigs attached 

Sep  2 19:12:58 Vault101 kernel: BTRFS warning (device sdg1): checksum verify failed on logical 158164238336 mirror 2 wanted 0x2a076ccd found 0x69294921 level 1

 

ran this to locate file when i go to the location its empty 

root@Vault101:~# find /mnt/cache -inum 158164238336
find: ‘/mnt/cache/appdata/plex/Library/Application Support/Plex Media Server/Metadata/TV Shows/3/495d21813339e6542d1713ccc0ed51d7c636e14.bundle/Contents/_stored/seasons/3/episodes/11’: Input/output error

 

did a scrub check currently still running i think, first time so no sure if correct command for cache as found another forum on similar issue - root@Vault101:~# /usr/local/emhttp/plugins/dynamix/scripts/btrfs_scrub 'start' '/mnt/cache' '-r'

but i have nothing to delete could i please have assistance and input on this please?
his issue caused docker to crash and then had to force a re boot on my server so currently it is doing a parity check.

vault101-diagnostics-20230902-1922.zip

vault101-syslog-20230902-1829.zip

 

 

Edit this appears to be spamming my syslogs now

Sep 2 19:49:08 Vault101 kernel: BTRFS error (device sdg1: state EA): bdev /dev/sdg1 errs: wr 1763, rd 1994530, flush 0, corrupt 2, gen 0

Sep 2 19:49:08 Vault101 kernel: BTRFS error (device sdg1: state EA): bdev /dev/sdg1 errs: wr 1763, rd 1994531, flush 0, corrupt 2, gen 0

Sep 2 19:49:08 Vault101 kernel: BTRFS error (device sdg1: state EA): bdev /dev/sdg1 errs: wr 1763, rd 1994532, flush 0, corrupt 2, gen 0

Edited September 2, 2023 by LoyalScotsman

[JorgeB]

Run a scrub and post new diags.

[LoyalScotsman]

7 minutes ago, JorgeB said:

Run a scrub and post new diags.

can you please confirm  the progress to do this ? 

doing a google found the following command to get cache stats and i think that dont look good.

think one of my cache disks has dropped off but not sure 

root@Vault101:~# btrfs dev stats /mnt/cache
[devid:1].write_io_errs    37728126
[devid:1].read_io_errs     25990032
[devid:1].flush_io_errs    0
[devid:1].corruption_errs  232473
[devid:1].generation_errs  0
[/dev/sdg1].write_io_errs    1763
[/dev/sdg1].read_io_errs     1996297
[/dev/sdg1].flush_io_errs    0
[/dev/sdg1].corruption_errs  2
[/dev/sdg1].generation_errs  0
[/dev/sde1].write_io_errs    243721484
[/dev/sde1].read_io_errs     231592189
[/dev/sde1].flush_io_errs    0
[/dev/sde1].corruption_errs  335203
[/dev/sde1].generation_errs  0

 

[LoyalScotsman]

1 hour ago, JorgeB said:

Run a scrub and post new diags.

found how to do it please see results and fresh logs, hope to here from you soon 

 

 

UUID:             95913f78-fc8f-491e-b13a-789a9383ef8a
Scrub started:    Sat Sep  2 21:19:17 2023
Status:           aborted
Duration:         0:29:00
Total to scrub:   127.51GiB
Rate:             70.81MiB/s
Error summary:    csum=34
  Corrected:      33
  Uncorrectable:  1vault101-diagnostics-20230902-2131.zip
  Unverified:     0
  

 

 

[JorgeB]

The scrub is aborting, that and all those device errors recommend copying what you can from the pool and re-formatting, also see here for better pool monitoring.

[LoyalScotsman]

no problem starting to copy stuff from the pool over to array just now, and re-format the pool and copy back over, hopefully all goes sucesffully cause alot more errors in the logs this morning