sr51nb9d Posted May 29 Share Posted May 29 Hey there, Suddenly I can't start my home assistant VM. I was on vacation when my unraid server went offline. I tried replacing the Cache drive and restarting from scratch via backups. Unfortunately with docker or VM's running, I get the below. I tried replacing the SATA cable. May 29 14:56:46 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:56:46 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 474, gen 0 May 29 14:56:46 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:56:46 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 475, gen 0 May 29 14:56:46 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:56:46 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 476, gen 0 May 29 14:56:46 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:56:46 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 477, gen 0 May 29 14:56:46 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:56:46 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 478, gen 0 May 29 14:57:16 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 479, gen 0 May 29 14:57:16 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 480, gen 0 May 29 14:57:16 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 481, gen 0 May 29 14:57:47 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:47 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 482, gen 0 May 29 14:57:47 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:47 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 483, gen 0 May 29 14:57:47 racer5 kernel: BTRFS warning (device sdc1): csum failed root 5 ino 102864 off 191668224 csum 0x8667902b expected csum 0xbc5b4684 mirror 1 May 29 14:57:47 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 484, gen 0 Does anyone have any ideas on how to fix this? btrfs check --readonly /dev/sdc1 Opening filesystem to check... Checking filesystem on /dev/sdc1 UUID: bd5f1f25-4aa0-4c44-b0ed-cc48da178f60 [1/7] checking root items [2/7] checking extents [3/7] checking free space tree [4/7] checking fs roots [5/7] checking only csums items (without verifying data) [6/7] checking root refs [7/7] checking quota groups skipped (not enabled on this FS) found 63641214976 bytes used, no error found total csum bytes: 41113636 total tree bytes: 62963712 total fs tree bytes: 17170432 total extent tree bytes: 2179072 btree space waste bytes: 5528641 file data blocks allocated: 105935065088 referenced 63543685120 Smartctl passes fine. racer5-diagnostics-20240529-1504.zip Quote Link to comment
JorgeB Posted May 30 Share Posted May 30 Btrfs is detecting data corruption, run a correcting scrub and post the results, also a good idea to run memtest. Quote Link to comment
sr51nb9d Posted May 30 Author Share Posted May 30 Thanks Jorge. Unfortunately they appear to be unfix-able. UUID: bd5f1f25-4aa0-4c44-b0ed-cc48da178f60 Scrub started: Thu May 30 05:21:15 2024 Status: finished Duration: 0:01:52 Total to scrub: 59.33GiB Rate: 542.44MiB/s Error summary: csum=329 Corrected: 0 Uncorrectable: 329 Unverified: 0 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73484632064 on dev /dev/sdc1, physical 33764573184, root 5, inode 81409, offset 23402172416, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1026, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73484632064 on dev /dev/sdc1 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73484054528 on dev /dev/sdc1, physical 33763995648, root 5, inode 81409, offset 23401594880, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1027, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73484054528 on dev /dev/sdc1 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73503387648 on dev /dev/sdc1, physical 33783328768, root 5, inode 81409, offset 23420928000, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1028, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73503387648 on dev /dev/sdc1 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73503330304 on dev /dev/sdc1, physical 33783271424, root 5, inode 81409, offset 23420870656, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1029, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73503330304 on dev /dev/sdc1 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73523089408 on dev /dev/sdc1, physical 33803030528, root 5, inode 81409, offset 23440629760, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1030, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73523089408 on dev /dev/sdc1 May 30 05:22:10 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 73523093504 on dev /dev/sdc1, physical 33803034624, root 5, inode 81409, offset 23440633856, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1031, gen 0 May 30 05:22:10 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 73523093504 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76412858368 on dev /dev/sdc1, physical 36692799488, root 5, inode 81409, offset 26313261056, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1032, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76412858368 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76483960832 on dev /dev/sdc1, physical 36763901952, root 5, inode 81409, offset 26384363520, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76483854336 on dev /dev/sdc1, physical 36763795456, root 5, inode 81409, offset 26384257024, length 4096, links 1 (path: appdata/serge/weights/Mixtral-8X7B-Instruct-v0_1.bin) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1033, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76483960832 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1034, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76483854336 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76591443968 on dev /dev/sdc1, physical 36871385088, root 5, inode 84302, offset 16424960, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db-2024-05-14) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1035, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76591443968 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76719296512 on dev /dev/sdc1, physical 36999237632, root 5, inode 84306, offset 1011712, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db-2024-05-20) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1036, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76719296512 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76877938688 on dev /dev/sdc1, physical 37157879808, root 5, inode 84310, offset 14839808, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.blobs.db) May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76877897728 on dev /dev/sdc1, physical 37157838848, root 5, inode 84310, offset 14798848, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.blobs.db) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1037, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76877938688 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1038, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76877897728 on dev /dev/sdc1 May 30 05:22:16 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 76877967360 on dev /dev/sdc1, physical 37157908480, root 5, inode 84310, offset 14868480, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.blobs.db) May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1039, gen 0 May 30 05:22:16 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 76877967360 on dev /dev/sdc1 May 30 05:22:19 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 78149033984 on dev /dev/sdc1, physical 38428975104, root 5, inode 86827, offset 1560576, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder/ab/ab9666ff6b34977202cab44b05fe1728d65303e0.jpg) May 30 05:22:19 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1040, gen 0 May 30 05:22:19 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 78149079040 on dev /dev/sdc1, physical 38429020160, root 5, inode 86827, offset 1605632, length 4096, links 1 (path: appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Cache/PhotoTranscoder/ab/ab9666ff6b34977202cab44b05fe1728d65303e0.jpg) May 30 05:22:19 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 78149033984 on dev /dev/sdc1 May 30 05:22:19 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1041, gen 0 May 30 05:22:19 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 78149079040 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: scrub_handle_errored_block: 33 callbacks suppressed May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101532372992 on dev /dev/sdc1, physical 61812314112, root 5, inode 102864, offset 1014145024, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: btrfs_dev_stat_inc_and_print: 33 callbacks suppressed May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1075, gen 0 May 30 05:23:00 racer5 kernel: scrub_handle_errored_block: 33 callbacks suppressed May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101532372992 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101549453312 on dev /dev/sdc1, physical 61829394432, root 5, inode 102864, offset 1031225344, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1076, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101549453312 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101549461504 on dev /dev/sdc1, physical 61829402624, root 5, inode 102864, offset 1031233536, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1077, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101549461504 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101852925952 on dev /dev/sdc1, physical 62132867072, root 5, inode 102864, offset 858279936, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1078, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101852925952 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101852930048 on dev /dev/sdc1, physical 62132871168, root 5, inode 102864, offset 858284032, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1079, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101852930048 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101901234176 on dev /dev/sdc1, physical 62181175296, root 5, inode 102864, offset 906588160, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1080, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101901234176 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101947916288 on dev /dev/sdc1, physical 62227857408, root 5, inode 102864, offset 953270272, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1081, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101947916288 on dev /dev/sdc1 May 30 05:23:00 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 101947920384 on dev /dev/sdc1, physical 62227861504, root 5, inode 102864, offset 953274368, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1082, gen 0 May 30 05:23:00 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 101947920384 on dev /dev/sdc1 May 30 05:23:01 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 102041509888 on dev /dev/sdc1, physical 62321451008, root 5, inode 102864, offset 1170399232, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:01 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1083, gen 0 May 30 05:23:01 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 102041509888 on dev /dev/sdc1 May 30 05:23:01 racer5 kernel: BTRFS warning (device sdc1): checksum error at logical 102041513984 on dev /dev/sdc1, physical 62321455104, root 5, inode 102864, offset 1170403328, length 4096, links 1 (path: domains/Home Assistant/haos_ova-12.3.qcow2) May 30 05:23:01 racer5 kernel: BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 0, rd 0, flush 0, corrupt 1084, gen 0 May 30 05:23:01 racer5 kernel: BTRFS error (device sdc1): unable to fixup (regular) error at logical 102041513984 on dev /dev/sdc1 I can't run a memtest at present. I'll have to look into that later when I can throw a GPU in here. Quote Link to comment
Solution JorgeB Posted May 30 Solution Share Posted May 30 You will need to delete/restore the corrupt files listed in the syslog, then run another scrub to confirm 0 errors, but I would recommend memtest first. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.