BTRFS Cache Drive Bad Tree Block (Solved)

[TheVexedGerman]

Since one of the SSDs in my cache pool started reporting SMART errors I thought it was time to replace it. Since there' a bug in the current Unraid version I thought I could just remove the drive so it wouldn't cause issues before adding a new one later. After stopping the array, removing the failing drive, and starting it again I was greeted by an "Filesystem Unmountable Error". Stopping the array and starting in maintenance mode and hitting the check button I got:

bad tree block 1083446640640, bytenr mismatch, want=1083446640640, have=0
Couldn't read tree root
ERROR: cannot open file system
Opening filesystem to check...

The troubleshooting steps from the FAQ all output some variation of the root being unable to be opened.

 

root@WaterVapor:/# mount -o degraded,usebackuproot,ro /dev/mapper/sdm1 /x
mount: /x: wrong fs type, bad option, bad superblock on /dev/mapper/sdm1, missing codepage or helper program, or other error.

root@WaterVapor:/# btrfs restore -D -v /dev/mapper/sdm1 /mnt/btrfs_save/
bad tree block 1083446640640, bytenr mismatch, want=1083446640640, have=0
Couldn't read tree root
Could not open root, trying backup super
warning, device 4 is missing
bad tree block 1021977804800, bytenr mismatch, want=1021977804800, have=0
ERROR: cannot read chunk root
Could not open root, trying backup super
warning, device 4 is missing
bad tree block 1021977804800, bytenr mismatch, want=1021977804800, have=0
ERROR: cannot read chunk root
Could not open root, trying backup super

root@WaterVapor:/# btrfs check --repair /dev/mapper/sdm1
enabling repair mode
WARNING:

        Do not use --repair unless you are advised to do so by a developer
        or an experienced user, and then only after having accepted that no
        fsck can successfully repair all types of filesystem corruption. Eg.
        some software or hardware bugs can fatally damage a volume.
        The operation will start in 10 seconds.
        Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting repair.
Opening filesystem to check...
bad tree block 1083446640640, bytenr mismatch, want=1083446640640, have=0
Couldn't read tree root
ERROR: cannot open file system

 

Does this mean the cache is FUBAR or is there still something that can be tried? Seems that even the dangerous repair option just gives up.

Edited December 9, 2021 by TheVexedGerman

[JorgeB]

9 minutes ago, TheVexedGerman said:

Does this mean the cache is FUBAR or is there still something that can be tried?

Most likely, you can try #btrfs on IRC or the mailing list like mentioned in the FAQ.

[TheVexedGerman]

OK, thanks.

 

So I gave up on fixing this and just formatted and restored the backup. Though I wonder why I'm even bothering with a "raid 1" BTRFS setup if it can't recover from the loss of a single disk anyway.

Edited December 9, 2021 by TheVexedGerman
Added "fix" for future finders.

[JorgeB]

54 minutes ago, TheVexedGerman said:

if it can't recover from the loss of a single disk anyway.

Something else must have happened, recommend also seeing here for better pool monitoring.

[Peter_7]

To supplement the report from my expirience this is happen usually during sleep/wake up, system freeze etc. Actually this is really common malfunction during using btrfs on production. When using on server the risk of loosing data is smaller as the server is always on, however when on laptop computer accidentally risk is kind of high. I have expedienced such zero page faulty as far as I remember 3 times always with "bad tree block 1021977804800, bytenr mismatch, want=1021977804800, have=0" zero page. I never advice to anyone to install on the desktop or laptop BTRFS as because it is a play with a fire soon or later You will loose the data for sure as an enthusiast of BTRS i have also say this product is still in infant state and one shall always be aware of that. For now the only information which one can have from the developer for that state this is because the hardware, the HW shall protect against write zero page on the disk, Nevertheless is very hard to investigate on that level what is going on and who's is the owner of the bug.