fry_the_solid
Members
-
Joined
Solutions
-
fry_the_solid's post in Rsync + --inplace + BTRFS = Lost Storage Space? was marked as the answerThank you again for the advice Jorge, this solution is so much better that what I was using before. Both my primary and backup server are using disks of the same size, and (single-disk) btrfs, so I am able to backup the disks 1:1. Fortunately for me it was a perfect solution. In case anyone is in the same situation and wants to look into it, start with btrbk. I spent about 3 hours writing my own script before I stumbled upon it. It's easy to configure and does just about everything you could want.
However, this alternative may not work for everyone. If the source filesystem was not btrfs, this wouldn't work. Similarly, if the disks in the destination are not equal in size to the source, it will be a challenge/not possible to backup using this solution either. It's true that if the disks are in the same btrfs pool this wouldn't be an issue, but then you need to use either raid0 with no redundancy or raid1cX where at most you're able to utilize 50% of your raw storage capacity. In the context of Unraid, I'd guess that this would be an issue for many users.
edit: I believe I understand the issue now, it is a problem with BTRFS. Whenever you overwrite existing data in BTRFS it is not actually overwritten but rather a new block is written to, but for whatever reason with these inplace transfers (and similar cases like writing to VM disk images) it does not delete the old block leading to unusable/wasted space. You can use a tool called btdu to view how your space is being used; the space that has been overwritten but not deleted is listed as "unreachable". The only solution I found were to run a btrfs defrag (though I didn't test this myself), or transfer the file off of the volume then back on, however neither of these are something you'd want to do often. All in all an unfortunate limitation (bug?) of BTRFS.