btrfs raid1 cache wrong free space

[milanp]

I have been using unraid(4 active hosts) for more than 6-7 years, and had no problems when using xfs for cache.
On one of the hosts,which we use as a test platform in our company,I added a second cache drive(same size,same type as the first one).
The requirements for using raid1 was to convert the filesystem to btrfs.
I managed to create the raid1 cache pool, and everything works as intended.
However, when checking the free size from unraid, main, under pool devices, unraid shows 42.6GB free.

If I go to console, start mc, and use command,show directory sizes, the /mnt/cache shows 1448GB out of 1838GB is used.

We use this node only as a virtualization platform, so nothing is used for storage (just a few backups).
We only have shares for appdata,domains and isos.
How can I reclaim the missing space without ruining anything ?

Edited July 27 by milanp

[JorgeB]

The GUI stats will usually be correct, post the output of:

 

btrfs fi usage -T /mnt/cache

 

[milanp]

This is the output:

Overall:
    Device size:                   3.64TiB
    Device allocated:              3.57TiB
    Device unallocated:           71.91GiB
    Device missing:                  0.00B
    Device slack:                    0.00B
    Used:                          3.56TiB
    Free (estimated):             39.70GiB      (min: 39.70GiB)
    Free (statfs, df):            39.69GiB
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)
    Multiple profiles:                  no

             Data    Metadata System                             
Id Path      RAID1   RAID1    RAID1     Unallocated Total   Slack
-- --------- ------- -------- --------- ----------- ------- -----
 1 /dev/sdd1 1.78TiB  4.00GiB  32.00MiB    35.95GiB 1.82TiB     -
 2 /dev/sdb1 1.78TiB  4.00GiB  32.00MiB    35.95GiB 1.82TiB     -
-- --------- ------- -------- --------- ----------- ------- -----
   Total     1.78TiB  4.00GiB  32.00MiB    71.91GiB 3.64TiB 0.00B
   Used      1.78TiB  2.49GiB 320.00KiB  

 

I cannot find where I am missing the space, I only need the cache drive for vm's.

[JorgeB]

As you can see the reported free space is correct, note that vdisks on btrfs can grow in time if not trimmed, see here, another option is to run a defrag, but only do that i you don't use snapshots, or move the vdisks to the array, then move back to the pool with cp --sparse=always

[milanp]

Ok, it seems that btrfs is growing by it own.

There are more than 20 linux vm's on this server, I don't use snapshots, and it's too much hasle to run defrag on each of them.

So if I move the vdisk's elsewhere, and copy each of them back with cp --sparse=always, do I have to do this operation once in a while ?

[JorgeB]

If you don't enable discard in the VMs, you may need to.

[JorgeB]

35 minutes ago, milanp said:

and it's too much hasle to run defrag on each of them.

P.S: I meant defrag the pool, not the VMs.

[milanp]

1 hour ago, JorgeB said:

If you don't enable discard in the VMs, you may need to.

Many thanks for the swift reply. I just need two more confirmations.

By enabling discard in the VMs, You mean to put something like this in each vm:

<disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>

Hopefully this solves my problem....

Are there any limitations using the discard option with using different controler type buses i.e. sata,VirtIO ?

Do I have to move the images, even if I put the discard option in the .xml, or it is enough to stop the vms, ,put the discard in the .xml and restarting the vms should reclaim the space?

Edited July 27 by milanp

[JorgeB]

5 minutes ago, milanp said:

You mean to put something like this in each vm:

Yep, it should work with virtIO and SCSI, not sure about SATA, note that you then should run trim inside the VM at least once after the change.

 

6 minutes ago, milanp said:

even if I put the discard option in the .xml, or it is enough to stop the vms, ,put the discard in the .xml and restarting the vms should reclaim the space?

Doing this and running TRIM on all VMs should reclaim most of the space.