I think this has happened before to me, back when I again replaced my cache - but I don't remember how it got resolved (itself?) and can't find any relevant thread.
(back then it was from a 500GB SSD to a 500GB NVMe)
I am scared if the solution back then was to re-install all my containers... I cannot remember though.

 

So I moved from a 500GB cache drive to a 1TB cache drive.

Process went fine (I actually did it my own way, which is just a bit different from the documented ways), which worked fine.
My VMs work OK, my dockers too.

 

Thing is, used space in my old 500GB cache, was 47%. So it should be LESS than 250GB of data, right?
No new files were written on the cache and it is not from new system files either because I saw the space reported even before enabling VM and docker systems.

Even the copy itself (from old to new cache) reported something around 350GB+?
Indeed new cache is 39% and reports 388GB of data! (more than 100GB more than the old used space!)

Note that I know that 47% (and less than 250GB used) of my old cache were for real, as I happened to use my empty cache space with big file copies that indeed exceeded 200GB (so I was for sure less than 300GB used).

 

Both disks formatted by UNRAID with btrfs.
I even scrubbed the new disk.

What exactly happened? 

Is it some kind of deduplication that btrfs will do by itself?
Some kind of compression?

Something scheduled by filesystem itself? (and in that case can I force it?)
Use of "inline" files or sparse files?
(note that I use a docker FOLDER not a docker FILE - which means thousands of tiny files)
Something else?
Any ideas?

 

How did you copy the files across?   Vdisks for VMs are normally Linux ‘sparse’ files which means the physical space used can be less than the logical size.   However it is possible when copying the vdisk files you used a method which removed the ‘sparse’ part so the files now occupy their logical (larger) size.

2 hours ago, itimpi said:

How did you copy the files across?   Vdisks for VMs are normally Linux ‘sparse’ files which means the physical space used can be less than the logical size.   However it is possible when copying the vdisk files you used a method which removed the ‘sparse’ part so the files now occupy their logical (larger) size.

 

I used Midnight Commander as pre-installed on the host machine concole.

I am not sure of the parameters it uses when moving and not sure it explains the 120+ GB difference (I have just two VMs on my cache disk).

If this is the issue, is it possible I can "re-sparse" the vdisks?

 

On other news, some (but only some) containers presented issues after all, so I re-installed all of them and now it tries to delete the old docker folder (I just renamed it, because it takes WAY too much time to delete - so will delete later)...

 

7 hours ago, NLS said:

If this is the issue, is it possible I can "re-sparse" the vdisks?

You can do that with cp --sparse=always

2 hours ago, JorgeB said:

You can do that with cp --sparse=always

I know that, but I don't know how to do it using MC.

Also, I don't know what to do NOW that I already copied the files (and already using them).
Can I re-shrink them?

 

EDIT: Well, something happened (I also deleted "docker-old" that was the residue of the file copy I did originally, but I don't think this is where the size difference comes from)... and now it is at... 188GB used (probably less than what was on my old disk!).
I still want to know if it is possible to shrink down "un-sparsed" VM disks.

 

Edited November 24, 20223 yr by NLS

10 minutes ago, NLS said:

I still want to know if it is possible to shrink down "un-sparsed" VM disks.

 

MC won't work AFAIK, if you have enough space on cache you can use
 

mv /mnt/path/to/vdisk.img /mnt/path/to/vdisk.old
cp --sparse=always /mnt/path/to/vdisk.old /mnt/path/to/vdisk.img
rm /mnt/path/to/vdisk.old

 

If there's not enough space on cache move to array then copy back with --sparse=always.

There is enough space.
I will try.

But before I try, is there a way to see real vs claimed size?
(to see if I actually succeeded)

 

EDIT: Found it.

Edited November 24, 20223 yr by NLS