Trying to Understand Cache Usage

[Xyler94]

Hello,

I've got an odd thing happening. I'm completely unsure what's been using up all my data in my cache pools.

I look at my shares, and calculating all my shares that use my cache pools comes up to around 348GB of usage, but unraid is reporting the cache pools are at 450GB roughly of usage. I don't know where the extra almost 100GB is coming from, and it's been filling my cache drives (500GB SSDs) full.

 

I was planning on buying more storage, but I'm not sure what's even causing my cache drives to fill up, and if will just be the same with more storage.

 

What's the best way to check up on what's going on here? I'm baffled at this particular oddity. 

[JorgeB]

Please post the diagnostics, but most likely the GUI is reporting the correct stats.

[Xyler94]

9 minutes ago, JorgeB said:

Please post the diagnostics, but most likely the GUI is reporting the correct stats.

Here is the diagnostics file.

xylerserver-diagnostics-20240820-1025.zip

[JorgeB]

    Free (statfs, df):          41.46GiB

 

This is what btrfs is reporting, so the GUI is correct, image type files can grow with time on btrfs, especially vdisks if not using discard, if you don't have snapshots you can try to run defrag, alternatively, move the files to the array and see what was using the space, if there were ballooned files, moving them back should then result in less used space.

[Xyler94]

2 minutes ago, JorgeB said:

    Free (statfs, df):          41.46GiB

 

This is what btrfs is reporting, so the GUI is correct, image type files can grow with time on btrfs, especially vdisks if not using discard, if you don't have snapshots you can try to run defrag, alternatively, move the files to the array and see what was using the space, if there were ballooned files, moving them back should then result in less used space.

Okay, sounds good.

I'll give it a shot. It may be my VMs causing this then, I'll have to shut down the VMs using the Cache pool, try a move and move back, and see if that helps.

Thank you for your time!

[JorgeB]

1 minute ago, Xyler94 said:

I'll have to shut down the VMs using the Cache pool, try a move and move back,

For best results move them back with cp --sparse=always and then add the discard option to the VM(s).

[Xyler94]

37 minutes ago, JorgeB said:

For best results move them back with cp --sparse=always and then add the discard option to the VM(s).

Just a quick question while I'm trying things,

By default, are the disks Thick or Thin provisioned? I check my Windows Server VM, and the vdisk size is 200GB, which is exactly what I set the limit as. This would indicate a thick provision from my understanding, no?

[JonathanM]

3 minutes ago, Xyler94 said:

I check my Windows Server VM, and the vdisk size is 200GB, which is exactly what I set the limit as. This would indicate a thick provision from my understanding, no?

Depends on how you check the size. At the command line use "ls -l -s" That's LS -L -S lowercase. That will show you both actual space used and the allocation limit.

[Xyler94]

6 minutes ago, JonathanM said:

Depends on how you check the size. At the command line use "ls -l -s" That's LS -L -S lowercase. That will show you both actual space used and the allocation limit.

It's the file size that Windows File Explorer shows when I look for the vdisk from the share itself.

[JonathanM]

Just now, Xyler94 said:

It's the file size that Windows File Explorer shows when I look for the vdisk from the share itself.

That will only show the reserved space, not what's actually occupied.

[Xyler94]

3 hours ago, JorgeB said:

    Free (statfs, df):          41.46GiB

 

This is what btrfs is reporting, so the GUI is correct, image type files can grow with time on btrfs, especially vdisks if not using discard, if you don't have snapshots you can try to run defrag, alternatively, move the files to the array and see what was using the space, if there were ballooned files, moving them back should then result in less used space.

This did the trick. Dropped the usage down to 334GB. 

 

Gonna do more checks to make sure all the VMs come online properly, but wow... I wonder why the files balloon like that, it's weird. 

[JorgeB]

14 minutes ago, Xyler94 said:

I wonder why the files balloon like that, it's weird

 

See the link above, make sure the VMs have discard enabled.

[Xyler94]

27 minutes ago, JorgeB said:

 

See the link above, make sure the VMs have discard enabled.

I followed the link, and did the steps for my Windows Server VM. It worked, Windows Server sees the drive as "Thin Provisioned" now

I did not notice any mention of Linux. Is SCSI controller baked into Linux? and I just need to add the " discard='unmap' " code for my Linux VMs?

[JorgeB]

13 hours ago, Xyler94 said:

Is SCSI controller baked into Linux?

Should be, but since some time you can also use the VirtIO option.

 

13 hours ago, Xyler94 said:

I just need to add the " discard='unmap' " code for my Linux VMs?

Yep

[Xyler94]

6 hours ago, JorgeB said:

Should be, but since some time you can also use the VirtIO option.

 

Yep

Thank you very much for your help Jorge,

With all your advice, the drive is now almost 200GB lighter. All VMs working just fine.

Edited August 21 by Xyler94