Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

ZFS cache pool : incorrect size with df -h

Featured Replies

Hello,

 

On my unraid server, i have this cache pool :

707912581_Capturedecran2024-02-11a18_33_52.thumb.png.b9a5c2e2e31344de6b313beb9195621d.png

 

It is 3x 1TB drives, with one mirror, for a total of 1.9 Tb useable capacity.

On the unraid console, when i do :

df -h

 

I have this output :

root@Yggdrasil:~# df -h
Filesystem                     Size  Used Avail Use% Mounted on
rootfs                          24G  1.6G   22G   7% /
tmpfs                           32M  2.2M   30M   7% /run
/dev/sda1                       58G  2.5G   55G   5% /boot
overlay                         24G  1.6G   22G   7% /lib
overlay                         24G  1.6G   22G   7% /usr
devtmpfs                       8.0M     0  8.0M   0% /dev
tmpfs                           24G     0   24G   0% /dev/shm
tmpfs                          128M  1.5M  127M   2% /var/log
tmpfs                          1.0M     0  1.0M   0% /mnt/disks
tmpfs                          1.0M     0  1.0M   0% /mnt/remotes
tmpfs                          1.0M     0  1.0M   0% /mnt/addons
tmpfs                          1.0M     0  1.0M   0% /mnt/rootshare
(array disks here)
cache                          939G   23G  916G   3% /mnt/cache
cache/system                   933G   17G  916G   2% /mnt/cache/system
cache/domains                  1.4T  417G  916G  32% /mnt/cache/domains
cache/appdata                  916G  2.3M  916G   1% /mnt/cache/appdata
cache/appdata/mosquitto        916G  256K  916G   1% /mnt/cache/appdata/mosquitto
cache/appdata/grafana          916G  1.3M  916G   1% /mnt/cache/appdata/grafana
cache/appdata/homeassistant    916G   93M  916G   1% /mnt/cache/appdata/homeassistant
cache/appdata/mysql            916G  119M  916G   1% /mnt/cache/appdata/mysql
cache/appdata/nginx            916G  1.0M  916G   1% /mnt/cache/appdata/nginx
(etc for others containers)

 

The "cache" is reported to be 939GB with only 23Gb used.

 

I dont really understand this value.

 

Is it because of ZFS ?

 

I use Glances (with Homepage), and i can't show the correct size of the cache pool :

361286456_Capturedecran2024-02-11a18_41_08.png.cfb1fef91265a38f4e7dde67c623d046.png

1408033121_Capturedecran2024-02-11a18_43_22.png.45699e424b122301c4f4091dc3f736b9.png

 

It seems to be reported as the df command line.

 

Really not a big deal, but maybe i can use this to learn something more.

 

Thanks all,

 

K.

Solved by JorgeB

  • Community Expert

Because it's by dataset, look st the datasets and add them all, just domains dataset is using 417G, and note that those use GiB, GUI uses GB.

 

 

 

 

  • Author

Hello JorgeB,

 

Indeed, it is by dataset.

But why the main one (cache) did not report the full size of the pool array ?

It seems to only report one drive of the three.

 

That is more that i did not understand.

 

Maybe i could have done better configuring it ?

 

Thanks,

 

K.

  • Community Expert
  • Solution
Just now, Keichi said:

But why the main one (cache) did not report the full size of the pool array ?

Because as far as df can see there's no data there, data is inside the datasets, and datasets are separate zfs filesystems, for example if you try to move some data from one dataset to another a full copy will be done, instead of a move.

  • Author

Ok, i understand.

 

So it is impossible to report the full size of a ZFS cache pool, with the tool i use.

 

Thanks for the help, and as always, you clear answer.

 

K.

 

 

  • Community Expert
58 minutes ago, Keichi said:

So it is impossible to report the full size of a ZFS cache pool, with the tool i use.

BTW, if you need to get the used or free space from a zfs pool for a script or something, instead of df you can use:

 

zfs get -Hp -o value available,used <pool name>

 

  • Author

Thanks!

But, this is for Homepage (https://github.com/gethomepage/homepage) which uses Glances integration (https://github.com/nicolargo/glances).

 

I will ask directly on Glances' Github for more information.

 

I just wanted to understand better the issue and from you gave me, something is possible :

root@Yggdrasil:~# zfs get -Hp -o value available,used /mnt/cache
966035955456
963024068864

Values seem correct to me.

 

Thanks again JorgeB!

 

K.

  • 1 year later...

Have come across this as well, where a pool in raidz1 of 6 22tb drives is reporting in the web interface Main, a total size of around 80-90 TB (it varies), while zfs correctly reports 6x20T drives as 120T (with 20T used for redundancy, of course).

As zfs becomes more integrated, definitely something I hope to see getting more accurate in the future.

My other 2 zfs pools seem to be correct though, at least.

  • Community Expert

Please post the diagnostics and the output from zfs get -Hp -o value available,used <pool name>

zfs get -Hp -o value available,used fast

60496117917184

27283594134544

For comparison:

zpool list fast

NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT

fast 120T 37.2T 82.8T - - 1% 31% 1.00x ONLINE -

tower-diagnostics-20251008-1304.zip

It’s interesting, it’s seemingly adding up the used/free numbers to make a total, so the “size” value keeps changing as the pool fills. 😂🤔

IMG_0802.jpeg

  • Community Expert
1 hour ago, eadnams said:

so the “size” value keeps changing as the pool fills

That will happen if you have block cloned data (reflinks) and/or dedup is enabled.

Interesting. That may explain it.

Is there a way to remove reflinks? I did do a cp operation briefly a while back, which default to reflinks being used. The original files are long gone now.

zpool get all fast | grep -i clon
fast  bcloneused                     7.26G                          -
fast  bclonesaved                    7.26G                          -
fast  bcloneratio                    2.00x                          -
fast  feature@block_cloning          active                         local

Edited by eadnams
Added the reflink information

  • Community Expert

If the original files were gone, i.e., there's only one copy of each file, they should not show up as extra capacity, so likely there are still multiple copies of some files.

Not that I can find. I was copying from a root folder outside a dataset, into a dataset. The root is empty now, just has the datasets in it.

Mostly just like size reporting in the dashboard to be correct.

Edited by eadnams
left out part of the explanation.

  • Community Expert

The dashboard should show exactly what that zfs command shows; the stats come from there.

Right, makes sense. It technically is, I suppose.

Any idea why it is inaccurate though?

I found and removed the reflinked data, have never had dedup enabled, but it still isn't reporting the incorrect amount for "size", and the "size" reported on the dashboard keeps changing on writes

zpool get all fast | grep -i clon
fast  bcloneused                     0                              -
fast  bclonesaved                    0                              -
fast  bcloneratio                    1.00x                          -
fast  feature@block_cloning          enabled                        local

zfs get -Hp -o value available,used fast
61063925599472
26707992507152

NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
fast         120T  36.5T  83.6T        -         -     1%    30%  1.00x    ONLINE  -
  raidz1-0   120T  36.5T  83.6T        -         -     1%  30.4%      -    ONLINE
    sdh1    20.0T      -      -        -         -      -      -      -    ONLINE
    sdn1    20.0T      -      -        -         -      -      -      -    ONLINE
    sdg1    20.0T      -      -        -         -      -      -      -    ONLINE
    sdk1    20.0T      -      -        -         -      -      -      -    ONLINE
    sdl1    20.0T      -      -        -         -      -      -      -    ONLINE
    sdi1    20.0T      -      -        -         -      -      -      -    ONLINE

Edited by eadnams

Looking through this: https://github.com/openzfs/zfs/discussions/15232 It seems this is a known OpenZFS bug that occurs with expansion. The recommendation is to use "zpool" instead of "zfs" for correct sizes/used/available stats. "zfs" just... gives the wrong numbers.

# zpool get free,allocated,size,capacity fast
NAME  PROPERTY   VALUE  SOURCE
fast  free       83.6T  -
fast  allocated  36.5T  -
fast  size       120T   -
fast  capacity   30%    -

That is all correct.

This is not:

# zfs get -o value available,used fast
VALUE
55.5T
24.3T
root@Tower:~# 

Should I do up a bug report? Its not Unraids fault, but Unraid could use the accurate information instead, to mitigate the zfs bug.

Edited by eadnams
Term clarification.

25 minutes ago, JorgeB said:

If the pool was raidz expanded the stats won't be correct; that's a known zfs bug, but the capacity should not change just by writing data, more info below:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/4/#findComment-1537164

Exactly. But it does seem to, as though its taking the "free" and "used" and just adding them together to get the total "size". But because the 'used' changes, and the 'free' doesn't always...

Though we can grab the more accurate/correct info (see my second comment), to bypass the zfs bug regardless.

Edited by eadnams
More details.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.