I'm looking for a solution to partitioning my single NVME ZFS cache fairly among a few shares. Respecting some type of storage limit on how much of the 2TB NVME cache disk do I want to allow it be used for a given share (but I do not want to block new file writes to the share; just skip the cache before the cache partition becomes technically "full" of data).

 

Not sure if I am making sense but basically I want to set something like this:
SHARE1 - store only on nvme_zfs_cache disk (no array). Guaranteed 100GB of disk space.

SHARE2 - new files on nvme, then array. I do not want this share to ever use more than 500GB out of my 2TB total nvme cache disk.

In short, once SHARE2 hits 500GB used out of the cache - I would expect FUSE to continue handling writes but on the array disks (slower spinning disks).

Can I achieve something like this with ZFS properties? It looks like when I have a zfs cache disk that datasets are created for those shares with "cache enabled" so I could set the ZFS quota property bit and force a dataset to have a maximum filesystem size as I want... how would unraid handle this? I hope that it would just go to the array once full (until the mover does its thing later on schedule)

 

root@Tower:~# zfs get all cache/domains
NAME           PROPERTY              VALUE                  SOURCE
cache/domains  type                  filesystem             -
cache/domains  creation              Fri Jul 28 22:31 2023  -
cache/domains  used                  96K                    -
cache/domains  available             289G                   -
cache/domains  referenced            96K                    -
cache/domains  compressratio         1.00x                  -
cache/domains  mounted               yes                    -
cache/domains  quota                 none                   default
cache/domains  reservation           none                   default
cache/domains  recordsize            128K                   default
cache/domains  mountpoint            /mnt/cache/domains     inherited from cache
cache/domains  sharenfs              off                    default
cache/domains  checksum              on                     default
cache/domains  compression           off                    inherited from cache
cache/domains  atime                 off                    inherited from cache
cache/domains  devices               on                     default
cache/domains  exec                  on                     default
cache/domains  setuid                on                     default
cache/domains  readonly              off                    default
cache/domains  zoned                 off                    default
cache/domains  snapdir               hidden                 default
cache/domains  aclmode               discard                default
cache/domains  aclinherit            restricted             default
cache/domains  createtxg             122                    -
cache/domains  canmount              on                     default
cache/domains  xattr                 sa                     inherited from cache
cache/domains  copies                1                      default
cache/domains  version               5                      -
cache/domains  utf8only              on                     -
cache/domains  normalization         formD                  -
cache/domains  casesensitivity       sensitive              -
cache/domains  vscan                 off                    default
cache/domains  nbmand                off                    default
cache/domains  sharesmb              off                    default
cache/domains  refquota              none                   default
cache/domains  refreservation        none                   default
cache/domains  guid                  832707413763775451     -
cache/domains  primarycache          all                    default
cache/domains  secondarycache        all                    default
cache/domains  usedbysnapshots       0B                     -
cache/domains  usedbydataset         96K                    -
cache/domains  usedbychildren        0B                     -
cache/domains  usedbyrefreservation  0B                     -
cache/domains  logbias               latency                default
cache/domains  objsetid              26                     -
cache/domains  dedup                 off                    default
cache/domains  mlslabel              none                   default
cache/domains  sync                  standard               default
cache/domains  dnodesize             auto                   inherited from cache
cache/domains  refcompressratio      1.00x                  -
cache/domains  written               96K                    -
cache/domains  logicalused           42K                    -
cache/domains  logicalreferenced     42K                    -
cache/domains  volmode               default                default
cache/domains  filesystem_limit      none                   default
cache/domains  snapshot_limit        none                   default
cache/domains  filesystem_count      none                   default
cache/domains  snapshot_count        none                   default
cache/domains  snapdev               hidden                 default
cache/domains  acltype               posix                  inherited from cache
cache/domains  context               none                   default
cache/domains  fscontext             none                   default
cache/domains  defcontext            none                   default
cache/domains  rootcontext           none                   default
cache/domains  relatime              off                    default
cache/domains  redundant_metadata    all                    default
cache/domains  overlay               on                     default
cache/domains  encryption            off                    default
cache/domains  keylocation           none                   default
cache/domains  keyformat             none                   default
cache/domains  pbkdf2iters           0                      default
cache/domains  special_small_blocks  0                      default
root@Tower:~# zfs list
NAME            USED  AVAIL     REFER  MOUNTPOINT
alwaysnvme      175M  30.3G      173M  /mnt/alwaysnvme
cache          10.8M   289G      104K  /mnt/cache
cache/domains    96K   289G       96K  /mnt/cache/domains
cache/system   7.87M   289G     7.87M  /mnt/cache/system
disk1          3.35M  16.2T      104K  /mnt/disk1
disk1/appdata  1.85M  16.2T     1.85M  /mnt/disk1/appdata
disk1/isos       96K  16.2T       96K  /mnt/disk1/isos
disk2          1.28M  12.6T       96K  /mnt/disk2
disk3          1.23M  12.6T       96K  /mnt/disk3

 

Quotas will not work with minimum free space, since Unraid will still see the total pool free space, I was thinking that reservations might but it would also be a problem with minimum free space, since with reservation the pool will always show that value as used, so also not possible to set the minimum free space correctly so that it would start writing to the array, for pools that don't need to overflow to the array it would work.

On 8/2/2023 at 7:14 AM, JorgeB said:

Quotas will not work with minimum free space, since Unraid will still see the total pool free space, I was thinking that reservations might but it would also be a problem with minimum free space, since with reservation the pool will always show that value as used, so also not possible to set the minimum free space correctly so that it would start writing to the array, for pools that don't need to overflow to the array it would work.

 

ah, thank you. This feels like something that should be supported? How could I make this a feature request for a future version?

You can make a feature request but doesn't look to me like an easy thing to implement.