January 11, 20251 yr Hey peeps. I've tried for months now to increase my ZFS cache memory usage but it's stuck at 32gb. I have 128gb in my server and want to take the cache up to atleast 64gb. I originally created the zfs.conf file manually, tweaked and messed with the file and syntax but no change. I've recently used the gui option in system drivers and no change. I've deleted the file and recreated using just the gui as one post suggested, checked for spaces etc but still no luck. Flash drive: config/modprobe.d/zfs.conf options zfs zfs_arc_max=64000000000 GUI: Tools/system drivers zfs (2.2.7-1) ZFS Custom /lib/modules/6.6.68-Unraid/extra options zfs zfs_arc_max=64000000000 Any thoughts would be appreciated
January 11, 20251 yr ... Orgianly thouhgt that was set at pool creation. this may be a OS system control varable as well... It sounds like you're attempting to adjust the ZFS ARC (Adaptive Replacement Cache) size on your Unraid server. This should work dynamically and does not require setting it at the time of pool creation... Let's troubleshoot and ensure everything is configured properly. Check current: cat /proc/spl/kstat/zfs/arcstats | grep -E 'c_max|c_min' This will show the current max (c_max) and min (c_min) ARC size in bytes. *warnnign do at your own risk -- Untested: Adjust ARC Cache Dynamically You can dynamically adjust the ARC cache size with the following command: echo <desired_size_in_bytes> > /sys/module/zfs/parameters/zfs_arc_max For example, to set it to 64GB: echo 64000000000 > /sys/module/zfs/parameters/zfs_arc_max verify: cat /sys/module/zfs/parameters/zfs_arc_max Ensure the value matches your input. Then on to the next/hard step. you need to edit the zfs config and potential run this in the go file / user script at boot to confirm and make chagnes permenat... Maybe we can leverlage unraid system drivers and use a modprobe config... To Persist the Setting To make the setting persistent across reboots: Confirm your zfs.conf file is correct: cat /boot/config/modprobe.d/zfs.conf Oh look, there it is: options zfs zfs_arc_max=64000000000
January 11, 20251 yr if using system drivers make sure it is saved and you hit rebuild. A reboot is required to apply modprobe config edits...
January 11, 20251 yr Author I've followed the above instructions: root@Server:~# cat /proc/spl/kstat/zfs/arcstats | grep -E 'c_max|c_min' c_min 4 4223258496 c_max 4 34359738368 root@Server:~# echo 64000000000 > /sys/module/zfs/parameters/zfs_arc_max root@Server:~# cat /sys/module/zfs/parameters/zfs_arc_max 64000000000 edited, saved and rebuilt system drivers and then rebooted the server. root@Server:~# cat /proc/spl/kstat/zfs/arcstats | grep -E 'c_max|c_min' c_min 4 4223258496 c_max 4 34359738368 still no change
January 11, 20251 yr 50 minutes ago, TheThingIs said: root@Server:~# echo 64000000000 > /sys/module/zfs/parameters/zfs_arc_max You have to save this to the flash drive, or changes are lost, create a file called /config/modprobe.d/zfs.conf and add to it the value you want, e.g., for 16GB options zfs zfs_arc_max=16765034496 Then reboot
January 11, 20251 yr Author I've done that (I think). It also gets saved in the same place when editing system driver.
January 11, 20251 yr Everything looks normal to me, not sure what it could be, try booting with a different flash drive, using a stock install, no key needed, just create the modprobe folder and copy the attached file there, rename it to zfs.conf and get the ARC stats after boot, just in case there's something else interfering. zfs - 64G.conf
January 11, 20251 yr Thanks JorgeB. To Help clarify. We want to make sure the modprobe config is being made on the USB and that the options are loaded. root@BMM-Unraid:~# cd /boot/config/modprobe.d/ root@BMM-Unraid:/boot/config/modprobe.d# ls nvidia.conf root@BMM-Unraid:/boot/config/modprobe.d# You should have a zfs.conf here with the options zfs info. this is what's loaded at boot. to confirm, here is a script to check if it exist and if not to set a 64GB Arc cache via modprobe #!/bin/bash # Define variables MODPROBE_DIR="/boot/config/modprobe.d" ZFS_CONF="$MODPROBE_DIR/zfs.conf" ZFS_OPTION="options zfs zfs_arc_max=64000000000" # Ensure the directory exists if [ ! -d "$MODPROBE_DIR" ]; then echo "Error: Directory $MODPROBE_DIR does not exist." exit 1 fi # Check if zfs.conf exists if [ -f "$ZFS_CONF" ]; then echo "$ZFS_CONF already exists." else echo "$ZFS_CONF does not exist. Creating it." echo "$ZFS_OPTION" > "$ZFS_CONF" echo "Added '$ZFS_OPTION' to $ZFS_CONF." fi Edited January 11, 20251 yr by bmartino1 typo
January 11, 20251 yr Also, remember that arc memory is to be dynamic: cat /proc/spl/kstat/zfs/arcstats | grep -E 'c_max|c_min' it should fit within the min max range... From UNRAID DOC: ZFS dynamically manages the ARC size based on system memory availability and workload demands. By default, Unraid may set the ARC size to a fraction of the total system memory to balance performance and resource utilization. This default setting is typically configured during system boot via the /etc/modprobe.d/zfs.conf file, which is auto-generated to limit the ZFS ARC to 1/8 of installed memory. From Unraid docs: https://docs.unraid.net/unraid-os/manual/zfs/placeholder/#:~:text=at this time.-,During system boot,-%2C the file /etc https://arstechnica.com/information-technology/2020/05/zfs-101-understanding-zfs-storage-and-performance/ I'm not aware of the memory limit being hardcoded into unriad for ZFS.
January 12, 20251 yr Author I understand it's dynamic but it won't ever go over 32gb leaving me with 77gb unused. Most of the time it's maxed at 32gb and confusingly that is 1/4 of my ram and not 1/8th. The file is there at /boot/config/modprobe.d/ I can't physically access the machine atm so testing a spare usb stick isn't possible. It's like the file is being ignored and (or superseded) by something else setting the cache to 32gb Edited January 12, 20251 yr by TheThingIs
January 12, 20251 yr 9 hours ago, TheThingIs said: It's like the file is being ignored and (or superseded) by something else setting the cache to 32gb Yep, that why it would be good to test with a stock install, but if that's not possible ATM I'm afraid that I don't have any other ideas.
January 12, 20251 yr Author Solution Last night I set the cache manually using root@Server:~# echo 64000000000 > /sys/module/zfs/parameters/zfs_arc_max and today my system is indeed using 64gb of ram! So I'll set a script to run at array start and hopefully should be good. Not an ideal solution but it works!! Big thank you to everyone involved for jumping in and helping, I really appreciate it. PS I've just had a thought that I originally installed ZFS before it was part of the OS. I vaguely remember setting a max for the arc cache via cli as I only had 64gb at the time. It could have been a SpaceInvader1 guide I followed. If that setting has persisted through to the integrated version of ZFS then how would I reverse that? PPS got it, it was in my go file!!! Edited January 12, 20251 yr by TheThingIs
January 12, 20251 yr I don't have a good way to explain or go over defaulting or resetting a zfs config. You could run this in the go file. echo 0 > /sys/module/zfs/parameters/zfs_arc_max I would need to use command zfs get all on a trial usb to see the defaults... from plugins to other scripts and other locations. Its hard to say. in theory by reloading via modprobe driver from the kernel and restoring that, but it should load from saved configs else where... unload and reload zfs... modprobe -r zfs modprobe zfs Locate Startup Scripts Check if any custom scripts are modifying ZFS settings during startup. Look in /boot/config/ or /boot/config/plugins/ for scripts that adjust ZFS parameters. Check /boot/config/go, the Unraid startup script, for ZFS-related commands. I'm not sure where unraid stores some of the zfs config settings... https://zfs-discuss.opensolaris.narkive.com/gBlqk5gT/unsetting-resetting-zfs-properties https://docs.oracle.com/cd/E19253-01/819-5461/gayns/index.html
February 23, 20251 yr On 1/11/2025 at 12:55 AM, bmartino1 said: ... Orgianly thouhgt that was set at pool creation. this may be a OS system control varable as well... It sounds like you're attempting to adjust the ZFS ARC (Adaptive Replacement Cache) size on your Unraid server. This should work dynamically and does not require setting it at the time of pool creation... Let's troubleshoot and ensure everything is configured properly. Check current: cat /proc/spl/kstat/zfs/arcstats | grep -E 'c_max|c_min' This will show the current max (c_max) and min (c_min) ARC size in bytes. *warnnign do at your own risk -- Untested: Adjust ARC Cache Dynamically You can dynamically adjust the ARC cache size with the following command: echo <desired_size_in_bytes> > /sys/module/zfs/parameters/zfs_arc_max For example, to set it to 64GB: echo 64000000000 > /sys/module/zfs/parameters/zfs_arc_max verify: cat /sys/module/zfs/parameters/zfs_arc_max Ensure the value matches your input. Then on to the next/hard step. you need to edit the zfs config and potential run this in the go file / user script at boot to confirm and make chagnes permenat... Maybe we can leverlage unraid system drivers and use a modprobe config... To Persist the Setting To make the setting persistent across reboots: Confirm your zfs.conf file is correct: cat /boot/config/modprobe.d/zfs.conf Oh look, there it is: options zfs zfs_arc_max=64000000000 Thank you, this worked for me on my UnRaid 7.0
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.