How to expand ZFS RAM allocation?

[Kirk4444]

Hi experienced Unraid users!

I have little experience with this, so I'm hoping someone from the community can help me.  I've just created a new server and have been brave enough to try ZFS!  LOL. 

I installed 64 gigs DDR5 RAM to support ZFS, but it seems to be barely using my RAM.  Per the Unraid notes, ZFS is only allocated 1/8 or your RAM.  My server really is only a NAS, so I want to give ZFS as much RAM as I can.  I thought I'd start by trying 1/2 of available RAM.  Here's what I did.

 

I went about trying to do this by:
- Shutting down server and putting USB Unraid boot drive into my PC. 
- I then went to the config folder
- went to modprode.d folder
- created a notepad doc with this text: options zfs zfs_arc_max=32000000000
- saved it as zfs.conf in the folder
- ejected USB from my PC and plugged back into my server and booted up.  Now have the below result. 



I'm not sure it helped.  I'm trying  to allocate 1/2 my RAM. 

 

Am I doing something wrong?

 

Thanks so much in advance!!!

[Kirk4444]

Sorry!!!  I figured out what I did wrong.  For others who may be inexperienced like me, chatgpt said don't use notepad like I did.  There was an extra empty space at the end of the string.  I downloaded notepad ++ and up'd my allocation to 48, and now check it out!  But now it's orange.  LOL, did I go for too much? 

[JorgeB]

14 hours ago, Kirk4444 said:

But now it's orange.

That's controlled by the default warning/critical disk utilization thresholds (Settings - Disk Settings)

 

14 hours ago, Kirk4444 said:

LOL, did I go for too much?

Depends on your needs, if you use VMs you can have issues starting one if it needs more RAM then currently free, since sometimes the ARC cannot release the RAM fast enough making the VM crash on first start, if you doesn't use VMs you should be able to use even more, e.g this is one of my servers, doesn't run any VMs so I use almost all free RAM for ARC:

 

 

[Kirk4444]

Thanks JorgeB!  Your comments were very helpful

[icarus69]

On 11/5/2023 at 6:32 PM, Kirk4444 said:

Hi experienced Unraid users!

I have little experience with this, so I'm hoping someone from the community can help me.  I've just created a new server and have been brave enough to try ZFS!  LOL. 

I installed 64 gigs DDR5 RAM to support ZFS, but it seems to be barely using my RAM.  Per the Unraid notes, ZFS is only allocated 1/8 or your RAM.  My server really is only a NAS, so I want to give ZFS as much RAM as I can.  I thought I'd start by trying 1/2 of available RAM.  Here's what I did.

 

I went about trying to do this by:
- Shutting down server and putting USB Unraid boot drive into my PC. 
- I then went to the config folder
- went to modprode.d folder
- created a notepad doc with this text: options zfs zfs_arc_max=32000000000
- saved it as zfs.conf in the folder
- ejected USB from my PC and plugged back into my server and booted up.  Now have the below result. 



I'm not sure it helped.  I'm trying  to allocate 1/2 my RAM. 

 

Am I doing something wrong?

 

Thanks so much in advance!!!

 

I'm also want to have my zfs system utilise more RAM. I'm curious about what the default setting for RAM allocation is before you added this conf file though. Do you have any idea of what it might be? And how has your experience been since making this change?

[JorgeB]

13 hours ago, icarus69 said:

I'm curious about what the default setting for RAM allocation is before you added this conf file though.

1/8 installed RAM

[icarus69]

4 hours ago, JorgeB said:

1/8 installed RAM

How does this relate to running VMs? Because I know that when setting up a VM you can allocate a specified amount of RAM to it. Does this bypass the RAM allocation rules we're talking about here? Also, I have 64GB RAM currently, and would like to start by doubling how much RAM is allocated, so from 1/8 to 2/8. What should I put in the zfs.conf file?

[JorgeB]

23 minutes ago, icarus69 said:

How does this relate to running VMs?

If you start the VMs first it's not a problem, ARC will only use whatever RAM is available, but if you have the ARC using more RAM than the one allocated to the VMs, and then start a VM, it may fail to start, since the ARC can be slow evicting the RAM.

[icarus69]

1 hour ago, JorgeB said:

If you start the VMs first it's not a problem, ARC will only use whatever RAM is available, but if you have the ARC using more RAM than the one allocated to the VMs, and then start a VM, it may fail to start, since the ARC can be slow evicting the RAM.

Ok gotcha. The OP talks only about allocating more RAM for ZFS. Would you ever recommend allocating more RAM for Docker? How much RAM does Unraid allocate for Docker by default? And would it be the same process i.e. create a docker.conf file and specify RAM allocation?

[JorgeB]

1 hour ago, icarus69 said:

Would you ever recommend allocating more RAM for Docker? How much RAM does Unraid allocate for Docker by default?

I believe there's no set allocation for docker in general, unless you limit the RAM for the containers, they will use whatever they need/want.

[Kulis]

Can I increase RAM allocation from unraid OS?

Do Is this the only solution I have to do this by changing the file to USB?

[JorgeB]

5 hours ago, Kulis said:

Can I increase RAM allocation from unraid OS?

Not sure what you mean, Unraid will use all the available RAM.

[Kulis]

I would like to allocate more RAM for ZFS from the UnraidOS GUID entry.

[JorgeB]

See here:

https://docs.unraid.net/unraid-os/release-notes/6.12.0/#zfs-pools