Jump to content
theone

Swap File Plugin for unRAID v5 and v6

97 posts in this topic Last Reply

Recommended Posts

2 hours ago, Spies said:

Just changed it to /mnt/disk4/cache and now it shows:

This means you have accidentally created a user share named "cache". Any top level folder from cache or array disks is automatically a user share named for the folder with default settings.

 

If you have the Fix Common Problems plugin installed it will give you a warning about this.

 

This is not generally a good idea because you will see it over the network and think it is referring to the cache disk when really it is a user share that happens to be named "cache".

Share this post


Link to post
10 hours ago, Tilton53 said:

Don't use this plugin with 6.4.0-RC2 or higher it causes emhttpd to error out and crash.

I just installed it to test this, and I don't know what you're seeing.  Can you elaborate?

Share this post


Link to post

Well it seems like the plugin doesn't work with 6.4.0-rc8q anymore?

The UI is responsive, the swap file is created, but I can't manage to get it used... :/

Share this post


Link to post

Anybody running this on 6.5.3?  My memory usage is creeping up and upgrading isn't a financially viable option given the insane RAM prices, so interested in giving this a try.

Share this post


Link to post

Ok, I've been using this for a few days - can I use a few questions please.

 

I've created a 32GB swapfile as I've added an extra unassigned SSD that has spare space.  It's been live for around 1.5 days and it's using 8GB so far, but I've got free memory.  When does it move data to the swap?  I thought it only did it when the physical ram was getting low?  What controls it?  If the swap data is accessed, is it moved back to ram?

 

Thanks

Share this post


Link to post
On 6/24/2018 at 9:20 AM, DZMM said:

Ok, I've been using this for a few days - can I use a few questions please.

 

I've created a 32GB swapfile as I've added an extra unassigned SSD that has spare space.  It's been live for around 1.5 days and it's using 8GB so far, but I've got free memory.  When does it move data to the swap?  I thought it only did it when the physical ram was getting low?  What controls it?  If the swap data is accessed, is it moved back to ram?

 

Thanks

 

It is controlled by swappiness. I guess you could create a userscript on startup that sets the swappiness. https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

 

I suggest the plugin author adds an option for this in the plugin. 

sysctl vm.swappiness=[value]
Edited by maciekish

Share this post


Link to post
42 minutes ago, maciekish said:

 

It is controlled by swappiness. I guess you could create a userscript on startup that sets the swappiness. https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

 

I suggest the plugin author adds an option for this in the plugin. 


sysctl vm.swappiness=[value]

Thanks I've set mine to 10 so it kicks in when 90% of memory used - I've got 64GB so this means it's there for insurance.  Default is 60, which is too 'aggressive' for my use case.

Share this post


Link to post

Out of interest, does all memory get moved to SWAP e.g. even memory in use by VMs? 

 

Is there a way to not swap VM memory, or only certain dockers where performance doesn't matter?

Share this post


Link to post

When trying to create a 8 GB swap file it stuck on 6.4 GB so I manually made one with fallocate and mkswap. I don't really understand why the plugin doesn't use that method...

On 11/20/2018 at 7:44 PM, DZMM said:

Out of interest, does all memory get moved to SWAP e.g. even memory in use by VMs? 

 

Is there a way to not swap VM memory, or only certain dockers where performance doesn't matter?

You shouldn't really notice much of a performance hit. And afaik it's not possible on linux and I think it's really difficult to implement exclusions.

Memory is assigned in pages. Most of the time blocks of 4 KB (or 2 MB or 1GB). The linux kernel handles swapping and it tries to swap out pages which are least recently used.  For example when a program starts it may load a lot of code in memory just for initialization but after that it won't be used so this code which is allocated in pages can be swapped out to free up memory. So yes, all memory is swapped out. (If I'm correct, it may be that Linux handles it differently, I'm only a CS student not an expert ;) ).

Share this post


Link to post

Can anyone tell me if this still can't work on a BTRFS formatted cache drive? I can create the swap file but it won't start to use it and since these posts are old enough, I have no idea if this is still a problem or not.

Share this post


Link to post
10 hours ago, cammelspit said:

Can anyone tell me if this still can't work on a BTRFS formatted cache drive?

AFAIK it is still not possible to have SWAP on a BTRFS filesystem. I think i read somewhere that it may become possible in Linux Kernel 4.21, but I dont think that is available yet in the wild, much less in unRAID.

Share this post


Link to post
On 2/11/2019 at 8:57 AM, primeval_god said:

AFAIK it is still not possible to have SWAP on a BTRFS filesystem. I think i read somewhere that it may become possible in Linux Kernel 4.21, but I dont think that is available yet in the wild, much less in unRAID.

Thank you kindly for the clarification. I actually have a spare 120GB SSD laying around, just have used up all my SATA ports or I would go and make it SWAP and temp file storage. With how picky this board is on RAM, I actually have a 32GB kit installed but it only sees 16GB of it, and the cost of a properly compatible 32GB kit it may be time to toss an HBA in there sooner than later. 😀

 

Thanks again!

Share this post


Link to post

Hi @theone,

I am having some trouble getting SWAP to work on my setup running unRAID 6.6.7. I have the plugin set to make the swap file on a drive mounted with the Unassigned Drives plugin. The filesystem is ext4, and on hitting start the swap file is created. But the plugin never shows a check for Swap File in use and there is no sign in Netdata that the system is using swap. Checking the system log the only output produced by the plugin is as follows:

 

Mar 11 20:59:24 Nexus emhttpd: req (7): cmd=%2Fplugins%2Fswapfile%2Fscripts%2Frc.swapfile&arg1=start&runCmd=Start&csrf_token=****************
Mar 11 20:59:24 Nexus emhttpd: cmd: /usr/local/emhttp/plugins/swapfile/scripts/rc.swapfile start
Mar 11 20:59:24 Nexus rc.swapfile[19406]: Swap file /mnt/disks/Kingston_DataTraveler_20/swapfile re-used and started
Mar 11 20:59:41 Nexus emhttpd: req (8): cmd=%2Fplugins%2Fswapfile%2Fscripts%2Frc.swapfile&arg1=start&runCmd=Start&csrf_token=****************
Mar 11 20:59:41 Nexus emhttpd: cmd: /usr/local/emhttp/plugins/swapfile/scripts/rc.swapfile start
Mar 11 20:59:41 Nexus rc.swapfile[19941]: Swap file /mnt/disks/Kingston_DataTraveler_20/swapfile re-used and started

 

Any help would be appreciated 

 

Edit:

I managed to figure this out. Turns out due to the speed of the drive I was using and the size of the SWAP file I was trying to create, the start script was taking long enough that the web request was timing out. Running the script from the console allowed it to finish correctly and everything is working now.

Edited by primeval_god
Solved my own problem

Share this post


Link to post
6 hours ago, norbertt said:

Can I use my swap folder with cache prefer settings? 

I haven't used this plugin, but I assume you would be better off using a setting that is ignored by mover, such as cache-only if it is already on cache. If it isn't already on cache you should probably just set it up again so that it is.

Share this post


Link to post
Posted (edited)

Does unRaid support zram and zswap?

I have much dealings with some SBC's and armbian...even with minimal RAM with the zram and zswap modules it can easily compress swap space in a compressed zram space with minimal dwell/lag in compressing/uncompressing swap, even though it uses physical ram the compressed space can easily double or triple the RAM that being used.

I have a Orange Pi R1 with 512MB ram, 256MB is used for zswap and 200% of that is used as compressed swap adding 256MB over what is actually on the board

and I also kick my swapiness up to 100 and it works like a champ

 

But with the low prices of DDR4, I might as well go ahead and upgrade to 16GB ram from 8GB

 

Oh and I also have the btrfs issue with creating a swapfile...fooey!

 

Yeah..I know it's ubuntu...but slackware should be able to do this MUCH better than debian/ubuntu based systems
https://askubuntu.com/questions/471912/zram-vs-zswap-vs-zcache-ultimate-guide-when-to-use-which-one#472227
Better one about slackware
https://www.linuxquestions.org/questions/slackware-14/requests-for-current-14-2-15-0-a-4175620463/page141.html

Linux should ALWAYS have a swap capability...
https://haydenjames.io/linux-performance-almost-always-add-swap-space/

Even if it is in a "virtual" compressed RAM swap space

Edited by WarHawk8080

Share this post


Link to post

 

I just realize that my unraid machine does not have swap enabled.

I did not find much information here in the forum.

 

Does unraid by default use a swap file or do I need to enable that manually somewhere?

Of course, I can do it using this plugin, I am just wondering if there is a standard mechanism in the unRAID UI for swap and I just didn't find it..

 

I also found that swap files are not supported on BTRFS on Linux earlier than version 5, which is a pity, because all my drives use BTRFS, also the SSD Cache drives.

 

I never had memory problems, but with recently more docker containers, some of which use a a few hundred MB RAM, I seem to reach a limit, CPU and Storage get very busy when memory fills up. I first thought that this is caused by swap thrashing, and then saw that the machine does not use swap at all....

 

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

Am I overthinking this? Will unraid select the best location and create a swap file when it finds a non-BTRFS formatted drive?

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

Share this post


Link to post

UnRAID is designed to run from RAM and not use a swap file at all.     It expects you to have enough RAM to run the workload you give it.

Share this post


Link to post

OK, got it. Lets say I wanted to have a swap file of 128GB, just in case...

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

 

Share this post


Link to post
14 minutes ago, b0m541 said:

OK, got it. Lets say I wanted to have a swap file of 128GB, just in case...

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

 

I am not super knowledgeable about swap, but I think the best place to put a swap file would be on a drive mounted with the Unassigned drives plugin. The cache pool would probably be the next best place. I am certain that you wouldnt want it in the unRAID array. As to SSD vs HDD, I would guess an SSD would be the way to go but it probably depends on your usage pattern. BTW 128GB seems like a lot for swap space.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.