Swap File Plugin for unRAID v5 and v6

[trurl]

2 hours ago, Spies said:

Just changed it to /mnt/disk4/cache and now it shows:

This means you have accidentally created a user share named "cache". Any top level folder from cache or array disks is automatically a user share named for the folder with default settings.

 

If you have the Fix Common Problems plugin installed it will give you a warning about this.

 

This is not generally a good idea because you will see it over the network and think it is referring to the cache disk when really it is a user share that happens to be named "cache".

[Tilton53]

Don't use this plugin with 6.4.0-RC2 or higher it causes emhttpd to error out and crash.

[Squid]

10 hours ago, Tilton53 said:

Don't use this plugin with 6.4.0-RC2 or higher it causes emhttpd to error out and crash.

I just installed it to test this, and I don't know what you're seeing.  Can you elaborate?

[Kukielka]

Well it seems like the plugin doesn't work with 6.4.0-rc8q anymore?

The UI is responsive, the swap file is created, but I can't manage to get it used...

[Shadowrunner]

Running this on 6.5 with no issues.

[DZMM]

Anybody running this on 6.5.3?  My memory usage is creeping up and upgrading isn't a financially viable option given the insane RAM prices, so interested in giving this a try.

[DZMM]

Ok, I've been using this for a few days - can I use a few questions please.

 

I've created a 32GB swapfile as I've added an extra unassigned SSD that has spare space.  It's been live for around 1.5 days and it's using 8GB so far, but I've got free memory.  When does it move data to the swap?  I thought it only did it when the physical ram was getting low?  What controls it?  If the swap data is accessed, is it moved back to ram?

 

Thanks

[maciekish]

On 6/24/2018 at 9:20 AM, DZMM said:

Ok, I've been using this for a few days - can I use a few questions please.

 

I've created a 32GB swapfile as I've added an extra unassigned SSD that has spare space.  It's been live for around 1.5 days and it's using 8GB so far, but I've got free memory.  When does it move data to the swap?  I thought it only did it when the physical ram was getting low?  What controls it?  If the swap data is accessed, is it moved back to ram?

 

Thanks

 

It is controlled by swappiness. I guess you could create a userscript on startup that sets the swappiness. https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

 

I suggest the plugin author adds an option for this in the plugin. 

sysctl vm.swappiness=[value]

Edited July 5, 2018 by maciekish

[DZMM]

42 minutes ago, maciekish said:

 

It is controlled by swappiness. I guess you could create a userscript on startup that sets the swappiness. https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

 

I suggest the plugin author adds an option for this in the plugin. 

sysctl vm.swappiness=[value]

Thanks I've set mine to 10 so it kicks in when 90% of memory used - I've got 64GB so this means it's there for insurance.  Default is 60, which is too 'aggressive' for my use case.

[DZMM]

Out of interest, does all memory get moved to SWAP e.g. even memory in use by VMs? 

 

Is there a way to not swap VM memory, or only certain dockers where performance doesn't matter?

[principis]

When trying to create a 8 GB swap file it stuck on 6.4 GB so I manually made one with fallocate and mkswap. I don't really understand why the plugin doesn't use that method...

On 11/20/2018 at 7:44 PM, DZMM said:

Out of interest, does all memory get moved to SWAP e.g. even memory in use by VMs? 

 

Is there a way to not swap VM memory, or only certain dockers where performance doesn't matter?

You shouldn't really notice much of a performance hit. And afaik it's not possible on linux and I think it's really difficult to implement exclusions.

Memory is assigned in pages. Most of the time blocks of 4 KB (or 2 MB or 1GB). The linux kernel handles swapping and it tries to swap out pages which are least recently used.  For example when a program starts it may load a lot of code in memory just for initialization but after that it won't be used so this code which is allocated in pages can be swapped out to free up memory. So yes, all memory is swapped out. (If I'm correct, it may be that Linux handles it differently, I'm only a CS student not an expert ;) ).

[cammelspit]

Can anyone tell me if this still can't work on a BTRFS formatted cache drive? I can create the swap file but it won't start to use it and since these posts are old enough, I have no idea if this is still a problem or not.

[primeval_god]

10 hours ago, cammelspit said:

Can anyone tell me if this still can't work on a BTRFS formatted cache drive?

AFAIK it is still not possible to have SWAP on a BTRFS filesystem. I think i read somewhere that it may become possible in Linux Kernel 4.21, but I dont think that is available yet in the wild, much less in unRAID.

[cammelspit]

On 2/11/2019 at 8:57 AM, primeval_god said:

AFAIK it is still not possible to have SWAP on a BTRFS filesystem. I think i read somewhere that it may become possible in Linux Kernel 4.21, but I dont think that is available yet in the wild, much less in unRAID.

Thank you kindly for the clarification. I actually have a spare 120GB SSD laying around, just have used up all my SATA ports or I would go and make it SWAP and temp file storage. With how picky this board is on RAM, I actually have a 32GB kit installed but it only sees 16GB of it, and the cost of a properly compatible 32GB kit it may be time to toss an HBA in there sooner than later. 😀

 

Thanks again!

[primeval_god]

Hi @theone,

I am having some trouble getting SWAP to work on my setup running unRAID 6.6.7. I have the plugin set to make the swap file on a drive mounted with the Unassigned Drives plugin. The filesystem is ext4, and on hitting start the swap file is created. But the plugin never shows a check for Swap File in use and there is no sign in Netdata that the system is using swap. Checking the system log the only output produced by the plugin is as follows:

 

Mar 11 20:59:24 Nexus emhttpd: req (7): cmd=%2Fplugins%2Fswapfile%2Fscripts%2Frc.swapfile&arg1=start&runCmd=Start&csrf_token=****************
Mar 11 20:59:24 Nexus emhttpd: cmd: /usr/local/emhttp/plugins/swapfile/scripts/rc.swapfile start
Mar 11 20:59:24 Nexus rc.swapfile[19406]: Swap file /mnt/disks/Kingston_DataTraveler_20/swapfile re-used and started
Mar 11 20:59:41 Nexus emhttpd: req (8): cmd=%2Fplugins%2Fswapfile%2Fscripts%2Frc.swapfile&arg1=start&runCmd=Start&csrf_token=****************
Mar 11 20:59:41 Nexus emhttpd: cmd: /usr/local/emhttp/plugins/swapfile/scripts/rc.swapfile start
Mar 11 20:59:41 Nexus rc.swapfile[19941]: Swap file /mnt/disks/Kingston_DataTraveler_20/swapfile re-used and started

 

Any help would be appreciated 

 

Edit:

I managed to figure this out. Turns out due to the speed of the drive I was using and the size of the SWAP file I was trying to create, the start script was taking long enough that the web request was timing out. Running the script from the console allowed it to finish correctly and everything is working now.

Edited March 15, 2019 by primeval_god
Solved my own problem

[norbertt]

Hi,

 

Can I use my swap folder with cache prefer settings? 

[trurl]

6 hours ago, norbertt said:

Can I use my swap folder with cache prefer settings? 

I haven't used this plugin, but I assume you would be better off using a setting that is ignored by mover, such as cache-only if it is already on cache. If it isn't already on cache you should probably just set it up again so that it is.

[WarHawk8080]

Does unRaid support zram and zswap?

I have much dealings with some SBC's and armbian...even with minimal RAM with the zram and zswap modules it can easily compress swap space in a compressed zram space with minimal dwell/lag in compressing/uncompressing swap, even though it uses physical ram the compressed space can easily double or triple the RAM that being used.

I have a Orange Pi R1 with 512MB ram, 256MB is used for zswap and 200% of that is used as compressed swap adding 256MB over what is actually on the board

and I also kick my swapiness up to 100 and it works like a champ

 

But with the low prices of DDR4, I might as well go ahead and upgrade to 16GB ram from 8GB

 

Oh and I also have the btrfs issue with creating a swapfile...fooey!

 

Yeah..I know it's ubuntu...but slackware should be able to do this MUCH better than debian/ubuntu based systems
https://askubuntu.com/questions/471912/zram-vs-zswap-vs-zcache-ultimate-guide-when-to-use-which-one#472227
Better one about slackware
https://www.linuxquestions.org/questions/slackware-14/requests-for-current-14-2-15-0-a-4175620463/page141.html

Linux should ALWAYS have a swap capability...
https://haydenjames.io/linux-performance-almost-always-add-swap-space/

Even if it is in a "virtual" compressed RAM swap space

Edited June 16, 2020 by WarHawk8080

[b0m541]

 

I just realize that my unraid machine does not have swap enabled.

I did not find much information here in the forum.

 

Does unraid by default use a swap file or do I need to enable that manually somewhere?

Of course, I can do it using this plugin, I am just wondering if there is a standard mechanism in the unRAID UI for swap and I just didn't find it..

 

I also found that swap files are not supported on BTRFS on Linux earlier than version 5, which is a pity, because all my drives use BTRFS, also the SSD Cache drives.

 

I never had memory problems, but with recently more docker containers, some of which use a a few hundred MB RAM, I seem to reach a limit, CPU and Storage get very busy when memory fills up. I first thought that this is caused by swap thrashing, and then saw that the machine does not use swap at all....

 

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

Am I overthinking this? Will unraid select the best location and create a swap file when it finds a non-BTRFS formatted drive?

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

[itimpi]

UnRAID is designed to run from RAM and not use a swap file at all.     It expects you to have enough RAM to run the workload you give it.

[b0m541]

OK, got it. Lets say I wanted to have a swap file of 128GB, just in case...

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

 

[primeval_god]

14 minutes ago, b0m541 said:

OK, got it. Lets say I wanted to have a swap file of 128GB, just in case...

What is the recommended route to go?

 

Put swap on HDD oder SSD? For that I would need to reformat a drive or the cache pool from BTRFS to XFS.

 

If the best way would be to take the cache SSD pool from BTRFS to XFS, what is the recommended strategy to get from a) to b)?

Note: I have empty drives lingering in the array, would be easy to reformat one from BTRFS to XFS, but I guess putting swap on the cache pool is better?

 

I am not super knowledgeable about swap, but I think the best place to put a swap file would be on a drive mounted with the Unassigned drives plugin. The cache pool would probably be the next best place. I am certain that you wouldnt want it in the unRAID array. As to SSD vs HDD, I would guess an SSD would be the way to go but it probably depends on your usage pattern. BTW 128GB seems like a lot for swap space.

[rampage]

BTRFS can't hold your swap file, out of luck

[mgutt]

Does this plugin automatically stop swap on array stopping? I accidentally found out, that my script prevented unmounting /mnt/cache. 

[canglan]

On version 6.9.1, the Linux kernel updated to 5.10 and it supports swapfile on BTRFS.

On filesystem BTRFS, you should follow this [url] to make a swapfile, and then it will work. 

Hope this will be fixed.

 

Edited March 28, 2021 by canglan