[Plugin] Swapfile for 6.9.1


Recommended Posts

It is an old TS-439 box, which is intel atom 1,8GHz with one RAM slot. Technically it should be able to deal with 4GB but it seems to be a bet with mixed chances.

There are some people who managed to upgrade to 2GB, I found one who said 4GB.
I've tried a couple of DDR3 SO-DIMMS and even got a new one with 4GB, nothing above 1GB is booting. The original module is a DDR3 1GB SO-DIMM 1333.

 

Link to comment
4 hours ago, Tjareson said:

Also I wasn't able to use a swapfile on btrfs, because swapon always came back with wrong attributes error or similar. 

The requirements for putting a swapfile on a btrfs filesystem are a bit tricky. The plugin does its best to automate it but here are a few gotchas, the steps you list below would not work on a BTRFS filesystem. On BTRFS the swapfile must be created in BTRFS subvolume with COW disabled.

 

4 hours ago, Tjareson said:

The only "issue" I would need to solve now, is to automatically issue a swapon command after the array starts and vice versa a swapoff before the array gets stopped. 

Any ideas where to implement that ideally? 

This is basically what the plugin does, swapon is does at the "started" event and swapoff at "stopping_svcs". Even with a manually created swapfile you should be able to point the plugin config at the file and have it manage it. Otherwise the user scripts plugin is a good way to run scripts on startup and shutdown. There are other ways which might get you closer to startup but i dont know enough about them to recommend them. Search the forum for adding commands to the go file.

 

Just speaking in general though swap is typically not considered as a replacement for having enough ram. This is especially true for unRAID where the OS loads into an runs completely from RAM.

Link to comment
5 hours ago, Tjareson said:

Personally I always feel bad to dispose otherwise well working hardware because of a software issue...

I hear you, and as long as you are willing to deal with the consequences, good for you.

 

However... you will never be able to keep old hardware running forever with the onslaught of security breaches and other pressures forcing ever increasing hardware requirements. If you choose to use Unraid for old hardware, you will hit more and more insurmountable barriers to staying secure. If you are ok with isolating the machine with external firewall measures, and accepting the risks of running outdated software, then go for it, and know that if you decide to move to more modern hardware you can continue to use your Unraid license with new up to date secure as possible versions.

 

Because of how Unraid is set up, RAM is a non-negotiable requirement. Other distributions may be able to be shoehorned in and continue to work for longer on limited RAM, but Unraid is a poor candidate for limited RAM installs. The market wants more features, not less.

Link to comment
31 minutes ago, primeval_god said:

 Otherwise the user scripts plugin is a good way to run scripts on startup and shutdown. There are other ways which might get you closer to startup but i dont know enough about them to recommend them. Search the forum for adding commands to the go file.


Thanks! That was the right hint - somehow I wasn't exepcting to find array start and stop events as a hook there. 

 

33 minutes ago, primeval_god said:

Just speaking in general though swap is typically not considered as a replacement for having enough ram. This is especially true for unRAID where the OS loads into an runs completely from RAM.

 

Yes, it is a bit of a workaround, I know, but after I've spent 40 bucks on a DDR3 already which didn't work with that board despite promising specs, I'll give that a shot now. Somehow the kernel paging strategy seems to be so effective that the machine works well. It's probably worth to mention that this is not a multi user environment here with 100 of clients interacting with the server all the time.
 

22 minutes ago, JonathanM said:

If you are ok with isolating the machine with external firewall measures, and accepting the risks of running outdated software, then go for it, and know that if you decide to move to more modern hardware you can continue to use your Unraid license with new up to date secure as possible versions.

 

Well, my understanding is that Unraid is not meant to being exposed in a DMZ or similar anyway. (For me the problem on that end would start already with not having 2FA or certificate based root logins, no integrated brute-force protection etc. ) So all good from my perspective. 

Actually I'm quite happy that it works so well with Unraid. Enough flexibility to tweak it a bit so that this old box here can still be useful for something. I'm currently wondering what will die first - harddisks or swap ssd... 😄 
1.png.9cf8d02fb60ad40b319a25564d4cad3f.png

 

Link to comment
7 minutes ago, Tjareson said:

 

Well, my understanding is that Unraid is not meant to being exposed in a DMZ or similar anyway.

Definitely, I'm talking about isolating it from any insecure devices INSIDE your LAN as well. More and more the threats are coming from compromised IOT devices that essentially become abandoned after manufacture.

 

Unraid is working towards being more secure from threats in general, and that is going to come at the price of requiring better hardware, as better encryption and 2FA become available.

 

Sounds like you are in a good place with this, I just wanted to keep your expectations low with your current setup.

 

13 minutes ago, Tjareson said:

I'm currently wondering what will die first - harddisks or swap ssd..

Honestly, if that 10 year old hard drive is well treated, I'm betting it will outlast the SSD. I've retired a good number of disks that were still working but the power to run them wasn't worth the storage they add. Old hard drives are really bad at watt per GB equations, especially when you can get a single drive that holds 18TB.

 

At some point you have to accept that the ongoing power draw is a greater evil than scrapping still useful kit. Run it as long as you can, then buy as much as you can possibly afford so you can use it as long as possible again. Buy once, cry once, or something like that.

Link to comment
7 hours ago, Tjareson said:

The only "issue" I would need to solve now, is to automatically issue a swapon command after the array starts and vice versa a swapoff before the array gets stopped. 

Any ideas where to implement that ideally? 

If you want to get in as early as possible during startup and as late as possible during shutdown then the best place for this would be the config/go and config/stop script files (the latter does not exist by default and would need creating) on the flash drive.   Slightly easier (but needing more of Unraid running) is via the User Scripts plugin.

  • Like 1
  • Thanks 1
Link to comment
  • 4 months later...

Could you change the file creation to use `fallocate -l` instead of `dd` (I assume) that way it's instant

 

EDIT:

I guess not https://askubuntu.com/a/1017312

 

RE-EDIT:

ok maybe it is ok on xfs?

http://manpages.ubuntu.com/manpages/kinetic/en/man8/swapon.8.html

Quote

Preallocated files created by fallocate(1) may be interpreted as files with holes too depending of the filesystem. Preallocated swap files are supported on XFS since Linux 4.18.

 

I'm making a PR now

 

I'm having a problem where after 10GB ish it hangs

Edited by Alicejacka
Link to comment
  • 6 months later...
  • 1 month later...
On 6/18/2022 at 7:43 AM, Alicejacka said:

Could you change the file creation to use `fallocate -l` instead of `dd` (I assume) that way it's instant

 

EDIT:

I guess not https://askubuntu.com/a/1017312

 

RE-EDIT:

ok maybe it is ok on xfs?

http://manpages.ubuntu.com/manpages/kinetic/en/man8/swapon.8.html

 

I'm making a PR now

 

I'm having a problem where after 10GB ish it hangs

Did this PR get implemented? Update log on the plugin shows last updated on 2021.12.09

Link to comment
  • 1 month later...
6 hours ago, itimpi said:

I notice that this plugin says in the Status Line of the GUI that it is creating a swap file even when I start the array in Maintenance mode.  Is it really trying to do this (which it should not) or is the message misleading?

If it says it is i would guess that it is. I forked this plugin from the original quite some time ago, but the core code remains roughly the same. I believe it runs its main script on the 'started' event.

Link to comment

So I used this plug in on two servers.  It seems like it just bogs down the system while the swap is used.  I use a raid 5 as the host for the swap file.  I guess my question is...  adding 8 gigs of swap to the system hurting me?  I can't tell that if I am using the swap file for intentions it wasn't not designed to do.  My dockers some times eat up my ram, so I was adding an extra 8 gigswap as a catch for the extra load so I wouldn't have to restart the specific containers that just eat the ram up.

Link to comment
1 hour ago, thehatch2002 said:

So I used this plug in on two servers.  It seems like it just bogs down the system while the swap is used.  I use a raid 5 as the host for the swap file.  I guess my question is...  adding 8 gigs of swap to the system hurting me?  I can't tell that if I am using the swap file for intentions it wasn't not designed to do.  My dockers some times eat up my ram, so I was adding an extra 8 gigswap as a catch for the extra load so I wouldn't have to restart the specific containers that just eat the ram up.

Yes swap is slower than ram and leaning on it to much will bog a system down. Using swap as insurance to prevent applications for crashing if they temporarily reach the limits of system ram is a good use case. However if you have multiple applications that are just eating up extra ram all the time its possible that are misconfigured somehow. Additionally you mention that you have the swap on raid 5, how so? If you mean your swapfile is in your main array I would recommend against that. Ideally swap should be on a fairly fast pool drive, and it doesnt really benefit from redundancy.  

Link to comment
On 4/22/2023 at 10:37 AM, primeval_god said:

Yes swap is slower than ram and leaning on it to much will bog a system down. Using swap as insurance to prevent applications for crashing if they temporarily reach the limits of system ram is a good use case. However if you have multiple applications that are just eating up extra ram all the time its possible that are misconfigured somehow. Additionally you mention that you have the swap on raid 5, how so? If you mean your swapfile is in your main array I would recommend against that. Ideally swap should be on a fairly fast pool drive, and it doesnt really benefit from redundancy.  

I have a ProLiant 12 bay server with a raid controller with 2x6 drives each in raid 5 configuration, 1 being a cache drive and the other being a parity drive for my external 40 drive jbod disk array.    I thought was cool idea, but after I ran it, seem like more was being put into the swap than I originally thought would actually happen.  I discontinued the swap file all together.  Seems like mine just wants to use the swap even before ran out of ram.

Link to comment
1 minute ago, thehatch2002 said:

I have a ProLiant 12 bay server with a raid controller with 2x6 drives each in raid 5 configuration, 1 being a cache drive and the other being a parity drive for my external 40 drive jbod disk array.    I thought was cool idea, but after I ran it, seem like more was being put into the swap than I originally thought would actually happen.  I discontinued the swap file all together.  Seems like mine just wants to use the swap even before ran out of ram.

I am sure someone or you will ask lol.  I didn't want to individually 6 logical drives for each drive and then use the os to manage the data.  Cache isn't important drive to me, and the other parity logical drive is the 2nd parity drive for my jbod array.

Link to comment
  • 1 month later...
13 hours ago, craigr said:

I know you say straight off no, but this really won't work on a RAID1 SSD pool?  I can see it not working well on other RAID configurations, but am vexed as to why not on RAID1?

 

Thanks.

Per BTRFS documentation 

Quote

There are some limitations of the implementation in BTRFS and Linux swap subsystem:

  • filesystem - must be only single device
  • filesystem - must have only single data profile
  • subvolume - cannot be snapshotted if it contains any active swapfiles
  • swapfile - must be preallocated (i.e. no holes)
  • swapfile - must be NODATACOW (i.e. also NODATASUM, no compression)

https://btrfs.readthedocs.io/en/latest/Swapfile.html

  • Thanks 1
Link to comment

I've got a 256GB Samsung PRO SATA SSD sitting in a drawer and even a Patriot PIRO 64GB, but getting to another SATA port on my LSI board is hindered by other hardware on my motherboard, and that's the last of my SATA ports.  Thinking about it, that 64GB SSD would be perfect.  Swap file would be its only job and it would almost never get used.  I ran out of memory a few days ago for the first time since we switched over to PLEX DVD from TiVo and have hit the wall three times now.  Suddenly 32GB for Dockers isn't enough.  

 

I went ahead an installed your app and used the 10TB cache drive for the swap file location.  I figure better that than run out of memory.  I also edited my dockers to limit their memory, but that will impact their performance all the time and I hardly ever run out of memory.  It's especially going to hurt nzbget.  It sometimes uses more than 16GB on its own if I let it.

Link to comment
On 5/26/2021 at 10:30 PM, primeval_god said:

Swapfile (Unraid 6.9.0 and up)

This plugin adds swapfile creation and management to unRAID. This is a fork of the original "Swap File Plugin for unRAID" by @theone. The goal of this fork is to revamp the plugin for unRAID 6.9.0 and add support for swapfile creation on BTRFS drives.

 

Notes:

  • You must remove the original Swap File Plugin before installing this plugin.
  • This plugin no longer contains the feature to auto-update on startup.
  • Dont forget to check the help text.

 

Limitations:

  • When using BTRFS swapfiles should only be placed on single disk BTRFS drives. BTRFS drive pools with RAID will not work correctly.
  • When using a BTRFS disk the swapfile must be in a subvolume. The plugin will automatically create a subvolume if it is allowed to create the folder in which the swapfile will be placed. Better handling of this requirement will be added soon.

Anyway, I never thanked you for writing this app.  Despite my difficulties in setting a proper location for my swap file, it is working fine on my 10TB cheche drive at the moment, and I will probably be able to work something out on an SSD drive.

 

The 64GB unbuffered ECC udimms I need are prohibitively expensive right now.  Because of your app at the very least, I can bide my time until a good deal comes along.  Also, prices are consistently dropping quickly now.  Reputable vendors are around $170 or more per stick, and I need four.  I have seen them go occasionally for $80 a stick in eBay history.  I'll be able to hold out.

 

So thanks 🙂

Link to comment
2 hours ago, craigr said:

Anyway, I never thanked you for writing this app.  Despite my difficulties in setting a proper location for my swap file, it is working fine on my 10TB cheche drive at the moment, and I will probably be able to work something out on an SSD drive.

 

The 64GB unbuffered ECC udimms I need are prohibitively expensive right now.  Because of your app at the very least, I can bide my time until a good deal comes along.  Also, prices are consistently dropping quickly now.  Reputable vendors are around $170 or more per stick, and I need four.  I have seen them go occasionally for $80 a stick in eBay history.  I'll be able to hold out.

 

So thanks 🙂

I am glad it is working for you. I am however not the one to thank. I did not actually write this plugin, I just to over when it was abandoned.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.