@primeval_god  Thank you for this plugin.

I have a 32GB ram running a Windows VM and a bunch of containers.  Earlier this week, out of the blue docker dropped off, can't ssh to Unraid, and everything is super slow (the VM was running ok).  so basically to recover everything I had to shut down and restart.  Most of the time for some reason, the Unraid USB is not longer detected I'd have to shutdown, pull the power, unplug my powered USB hub, push on button to make sure all power is out, and then plug everything back in, turn on and finally go to BIOS to reconfigure boot device to from Unraid USB.  

Yes, I might have issues with power USB hub backfed on to the motherboard.  but I narrowed it down to Duplicati container chewing up all the memory and caused the errors below

 

Nov 23 03:46:17 node kernel: kworker/u64:15 invoked oom-killer: gfp_mask=0x8c40(GFP_NOFS|__GFP_NOFAIL), order=0, oom_score_adj=0

Nov 23 03:46:17 node php-fpm[10831]: [WARNING] [pool www] child 540983 exited on signal 9 (SIGKILL) after 14.973309 seconds from start

Nov 23 03:47:27 node kernel: hwmon hwmon1: Failed to acquire mutex
Nov 23 03:47:27 node kernel: hwmon hwmon1: update_ec_sensors() failure

 

I've no idea how they are all related, and setting memory limits to duplicate containers made it even worse.  

Finally I installed the swapfile on 7.0.0 beta 4, got the swap file working and managed to complete the backup in Duplicati without anything error.

 

Some say swapfile is not required and if swap file is used, the performance will be horrible.  I do have 10GB ram free at the time of this post,  but I won't have any RAM free while duplicati is running a backup.  so in my case, swapfile has its purpose, because I wouldn't want to upgrade ram just so Duplicati can run backup once every 2 weeks.  Obviously, performance isn't an issue as last backup took about 10 hours and the limiting factor is probably my internet speed.

 

Thank you again.

would like to see an update espcail on the web display for this plugin:
 

 

fresh download:

 

make the folder i want it to be on...

Save and impl;et the setting:

dead.

It will not make the folder...

 

I can premake this then...
now i have a share fodler called swap and must run my user scirp[t to fix for a work around...



v7 rc1 bug idk, idc... regradless its a problem/fixable...
 

Above was a bad example that one is zfsa only ... this is a good example fresh downlaod for the borg:


no share exist:

no array pool devices only:

 

huh... weird inconsentcy with other v7 rc1 borg worked as it should...

 

saved configuration and then hit start...

On my 64gb server i usually have 16gb free, but sometimes i launch a heavy operation and then everything freezes for OOM.

 

I only have btrfs drives in raid1: the instructions say that it can't be done. I ignored the instructions and I got

 

Quote

Feb  3 11:11:10 Server kernel: BTRFS warning (device sdd1): swapfile must have single data profile

 

in the syslog, would be that a problem?

 

Otherwise at work i have many "useless" 16gb usb drives, i could put that there (i have hundreds of them, i don't care if they break, the important is that i don't get freeze for OOM errors that once monthly)

 

edit: from the settings page i see that the swap is not in use so yes, placing it on a btrfs raid1 drive, as the instructions said, won't work

 

 

Edited February 3, 20251 yr by PixelPrint

4 hours ago, PixelPrint said:

I only have btrfs drives in raid1: the instructions say that it can't be done.

Yes this is a limitation of the BTRFS file system so far as i know. No way around it that i am aware of.

 

4 hours ago, PixelPrint said:

Otherwise at work i have many "useless" 16gb usb drives, i could put that there (i have hundreds of them, i don't care if they break, the important is that i don't get freeze for OOM errors that once monthly)

I did this for a long time, one word of warning you want to use a very low swappiness values (maybe 0) to ensure swap is used only when absolutely necessary. The performance of USB flash drives tends to be terrible. I stopped using them for this purpose because i found that my system ground to a near halt when things started swapping out. 

hi @primeval_god,

my cache pool is in zfs, so i start looking for an alternative solution and ended up reading this blog post : https://blog.nerdcruft.net/2017/02/06/linux-swap-over-loopback-devices/

so, I have written a small bash script to implement this using user script plugin:

 

#!/bin/bash

# Chemin du fichier de swap
SWAP_FILE="/mnt/cache_nvme/system/swapfile"

# Taille du swap en Mo
SWAP_SIZE=4096  # 4 Go

# Vérifier si le fichier de swap existe, sinon le créer
if [ ! -f "$SWAP_FILE" ]; then
    echo "Création du fichier de swap..."
    dd if=/dev/zero of="$SWAP_FILE" bs=1M count=$SWAP_SIZE
    chmod 600 "$SWAP_FILE"
fi

# Trouver un loopback disponible
LOOP_DEVICE=$(losetup -f)

# Associer le fichier de swap au loopback
losetup "$LOOP_DEVICE" "$SWAP_FILE"

# Formater en swap
mkswap "$LOOP_DEVICE"

# Activer le swap
swapon "$LOOP_DEVICE"

# Régler le swappiness
SWAPINESS=42
echo $SWAPINESS > /proc/sys/vm/swappiness

echo "Swap activé sur $LOOP_DEVICE avec une taille de $SWAP_SIZE Mo et un swappiness de $SWAPINESS."

if it can help you to implement a workaround for your plugin with zfs...

3 hours ago, milouz said:

hi @primeval_god,

my cache pool is in zfs, so i start looking for an alternative solution and ended up reading this blog post : https://blog.nerdcruft.net/2017/02/06/linux-swap-over-loopback-devices/

if it can help you to implement a workaround for your plugin with zfs...

I dont think this is a good idea. Everything i have read suggests that using swap on ZFS is a bad idea (even if technically possible). The article you linked doesnt mention zfs at all. I have no idea if it would actually work around whatever issues there are with swap on zfs. 

5 hours ago, milouz said:

hi @primeval_god,

my cache pool is in zfs, so i start looking for an alternative solution and ended up reading this blog post : https://blog.nerdcruft.net/2017/02/06/linux-swap-over-loopback-devices/

so, I have written a small bash script to implement this using user script plugin:

 

#!/bin/bash

# Chemin du fichier de swap
SWAP_FILE="/mnt/cache_nvme/system/swapfile"

# Taille du swap en Mo
SWAP_SIZE=4096  # 4 Go

# Vérifier si le fichier de swap existe, sinon le créer
if [ ! -f "$SWAP_FILE" ]; then
    echo "Création du fichier de swap..."
    dd if=/dev/zero of="$SWAP_FILE" bs=1M count=$SWAP_SIZE
    chmod 600 "$SWAP_FILE"
fi

# Trouver un loopback disponible
LOOP_DEVICE=$(losetup -f)

# Associer le fichier de swap au loopback
losetup "$LOOP_DEVICE" "$SWAP_FILE"

# Formater en swap
mkswap "$LOOP_DEVICE"

# Activer le swap
swapon "$LOOP_DEVICE"

# Régler le swappiness
SWAPINESS=42
echo $SWAPINESS > /proc/sys/vm/swappiness

echo "Swap activé sur $LOOP_DEVICE avec une taille de $SWAP_SIZE Mo et un swappiness de $SWAPINESS."

if it can help you to implement a workaround for your plugin with zfs...

if you want swap on zfs review this:

Where I give you how, pros cons and using zfs block device to make the swap...

your better off using zram at that point and if using zfs and run out of ram you can enter a bad loop as zfs uses ram to save data and when swap takes the data and writes to the hd for the ram and continues writes and read form itself...

and look into ram disks... using a part of the ram to act as a disk for swap like in ram only...
 

Reasons to Use Swap on ZFS:

(Recommend to have a single cache disk as btrfs)

 

Handling Memory Overcommitment: If your system is running out of RAM (especially with Docker containers and VMs), swap provides a fallback mechanism to prevent processes from crashing.

 

Avoiding Out-of-Memory (OOM) Errors: Unraid might kill processes if the system runs out of memory. Having swap prevents abrupt service interruptions due to OOM kills.

 

Temporary Relief for High Memory Loads: Swap provides temporary relief during memory spikes, especially for infrequently accessed data, even if it incurs performance penalties.

 

No Other Disks Available: If all your disks are ZFS-based and you have no other storage options (like SSDs or non-ZFS devices), a ZFS-backed swap may be your only choice.

 

Reasons to Avoid Swap on ZFS:

(having the dataset/zfs swap file on a zfs pool)
 

Performance Penalty: ZFS is designed for data integrity and CoW, which can conflict with the typical access patterns of swap. Even when using sync=disabled, ZFS overhead may lead to slower swap performance compared to raw disk or dedicated swap partitions.

 

High Disk Activity: Swap generates frequent small writes, which can lead to excessive disk activity on ZFS pools, reducing their performance and potentially increasing wear on SSDs or HDDs.

 

Risk of Deadlock: ZFS itself consumes memory for managing its datasets (ARC, metadata, etc.). If the system starts swapping ZFS metadata or ARC data, it can lead to a feedback loop where ZFS struggles to free up memory, potentially causing system instability.

 

Better Alternatives Exist: Swap is typically a last resort for extending memory. Using RAM optimally (e.g., limiting VM or Docker resource usage) or adding more physical memory is preferable.
################

I Personally do not recommend running swap off of ZFS pool, dataset, disks... but it is indeed possible... But other technologies and things exist and should be used for zfs... So USE at your own risk!

 

Thanks for your insights.

The goal is to prevent oom with swapiness really low, i'll try just to see!