RAM-Disk for Docker status/log files

[Unraidmule]

I am testing this on Unraid v6.12.10

 

What is the best way to know if it is working as intended or functioning correctly? Just log SSD usage using power on hours and the TBW? That's what I have been doing at the moment.

 

Thank you for this!

[Kilrah]

41 minutes ago, Unraidmule said:

 

What is the best way to know if it is working as intended or functioning correctly?

It shows being installed and the backup being done every half hour in the syslog.

Edited April 9 by Kilrah

[Unraidmule]

4 hours ago, Kilrah said:

It shows being installed and the backup being done every half hour in the syslog.

 

Thanks for the reply, I checked the syslog and found an error:

 

docker: Error: RAM-Disk Mod found incompatible files: 8d6094c1d113eb67411e18abc8aaf15d /etc/rc.d/rc.docker 9f0269a6ca4cf551ef7125b85d7fd4e0 /usr/local/emhttp/plugins/dynamix/scripts/monitor

 

IDK how to fix the issue or why it is occurring.

[Kilrah]

Are you on Unraid 6.12.10?

[Unraidmule]

6 hours ago, Kilrah said:

Are you on Unraid 6.12.10?

Yes I upgraded to version 6.12.10 before trying this mod. 

[Kilrah]

1 hour ago, Unraidmule said:

Yes I upgraded to version 6.12.10 before trying this mod. 

And you're using this one?

The one pointed to by the first post is for an older version.

 

[Unraidmule]

5 hours ago, Kilrah said:

And you're using this one?

The one pointed to by the first post is for an older version.

 

 

Thanks for pointing that out to me, just checked the go file, and I'm using:

 

RAM-Disk for Docker json/log files v1.6 for 6.12.5

 

I will upgrade to latest and report back!

 

I really appreciate the help Kilrah.

 

EDIT:

 

Ran the latest script and found these two in the syslog

 

Before starting docker:

 

Tower kernel: RAMDISK: [mem 0x64b36000-0x6689ffff]

 

After starting docker:

 

Tower docker: RAM-Disk created

 

 

So it looks like the script is working now.

 

Thank you for all the help! Will report back with TBW once I give it enough time.

 

 

I have some extra parameters on some containers I tried as a solution before this script, for example:

 

--log-driver none --no-healthcheck --mount type=tmpfs,destination=/tmp,tmpfs-mode=1777,tmpfs-size=256000000


--log-driver syslog --log-opt syslog-address=udp://127.0.0.1:541 --no-healthcheck --mount type=tmpfs,destination=/tmp,tmpfs-mode=1777,tmpfs-size=256000000

 

Should they be removed so they don't interfere with the script?

Edited April 10 by Unraidmule
Update with syslog information

[mgutt]

On 2/13/2024 at 5:40 AM, Mainfrezzer said:

6.12.8

Could you change this in your post to "6.12.8 to 6.12.10"? The link in the first post was updated.

[mgutt]

On 4/10/2024 at 12:15 PM, Unraidmule said:

Should they be removed so they don't interfere with the script?

This won't interfere, but you already avoid writing several log files. Maybe those are useful for you? Then you can now enable them by removing "--log-driver none" and "--log-driver syslog --log-opt syslog-address=udp://127.0.0.1:541" parts.

 

Even "--no-healthcheck" could be removed as well.

[mgutt]

I found a new container which permanently wrote to my SSD:

 

 

[Unraidmule]

4 hours ago, mgutt said:

This won't interfere, but you already avoid writing several log files. Maybe those are useful for you? Then you can now enable them by removing "--log-driver none" and "--log-driver syslog --log-opt syslog-address=udp://127.0.0.1:541" parts.

 

Even "--no-healthcheck" could be removed as well.

Thanks for the reply. Will test out removing those parameters completely. With the script and changing to folder structure I got my writes from ~5 GB per hour to ~2 GB per hour.

 

Is there a solution to reduce it further like having some appdata in RAM disk which also syncs back for chatty dockers like the arr apps?

[mgutt]

18 minutes ago, Unraidmule said:

Is there a solution to reduce it further

 

Yes. Read this post:

 

 

 

 

[karower]

Hi,

I just updated to Unraid 6.12.10 and updated to the script version v1.6 for 6.12.10.

Now I get the following error:

 

May  6 20:17:52 Server docker: Error: RAM-Disk Mod found incompatible files: 60ad47ed6cadcd544f92bd012d30a4f2 /etc/rc.d/rc.docker 99915e9e302b0df53cb3739951f4e33f /usr/local/emhttp/plugins/dynamix/scripts/monitor

 

What does that mean?

 

[Kilrah]

That you didn't use the correct version 

 

 

Edited May 6 by Kilrah

[karower]

Mmh, as mentioned above, I used exactly this version.

 

[bmartino1]

?Is this now a page file of sorts....

This is what docker shm and the swap plugin can do?


Docker extra parameter add --shm-size="3G"
 

[Mainfrezzer]

1 hour ago, karower said:

Mmh, as mentioned above, I used exactly this version.

 

 

 

3 hours ago, karower said:

 

 

May  6 20:17:52 Server docker: Error: RAM-Disk Mod found incompatible files: 60ad47ed6cadcd544f92bd012d30a4f2 /etc/rc.d/rc.docker 99915e9e302b0df53cb3739951f4e33f /usr/local/emhttp/plugins/dynamix/scripts/monitor

 

 

 

shows "99915e9e302b0df53cb3739951f4e33f" which is absolutely not whats in 6.12.10 by default.

You did something in the meantime with the file.

Edit, im a doofus, too many similar posts  but yeah. the file gets modified. Good luck^^

Edit 2: was having a look but i have no clue what rc.docker version that could be from. But yeah basically the whole 6.12.x release had the same monitor file. Theres something seriously changing your files before the go file gets a hand on both of them. Have a look at your scripts and check whats messing with those 2

Edited May 6 by Mainfrezzer

[mgutt]

1 hour ago, bmartino1 said:

This is what docker shm and the swap plugin can do?

My script moves files which are NOT in /dev/shm and/or in the swap partition, into a ram disk. All of them are doing completely different things.

 

You COULD in addition use shm-size for your containers, but I think 99% of the containers won't benefit, as I never faced a container, which permanently writes to /dev/shm. Does anyone know some?

 

And you should NOT think about a swap partition if you have enough ram. The main reason for a swap partition is to avoid going out of ram. You won't gain any benefits from a swap disk, instead it produces more wear on your SSD (which is the opposite of the target if this thread).

[mgutt]

2 hours ago, karower said:

Mmh, as mentioned above, I used exactly this version.

Did you paste it maybe twice or similar? So it changes the file and then it tries to do it again which causes the error message?

[karower]

Yes that's it.

I executed the script twice apparently.

 

Thanks.

Edited May 7 by karower

[bmartino1]

19 hours ago, mgutt said:

My script moves files which are NOT in /dev/shm and/or in the swap partition, into a ram disk. All of them are doing completely different things.

 

You COULD in addition use shm-size for your containers, but I think 99% of the containers won't benefit, as I never faced a container, which permanently writes to /dev/shm. Does anyone know some?

 

And you should NOT think about a swap partition if you have enough ram. The main reason for a swap partition is to avoid going out of ram. You won't gain any benefits from a swap disk, instead it produces more wear on your SSD (which is the opposite of the target if this thread).

Thank you for this info. https://www.kingston.com/en/blog/pc-performance/what-is-ram-disk

Yes, example is some Pihole dockers and some network related dockers use or attempt to touch a swap file / shm for quick packet access and manipulation.

ok, so this gets added to the go file:

On 2/12/2024 at 10:40 PM, Mainfrezzer said:

6.12.7-rc2
As before, will update as we go along.
6.12.10
 

# -------------------------------------------------
# RAM-Disk for Docker json/log files v1.6 for 6.12.10
# -------------------------------------------------

# check compatibility
echo -e "8d6094c1d113eb67411e18abc8aaf15d /etc/rc.d/rc.docker\n9f0269a6ca4cf551ef7125b85d7fd4e0 /usr/local/emhttp/plugins/dynamix/scripts/monitor" | md5sum --check --status && compatible=1
if [[ $compatible ]]; then

  # create RAM-Disk on starting the docker service
  sed -i '/nohup/i \
  # move json/logs to ram disk\
  rsync -aH --delete /var/lib/docker/containers/ ${DOCKER_APP_CONFIG_PATH%/}/containers_backup\
  mountpoint -q /var/lib/docker/containers || mount -t tmpfs tmpfs /var/lib/docker/containers || logger -t docker Error: RAM-Disk could not be mounted!\
  rsync -aH --delete ${DOCKER_APP_CONFIG_PATH%/}/containers_backup/ /var/lib/docker/containers\
  logger -t docker RAM-Disk created' /etc/rc.d/rc.docker

  # remove RAM-Disk on stopping the docker service
  sed -i '/tear down the bridge/i \
  # backup json/logs and remove RAM-Disk\
  rsync -aH --delete /var/lib/docker/containers/ ${DOCKER_APP_CONFIG_PATH%/}/containers_backup\
  umount /var/lib/docker/containers || logger -t docker Error: RAM-Disk could not be unmounted!\
  rsync -aH --delete ${DOCKER_APP_CONFIG_PATH%/}/containers_backup/ /var/lib/docker/containers\
  logger -t docker RAM-Disk removed' /etc/rc.d/rc.docker

  # Automatically backup Docker RAM-Disk
  sed -i '/^<?PHP$/a \
  $sync_interval_minutes=30;\
  if ( ! ((date("i") * date("H") * 60 + date("i")) % $sync_interval_minutes) && file_exists("/var/lib/docker/containers")) {\
    exec("\
      [[ ! -d /var/lib/docker_bind ]] && mkdir /var/lib/docker_bind\
      if ! mountpoint -q /var/lib/docker_bind; then\
        if ! mount --bind /var/lib/docker /var/lib/docker_bind; then\
          logger -t docker Error: RAM-Disk bind mount failed!\
        fi\
      fi\
      if mountpoint -q /var/lib/docker_bind; then\
        rsync -aH --delete /var/lib/docker/containers/ /var/lib/docker_bind/containers && logger -t docker Success: Backup of RAM-Disk created.\
        umount -l /var/lib/docker_bind\
      else\
        logger -t docker Error: RAM-Disk bind mount failed!\
      fi\
    ");\
  }' /usr/local/emhttp/plugins/dynamix/scripts/monitor

else
  logger -t docker "Error: RAM-Disk Mod found incompatible files: $(md5sum /etc/rc.d/rc.docker /usr/local/emhttp/plugins/dynamix/scripts/monitor | xargs)"
fi

 

a ram disk exists. Now what? do supported docker just use it?

Does this survive in place upgrades?

[mgutt]

1 hour ago, bmartino1 said:

Yes, example is some Pihole dockers and some network related dockers use or attempt to touch a swap file / shm for quick packet access and manipulation.

Nothing can write to swap except the Linux Kernel which calculates how important the used memory is and then moves really rare used blocks to the swap partition. This article is "pro" swap (which I'm not as an unRAID server usually has more than enough ram installed or do you have 4GB installed?), but describes how it works:

https://chrisdown.name/2018/01/02/in-defence-of-swap.html

 

PS: I tested swap under Unraid in the past and it is rarely/not used.

 

In contrast all files written to /dev/shm are alreadx written to the ram as this is by default in Linux a ram disk (tmpfs = ram):

tmpfs 32G 0 32G 0% /dev/shm

 

As you can see my unraid server uses 0% of it after month of uptime.

 

And there is no need to mount /dev/shm to /dev/shm in the container configuration as /dev/shm is by default a RAM disk in ALL containers:

https://docs.docker.com/engine/reference/run/

 

Example:

adguard - sh

/opt/adguardhome/work # df -h                                                     Filesystem                Size      Used Available Use% Mounted on
overlay                   3.6T      2.4T      1.3T  65% /                         tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm

 

More details about the "shm" filesystem:

https://unix.stackexchange.com/a/124788/101920

 

Regarding the Kingston page: Windows bullshit. Not comparable with Linux.

 

1 hour ago, bmartino1 said:

Now what? do supported docker just use it?

There is no special "support" needed as it replaces a default docker path against a ram disk as explained in my first post:

https://forums.unraid.net/topic/136087-ram-disk-for-docker-statuslog-files/#comments

 

 

Means: It is used by ALL containers.

 

If you want to check if it is running: Check your logs as mentioned in my first post.

 

1 hour ago, bmartino1 said:

Does this survive in place upgrades?

It depends if limetech changes the files which are manipulated by the script. Sometimes yes, sometimes not. If not: it returns an error message in the logs.

 

If you like to receive notifications in this case I can recommend this script:

 

 

 

 

[bmartino1]

Thank you for that information, I randomly stumbled into this post reviewing old code and scripts. I did read the first post, but not all of it was clear-cut. Thank you for answering my questions.

I agree with the premise there is enough ram, why is the software coded to use my ssd/nvme for swamp memory when there is enough ram...

I have 32 GB of ram in my system


I don't think I have used more than 16-20 GB on this system in total with everything running and taxed...

Some dockers I have run/tested and deployed to others with similar hardware... I have found that some dockers images that are built on Debian and that used Debian sometimes generate a swap error...(Example unifi docker, Linux.io dockers...etc...) I have found Linux systems and other Damon that can write and use the swap file. It may even be just be a check file for other code... This has been my experience, which is why I asked...


I installed the plugin and crated the file but not start swap and haven't seen that error since.

For docker, in my use case it can be used to stop i/o thrashing... and to stop the template docker run error of swap... some nano docs also writes in swap before saving to the filesystem... Other file systems like truenas scale have been known to touch the swap file before adding to the zfs pool... 

I have found if I installed the swap plugin and have it created, swap is no longer a problem. So a swap file exist but is not used. (tested with other tools). Then I never see the docker errors again. To me its more of a stop gap. But unraids docker system depending on docker image will use host swap to include some qemu/vfio task. I don't think it is a issue with resource but with security and data access.

Regarding the SHM over tmpfs. This was a weird one off with some Ddos pen testing with dns server the shm size in the unraid pihole docker error-ed and per piehole documentation that was the fix to increase it memory. I have yet to actually need the use of the swap/page file nor the shm. Then trying to understand its use case with this new concept of the ram disk.

Outside of wanting to use my ram with this ram disk instead of my ssd cache disk swap for ram. I'm not 100% sure this fits my use case. as it was more to stop errors. nice to learn new software and hardware technologies like this and its use case with a ram disk.

Thank you again for info on the syslog key search term script, I will definite be taking a look at that.

Edited May 8 by bmartino1

[mgutt]

7 hours ago, bmartino1 said:

some dockers images that are built on Debian and that used Debian sometimes generate a swap error.

It's not an error. Only a warning. I don't know why this message is returned recently. It is only confusing for the user.

 

It has nothing to do with Debian containers. It's returned if the container uses the setting "--memory xGB" (which limits the ram usage if the container = warning it could be cause crashing the container as no swap exists = which is perfectly ok I think as the container should crash if it is going crazy in RAM usage).

 

 

 

 

 

 

[Crovaxon]

Arrived here following after the crumbs of "exit status 255" errors, I had been using an old version of the script. Removed it from my go file and set up the new one as a userscript now. Seems all good and working now according to logs.
Again thanks for this script and the effort to extend the life of cache SSDs  

Edited June 19 by Crovaxon