ronia

Pretty sure this is not how that setting works or has ever worked. The tooltip: "Yes - setting effectively disables this plugin" It disables the plugin, NOT the mover. This only reverts the mover back to the stock Unraid implementation without the filters. The only way to disable the mover entirely is to remove the secondary storage entirely for each share OR to set an extremely high threshold value for free down and an extremely low value for prime up (i think? I no longer use array -> cache shares).

Ignore is currently working for me. Make sure you're specifying the right location as I also made that mistake originally. For example: /mnt/cache/data/cross-seed-links/ /mnt/cache/data/cross-seeds/ /mnt/cache/data/downloads/ /mnt/cache/data/irc/ /mnt/cache/data/torrents/ /mnt/cache/data/media/music/ /mnt/cache/data/scripts/ And not this format: /mnt/user/data/...

At least for me, I'd prefer the threshold to be 90% as I've had issues in the past with appdata when the cache fills higher than 90%. So a global configuration option would be nice. That being said, it's not that big of a deal as honestly most cache:prefer shares should probably just be cache-only shares with a backup solution elsewhere. It shouldn't be this addons sole responsibility to fix the cache implementation for all of unraid 🤣 Not to belittle what @Masterwishx is doing here, but this should really be base functionality in unraid. So many people use this plugin and rely on it and I think the upgrade from v6 to v7 has highlighted just how critical a component it is. If masterwishx didn't step up here, this could have been a much bigger issue for the v7 migration. So thanks again @Masterwishx.

Oh interesting. I didn't know that and that's probably what the issue was. Does this apply to files as well? My files are specified as: /mnt/user/data/file.ext and are also not being ignored correctly.

I believe this is broken. I didn't have time to test further, but I also experienced something similar. I wrote this script that you can run as a mover pre script instead: #!/bin/bash # List of directories for generating ignore.txt ignore_directories=( "/mnt/user/data/cross-seed-links" "/mnt/user/data/cross-seeds" "/mnt/user/data/downloads" "/mnt/user/data/irc" "/mnt/user/data/torrents" "/mnt/user/data/workspace" "/mnt/user/data/media/music" ) # List of directories for touching files # WORKAROUND: ideally, we would just use the ignore list and only touch torrent directories to update hardlinks, but the ignore list doesn't seem to be working. touch_directories=( "/mnt/user/data/torrents" "/mnt/user/data/cross-seed-links" "/mnt/user/data/cross-seeds" "/mnt/user/data/downloads" "/mnt/user/data/irc" "/mnt/user/data/workspace" "/mnt/user/data/media/music" ) # Output file for the list of files output_file="/dev/shm/ignore.txt" # Clear the output file if it exists > "$output_file" # Iterate over ignore_directories for dir in "${ignore_directories[@]}"; do if [ -d "$dir" ]; then echo "Processing directory: $dir" # Find all files in the directory and append their paths to ignore.txt find "$dir" -type f -print >> "$output_file" # Check if the directory is in touch_directories and touch its files recursively if [[ " ${touch_directories[*]} " == *" $dir "* ]]; then echo "Recursively touching files in directory: $dir" find "$dir" -type f -exec touch {} \; fi else echo "Error: $dir is not a valid directory" >&2 fi done echo "Operation completed. File paths saved in $output_file, and applicable files were recursively touched where required." Feel free to adjust to your needs. Some notes: - The variable touch_directories is the workaround here - All directories in touch_directories must also be in ignore_directories as well. - An ignore.txt will be generated, but it will mainly not work. - The idea behind this is that you are touching every file before the mover is invoked, so they are either the least prioritized files or they will not be moved based on age settings. Personally my age is set to 15 days, so these files will never be moved. - You can redirect the output_file anywhere you'd like, but it acts more like a log. - IMPORTANT, this workaround is messing with the CTIME and MTIME of all files. So if these are important to you, do not use this.

This, but you can actually work around it by setting up another share (any share, it doesn't matter), to override the default settings. This will force the mover to reapply the global settings for subsequent shares. For example, I used this on a share that I didn't really want anything moved off unless I needed emergency space:

@Masterwishx I think this data might be useful for the current issue: I have four shares: - 2x are cache:prefer, 2x are cache:yes I had an override originally set for the 2x cache:prefer that set mover tuning to 95% and 0% thresholds to avoid the original issue with cache:prefer shares. With your new fix, this shouldn't be a problem as I think you force all cache:prefer shares to 98% and 0%. So I removed the override and indeed cache:prefer looks good. With no per-share overrides setup, the log looks something like this: The cache:yes share moving threshold is set to 99%, but my global settings are set to 65%. Now If I add a single override (unrelated to the [plex_lib] share, the global setting seems to apply again: I think the key is this: "Restore global settings" log entry. I believe that the global settings are not being applied at the beginning and only get applied during the "restore global settings" step, which is why [plex_lib] is getting the correct setting in the second example.

I'm having issues in general with the mover. I didn't have Reynald's installed prior to unraid 7, so I'm not sure if this is a setup issue or a bug with the mover tuning plugin and v7. It seems to be ignoring the share's mover action. This particular share is setup as such: however, the mover seems to be trying to move it from my cache drive to the array: Jan 12 03:51:27 Dammerung move: Moving "/mnt/cache/./data/workspace/scripts/ffmpeg-2024-11-18-git-970d57988d-full_build/LICENSE" to /mnt/user0/ Jan 12 03:51:28 Dammerung move: 1813 files remaining from caches to array 569GiB In fact, it tried moving my appdata to /mnt/user0 as well until I dropped the array from the secondary storage.

Thanks. I used your solution and it worked great. Another thing you can do is create a file: /boot/config/plugins/customtab/resize.php Then paste the full contents of the <script>...</script> into that file. Then replace this line in /usr/local/emhttp/plugins/customtab/scripts/createTab.php line 121 (the logic that affects 'URL' type tabs): file_put_contents("/usr/local/emhttp/plugins/customtabtemp/$name".mt_rand().".page",$page); Replace with: $mtname = "$name".mt_rand(); file_put_contents("/usr/local/emhttp/plugins/customtabtemp/$mtname.page",$page); file_put_contents("/usr/local/emhttp/plugins/customtabtemp/$mtname.page", file_get_contents('/boot/config/plugins/customtab/resize.php'), FILE_APPEND | LOCK_EX); Bonus points, you can replace the file within /boot/config/plugins/customtab/customtab-[date stamp].txz and re-tar the file and I believe this should make it persistent across reboots.

I think I know what log #2 is. Looking at the timestamps, it seems like this coincides with the accumulation duration. So this is just probably the checker being disabled. Update: Log #1 seems to have disappeared after disabling ssh and Log #3 appears to be from an app starting/stopping (as discussed here: https://forums.unraid.net/topic/77703-solved-eth0-renamed-from-vethxxx-vethxxx-renamed-from-eth0/).

Hello, I am trying to interpret a few log entries in the system log and could use some advice/clarification: Log #1: Jun 30 06:14:30 Dammerung sshd[28970]: Connection from 192.168.1.1 port 60192 on 192.168.1.219 port 22 rdomain "" Jun 30 06:14:30 Dammerung sshd[28970]: error: kex_exchange_identification: Connection closed by remote host Jun 30 06:14:30 Dammerung sshd[28970]: Connection closed by 192.168.1.1 port 60192 Jun 30 10:34:31 Dammerung sshd[7721]: Connection from 192.168.1.1 port 48508 on 192.168.1.219 port 22 rdomain "" Jun 30 10:34:31 Dammerung sshd[7721]: error: kex_exchange_identification: Connection closed by remote host Jun 30 10:34:31 Dammerung sshd[7721]: Connection closed by 192.168.1.1 port 48508 Jun 30 10:58:01 Dammerung sshd[31431]: Connection from 192.168.1.1 port 50198 on 192.168.1.219 port 22 rdomain "" Jun 30 10:58:01 Dammerung sshd[31431]: error: kex_exchange_identification: Connection closed by remote host Jun 30 10:58:01 Dammerung sshd[31431]: Connection closed by 192.168.1.1 port 50198 I think this means that the router (gateway?) received an SSH request on port 60192 and forwarded it to my server at 192.168.1.219. Kinda strange since I thought I had configured my router correctly to reject external SSH requests. To be safe, I've disabled SSH on my server since I'm not really using it right now anyways. Also, given the frequency and range of ports that are being sent from, it seems like it's probably a port scanner? Log #2: Jul 1 08:00:01 Dammerung kernel: mdcmd (37): nocheck pause Jul 1 08:00:01 Dammerung kernel: Jul 1 08:00:01 Dammerung kernel: md: recovery thread: exit status: -4 I tried looking up what this exit status code meant, but found a lot of results on parity errors during parity check. To be fair, July 1st IS parity day for me, but there's no parity errors and I don't have correction enabled on error. This is slightly concerning for me, since I definitely don't want the parity checker to automatically correct any errors. I definitely want to be notified if it found any errors. Log #3: Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethcaedd2b) entered disabled state Jun 29 20:10:47 Dammerung kernel: veth7fe8ad3: renamed from eth0 Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethcaedd2b) entered disabled state Jun 29 20:10:47 Dammerung kernel: device vethcaedd2b left promiscuous mode Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethcaedd2b) entered disabled state Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered blocking state Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered disabled state Jun 29 20:10:47 Dammerung kernel: device vethdd8dc60 entered promiscuous mode Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered blocking state Jun 29 20:10:47 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered forwarding state Jun 29 20:10:48 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered disabled state Jun 29 20:10:48 Dammerung kernel: eth0: renamed from veth730cec0 Jun 29 20:10:48 Dammerung kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethdd8dc60: link becomes ready Jun 29 20:10:48 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered blocking state Jun 29 20:10:48 Dammerung kernel: br-3437dc8a3ec7: port 5(vethdd8dc60) entered forwarding state Jun 29 20:10:55 Dammerung kernel: docker0: port 5(vethe13c685) entered disabled state Jun 29 20:10:55 Dammerung kernel: vethd8b9cdc: renamed from eth0 Jun 29 20:10:55 Dammerung kernel: docker0: port 5(vethe13c685) entered disabled state Jun 29 20:10:55 Dammerung kernel: device vethe13c685 left promiscuous mode Jun 29 20:10:55 Dammerung kernel: docker0: port 5(vethe13c685) entered disabled state Jun 29 20:10:56 Dammerung kernel: docker0: port 5(vethb13c354) entered blocking state Jun 29 20:10:56 Dammerung kernel: docker0: port 5(vethb13c354) entered disabled state Jun 29 20:10:56 Dammerung kernel: device vethb13c354 entered promiscuous mode Jun 29 20:10:56 Dammerung kernel: docker0: port 5(vethb13c354) entered blocking state Jun 29 20:10:56 Dammerung kernel: docker0: port 5(vethb13c354) entered forwarding state Jun 29 20:10:56 Dammerung kernel: eth0: renamed from veth760f68a Jun 29 20:10:56 Dammerung kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethb13c354: link becomes ready This one isn't actually a warning or an error. I'm just curious what it means. It appears sporadically every day at random times. Is it some kind of instability? Is it indicating that the link dropped and needed to renegotiate? Is it just operating system black magic?

I wanted something similar to this as well and this is what I came up with: #!/bin/bash START_TIME=`date +%s` DATE=`date` # Define variables CACHE_DRIVE="/mnt/cache" TARGET_DIRS=("/mnt/cache/plex_lib") DUPLICATES_LOG="/dev/shm/duplicates.log" # Note, this file must be created by another script that tracks duplicated files (ie. tv_series_to_cache script) OUTPUT_DIR="/dev/shm/cache_mover" OUTPUT_FILE="$OUTPUT_DIR/ignore.txt" OUTPUT_FILE_SHADOW="$OUTPUT_DIR/ignore_shadow.txt" # Shadow file so OUTPUT_FILE is always in valid state MOVE_FILE="$OUTPUT_DIR/move_queue.log" # Note, shadow file not needed as move file is not used by any process MOVE_LOG="$OUTPUT_DIR/moved.log" # List of files that are moved if mover is expected to be invoked LOG_FILE="$OUTPUT_DIR/verbose.log" TRIM_RATIO_TARGET=50 # Ratio of used space to target after moving/cleaning MOVER_RATIO_TARGET=80 # Must match the value setup in mover scheduler #EXTENSIONS=("mkv" "srt" "mp4" "avi" "rar") VERBOSE=false ### Perform size calculations on cache drive, target directories and trim target ### cd $CACHE_DRIVE # Get remaining size of cache remaining_size=$(df --output=avail $CACHE_DRIVE | awk 'FNR ==2 {print}') remaining_size=$(( $remaining_size * 1024 )) # Get cache usage cache_used=0 target_dirs_used=0 non_target_dirs_used=0 for dir in */; do dsize=$(du -bs $dir | awk '{print $1}') cache_used=$(($cache_used + $dsize)) for target_dir in "${TARGET_DIRS[@]}"; do full_path=$CACHE_DRIVE/$dir if [ $full_path = $target_dir/ ]; then target_dirs_used=$((target_dirs_used + $dsize)) else non_target_dirs_used=$((non_target_dirs_used + $dsize)) fi done #echo "$dir $dsize" done total_size=$(($cache_used + $remaining_size)) #echo "total_size $total_size" # Get ratio of target_dirs to total size of cache target_dirs_ratio=$(( 100 * $target_dirs_used / $total_size )) remaining_size_ratio=$(( 100 * $remaining_size / $total_size )) max_possible_target_ratio=$(( target_dirs_ratio + remaining_size_ratio )) cache_used_ratio=$(( 100* $cache_used / $total_size )) #echo "target_dirs_ratio=$target_dirs_ratio" if (( max_possible_target_ratio < 100-TRIM_RATIO_TARGET )); then # notification /usr/local/emhttp/webGui/scripts/notify -i alert -s "cache_mover script failed!" -d "Trim ratio target ($((100-TRIM_RATIO_TARGET))%) is greater than possible free space of monitored targets ($max_possible_target_ratio%). Ignore file could not be created." exit fi trim_size=$(( total_size * TRIM_RATIO_TARGET / 100 )) #echo "TARGET_DIRS needs to be trimmed to $trim_size bytes" # Ensure the output directory exists mkdir -p "$OUTPUT_DIR" # Ensure the moved log exists touch $MOVE_FILE # Ensure the duplicates log exists touch $DUPLICATES_LOG # Cleanup previous temporary files rm -f "$OUTPUT_DIR/temp_metadata.txt" "$OUTPUT_DIR/temp_all_files.txt" "$OUTPUT_DIR/temp_deleted.txt" touch "$OUTPUT_DIR/temp_deleted.txt" rm -f $OUTPUT_FILE_SHADOW rm -f $MOVE_FILE # LOG_FILE intentionally kept persistent for target_dir in "${TARGET_DIRS[@]}"; do # Step 1: Change directory to the target directory cd "$target_dir" || exit # Step 2: Find files with specified extensions and obtain metadata (loop through extensions) #for ext in "${EXTENSIONS[@]}"; do # find "$(pwd)" -type f -iname "*.$ext" -exec stat --printf="%i %Z %n\0" {} + >> "$OUTPUT_DIR/temp_metadata.txt" #done # Step 2(alt): Find all files. No filter. find "$(pwd)" -type f -iname "*" -exec stat --printf="%i %Z %n\0" {} + >> "$OUTPUT_DIR/temp_metadata.txt" done # Step 3: Sort metadata by ctime (second column) in descending order sort -z -k 2,2nr -o "$OUTPUT_DIR/temp_metadata.txt" "$OUTPUT_DIR/temp_metadata.txt" # Step 4: Get the newest files up to the specified size limit total_size=$non_target_dirs_used move_size=0 processed_inodes=() while IFS= read -r -d $'\0' line; do read -r inode ctime path <<< "$line" # Check if folder was deleted deleted=false while IFS=':' read -r ln_num; do ln_num=$(echo "$ln_num" | xargs) if [ -z "$ln_num" ]; then break else deleted=true fi done <<< "$(grep -n "$(dirname "${path}")" "$OUTPUT_DIR/temp_deleted.txt")" if [ "$deleted" = true ]; then continue fi # Keep track of all files echo $path >> $OUTPUT_DIR/temp_all_files.txt # Skip if the inode has already been processed if [[ "${processed_inodes[*]}" =~ $inode ]]; then continue fi size=$(stat --printf="%s" "$path") if ((total_size + size <= trim_size)); then if $VERBOSE; then echo "$DATE: Processing file: $total_size $path" >> $LOG_FILE # Debug information to log fi #echo "$path" >> "$OUTPUT_FILE_SHADOW" # Appending only path and filename to the file total_size=$((total_size + size)) # Mark the current inode as processed processed_inodes+=("$inode") # Step 4a: List hardlinks for the current file #hard_links=$(find "$TARGET_DIR" -type f -samefile "$path") #if [ -n "$hard_links" ]; then # echo "$hard_links" >> "$OUTPUT_FILE_SHADOW" #else # echo $path >> $OUTPUT_FILE_SHADOW #fi # Step 4a(alt): Script does not support hardlinks, but is significantly faster and supports multiple TARGET_DIR echo $path >> $OUTPUT_FILE_SHADOW else # TODO: I don't actually think the duplicates logic is necessary. rsync seems to be able to reconcile duplicates when performed manually. Mover uses rsync so it shouldn't be a problem? # Check if file is already duplicated on the array #duplicated=false #while IFS=':' read -r ln_num array_path cache_path; do # # Trim trailing spaces # ln_num=$(echo "$ln_num" | xargs) # array_path=$(echo "$array_path" | xargs) # cache_path=$(echo "$cache_path" | xargs) # # if [ -z "$ln_num" ]; then # break # else # duplicated=true # fi # # # Rsync cache version based on crc. Skip mod-time & size. # rsync -avcP -X "$cache_path"/ "$array_path"/ #done <<< "$(grep -n "$(dirname "${path}")" $DUPLICATES_LOG)" #if [ "$duplicated" = true ]; then # # Remove log entry # sed -i "\#$(dirname "${path}")#d" $DUPLICATES_LOG # # size=$(du -b "$(dirname "${path}")" | awk '{print $1}') # # # Delete cache folder # rm -rf "$(dirname "${path}")" # echo "$(dirname "${path}")" >> "$OUTPUT_DIR/temp_deleted.txt" #fi #if $VERBOSE; then # if [ "$duplicated" = true ]; then # echo "$DATE: Removing duplicate folder: $move_size "$(dirname "${path}")"" >> $LOG_FILE # Debug information to log # else # echo "$DATE: Moving file: $move_size $path" >> $LOG_FILE # Debug information to log # fi #fi move_size=$((move_size + size)) ## Do not add to the move file log if previously added #if ! grep -q "$path" $MOVE_FILE ; then # echo "$DATE: $path" >> $MOVE_FILE #fi # Add the path to the move file echo "$path" >> $MOVE_FILE continue #break fi done < "$OUTPUT_DIR/temp_metadata.txt" # Step 5: Cleanup temporary files rm "$OUTPUT_DIR/temp_metadata.txt" # Step 6: Overwrite the output file with the shadow file. rm -f $OUTPUT_FILE mv $OUTPUT_FILE_SHADOW $OUTPUT_FILE # Step 7: Update move log if mover is expected to run if [[ $cache_used_ratio -ge $MOVER_RATIO_TARGET ]]; then cat $MOVE_FILE >> $MOVE_LOG fi END_TIME=`date +%s` if $VERBOSE; then echo "$DATE: File list generated and saved to: $OUTPUT_FILE" >> $LOG_FILE echo "$DATE: Execution time: $(($END_TIME - $START_TIME)) seconds." >> $LOG_FILE fi Notes: - ignore the "duplicates" log logic. I use this in conjunction with an "array to cache" script that polls tautulli for in progress media and sends it back to the cache. It also probably doesn't work. - my version does not deal with hard links, unlike OPs. It's assumed that the media library is hard link free or being taken care of externally. - I think all you would need to do is cut the portions involving "TRIM_RATIO_TARGET" and "MOVER_RATIO_TARGET" to OPs script. (his is also cleaner because I'm still working/debugging mine)

Sorry to necro this thread, but it was the first topic that came up when I googled "what happens if unraid has two copies of the same file in cache and array". In response to JonathanM, the behavior is like this: 1. Create a test.txt in /mnt/cache with contents "this is on cache" 2. Create a test.txt in /mnt/disk1 with contents "this is on disk1" 3. Create a test.txt in /mnt/disk2 with contents "this is on disk2" Since I was curious about what would happen (and also a little scared about violating cache coherency), this was the result: 1. cat /mnt/user/test.txt -> "this is on cache" 2. delete /mnt/cache/test.txt 3. cat /mnt/user/test.txt -> "this is on disk1" 4. delete /mnt/disk1/test.txt 5. cat /mnt/user/test.txt -> this is on disk2" So it seems like the cache copy is what will be visible to the /mnt/user/ and then each disk's copy is ranked in order of priority based on some kind of inherent priority. Also, I created the cache copy first. If the priority was based on creation/modification date, then it would have displayed disk2's copy as it was technically "newest". This was not the case, so I think you can rule out atime/ctime/mtime. I think this behavior is extremely useful if you're trying to use the cache pool as a real cache and implementing background scripts to intelligently move files between cache and array. Context: I have stumbled across this post while trying to create my own script that rsyncs the contents of an entire tvshow/season from cache to array if it is currently being watched. Odds are, if someone watches episode 1, there is a good chance they will watch episode 2 and so on. My original implementation was to rsync the array copy to the cache, touch it and then delete the array copy (cache coherency) and let the mover move it back after it becomes old again. But actually, now I think I will simply rsync the array copy to cache and leave the array copy. I will write ANOTHER script to handle "old cache copies" when the cache becomes full. The benefits of this are: 1. The array disk is only written once. 2. If a user watches episodes 1 and 2 and then watches episodes 3 and 4 the next day, the array disk is spun up twice. If all episodes are copied to the cache, the disk is spun up once to read the entire season once, but then spun down and left untouched regardless of rewatching episodes, watching only partial episodes at a time, etc.

Hi Everyone, This is an expansion/update on these two extremely helpful guides: Unfortunately, if you're like me, you didn't read either of those guides and purchased 2 years worth of NordVPN and realized that NordVPN does not provide an easy configuration file to setup your WG tunnel. After much experimenting and failure, I believe I have found the solution to manually create the wireguard tunnel. First off, you want to follow this guide for getting your nordvpn private/public keys manually. Note: you will need a Linux OS (an unraid VM works too. I personally spun up a quick ubuntu instance to do just this). Original guide here: https://gist.github.com/bluewalk/7b3db071c488c82c604baf76a42eaad3 But for the most part, in the comments, darki73 created a script for pulling out the relevant information. Re-posted below for convenience: #!/usr/env/bin bash required_packages=() check_if_connected() { if [ -n "$(nordvpn status | grep "Status: Connected")" ]; then return 0 else return 1 fi } # Check whether jq package is installed if ! command -v jq &> /dev/null; then required_packages+=("jq") fi # Check whether wireguard package is installed if ! command -v wg &> /dev/null; then required_packages+=("wireguard") fi # Check if curl package is installed if ! command -v curl &> /dev/null; then required_packages+=("curl") fi # Check if nordvpn package is installed if ! command -v nordvpn &> /dev/null; then required_packages+=("nordvpn") fi # Install missing packages required to generate the configuration file if [ ${#required_packages[@]} -gt 0 ]; then sudo apt install -y "${required_packages[@]}" fi if ! check_if_connected; then nordvpn connect fi interface_name=$(sudo wg show | grep interface | cut -d " " -f 2) private_key=$(sudo wg show $interface_name private-key | cut -d " " -f 2) my_address=$(ip -f inet addr show $interface_name | grep inet | awk '{print $2}' | cut -d "/" -f 1) api_response=$(curl -s "https://api.nordvpn.com/v1/servers/recommendations?&filters\[servers_technologies\]\[identifier\]=wireguard_udp&limit=1") host=$(jq -r '.[]|.hostname' <<< $api_response) ip=$(jq -r '.[]|.station' <<< $api_response) city=$(jq -r '.[]|(.locations|.[]|.country|.city.name)' <<< $api_response) country=$(jq -r '.[]|(.locations|.[]|.country|.name)' <<< $api_response) server_public_key=$(jq -r '.[]|(.technologies|.[].metadata|.[].value)' <<< $api_response) server_identifier=$(echo $host | cut -d "." -f 1) configuration_file="nordvpn-$server_identifier.conf" { echo "# Configuration for $host ($ip) in $city, $country" echo "[Interface]" echo "Address = $my_address" echo "PrivateKey = $private_key" echo "" echo "[Peer]" echo "PublicKey = $server_public_key" echo "AllowedIPs = 0.0.0.0/0" echo "Endpoint = $host:51820" } > "$configuration_file" if check_if_connected; then nordvpn disconnect fi You'll need to sudo apt-get all the relevant tools above. To be clear: The above script/manual steps pull out the relevant information. It does not create a useable configuration file. That would clearly be too easy. Specifically from this step you want: - Your nordvpn private key (henceforth: private key) - Your nordvpn public key (henceforth: public key) - Nordvpn's public key (henceforth: public key 2) - Nordvpn's server address (henceforth: nord server) - Your LynxVPN IP address (henceforth: lynx IP) Now you're ready to setup the tunnel. 1. Head over to Settings> Network Services>VPN Manager in your Unraid server. 2. Add tunnel. (Mine became wg1). 3. Setup the interface to look something like this: 4a. Setup the peer to look something like this (for system): 4b. Setup the peer to look something like this (for dockers): 5. If the tunnel works you should see the handshake like above. Also, you can sanity test by downloading the firefox docker and assigning the network type to "Custom: wg2" (or whatever wg tunnel instance). Then you can navigate to what's my IP or other website to verify the VPN address. Hope this helps!

I just wanted to comment and say that this script is brilliant and it taught me a lot about how caching works in general. I was originally very confused as I was reading through the script since nothing really seems to be "writing the file to cache". I thought that there would be some cache handle or an API, but all I found was: seconds=$( { time head -c "$preload_head_size" "$file" >/dev/null; } 2>&1 ) Surely, I thought, this can't be it since it's reading the first $preload_head_size into /dev/null. Eventually, I realized that the act of reading the header at all causes the operating system to immediately read it into cache on it's own. Thus @onesbug, I believe the answer to your question is that it already works for Jellyfin. In fact, if I've understood this correctly, this is platform agnostic. The OP is reading all the newest files N bytes into the void, of which the act itself is causing the operating system to 'store' it in memory. The operating system doesn't know we're trying to preload video files. All it knows is that you've read "something" and it's keeping a copy of it in memory just in case you want to read that "something" again. This is also probably why it isn't "reserving memory". The contents are "cached" for as long as nothing else is read and requires that space. You simply need to omit the entire section: # check if paths are used in docker containers I actually don't know why you need to know about the docker container. It seems to be largely a sanity check as none of the information from 'docker container inspect' is used later in the script. In fact, if you want to get this to work with cache pools using SSDs (I suspect that OP originally wrote this script before cache pools were introduced), you can change the first part of the script like so: video_paths=( "/mnt/user0/Movie" "/mnt/user0/TV" ) and modify the sanity check as: # check if paths are used in docker containers if docker info > /dev/null 2>&1; then # get docker mounts of all running containers # shellcheck disable=SC2016 docker_mounts=$(docker ps -q | xargs docker container inspect -f '{{$id := .Id}}{{range .Mounts}}{{if .Source}}{{printf $id}}:{{.Source}}{{println}}{{end}}{{end}}' | grep -v -e "^$") # for path in "${video_paths[@]}"; do # if [[ $docker_mounts != *"$path"* ]]; then # /usr/local/emhttp/webGui/scripts/notify -i alert -s "Plex Preloader failed!" -d "$path is not used by a docker container!" # exit 1 # fi # done fi Since the script will complain that /mnt/user0 (the non-cache pool) is not part of any docker container (which makes sense as it wouldn't). Hope this helps anyone else.

Hey thanks for this! This was pretty much exactly what I needed the mover to do with some slight modifications. I've attached my version below in case this is useful for anyone else: #!/bin/bash START_TIME=`date +%s` DATE=`date` # Define variables TARGET_DIRS=("/mnt/cache/plex_lib") OUTPUT_DIR="/dev/shm/cache_mover" OUTPUT_FILE="$OUTPUT_DIR/ignore.txt" MOVE_FILE="$OUTPUT_DIR/moved.log" LOG_FILE="$OUTPUT_DIR/verbose.log" MAX_SIZE="500000000000" # 500 gigabytes in bytes #EXTENSIONS=("mkv" "srt" "mp4" "avi" "rar") VERBOSE=false # Ensure the output directory exists mkdir -p "$OUTPUT_DIR" # Ensure the moved log exists touch $MOVE_FILE # Cleanup previous temporary files rm -f "$OUTPUT_DIR/temp_metadata.txt" "$OUTPUT_DIR/temp_all_files.txt" rm -f $OUTPUT_FILE # MOVE_FILE and LOG_FILE intentionally kept persistent for target_dir in "${TARGET_DIRS[@]}"; do # Step 1: Change directory to the target directory cd "$target_dir" || exit # Step 2: Find files with specified extensions and obtain metadata (loop through extensions) #for ext in "${EXTENSIONS[@]}"; do # find "$(pwd)" -type f -iname "*.$ext" -exec stat --printf="%i %Z %n\0" {} + >> "$OUTPUT_DIR/temp_metadata.txt" #done # Step 2(alt): Find all files. No filter. find "$(pwd)" -type f -iname "*" -exec stat --printf="%i %Z %n\0" {} + >> "$OUTPUT_DIR/temp_metadata.txt" done # Step 3: Sort metadata by ctime (second column) in descending order sort -z -k 2,2nr -o "$OUTPUT_DIR/temp_metadata.txt" "$OUTPUT_DIR/temp_metadata.txt" # Step 4: Get the newest files up to the specified size limit total_size=0 move_size=0 processed_inodes=() while IFS= read -r -d $'\0' line; do read -r inode ctime path <<< "$line" # Keep track of all files echo $path >> $OUTPUT_DIR/temp_all_files.txt # Skip if the inode has already been processed if [[ "${processed_inodes[*]}" =~ $inode ]]; then continue fi size=$(stat --printf="%s" "$path") if ((total_size + size <= MAX_SIZE)); then if $VERBOSE; then echo "$DATE: Processing file: $total_size $path" >> $LOG_FILE # Debug information to log fi #echo "$path" >> "$OUTPUT_FILE" # Appending only path and filename to the file total_size=$((total_size + size)) # Mark the current inode as processed processed_inodes+=("$inode") # Step 4a: List hardlinks for the current file #hard_links=$(find "$TARGET_DIR" -type f -samefile "$path") #if [ -n "$hard_links" ]; then # echo "$hard_links" >> "$OUTPUT_FILE" #else # echo $path >> $OUTPUT_FILE #fi # Step 4a(alt): Script does not support hardlinks, but is significantly faster and supports multiple TARGET_DIR echo $path >> $OUTPUT_FILE else if $VERBOSE; then echo "$DATE: Moving file: $move_size $path" >> $LOG_FILE # Debug information to log fi move_size=$((move_size + size)) # Do not add to the move file log if previously added if ! grep -q "$path" $MOVE_FILE ; then echo "$DATE: $path" >> $MOVE_FILE fi continue #break fi done < "$OUTPUT_DIR/temp_metadata.txt" # Step 5: Cleanup temporary files rm "$OUTPUT_DIR/temp_metadata.txt" END_TIME=`date +%s` if $VERBOSE; then echo "$DATE: File list generated and saved to: $OUTPUT_FILE" >> $LOG_FILE echo "$DATE: Execution time: $(($END_TIME - $START_TIME)) seconds." >> $LOG_FILE fi Summary: - Removed hardlinks from step 4a. I will not have hardlinks in my share, and it significantly improves execution time - Removed looping over an extension list. I didn't feel like there was anything in my share that couldn't be moved between cache and array - Added execution time logging - Added a 'temp_all_files.txt" file. This was useful in convincing myself that the ignore.txt was capturing everything on the share and nothing was missed. You can just diff the two files together when the cache is below MAX_SIZE and there should be identical. - Added a verbose logging mode - Added a 'moved.log' for tracking what has been moved between cache/array - Changed TARGET_DIR to TARGET_DIRS to loop over all potential shares. At the moment this is just my plex library, but I have some ideas where else this could be used. - Moved the output directory to /dev/shm. My mover is setup to run on the hour, so this reduces the number of read/writes on my cache drive