kizer

Until you make an accidental change in your settings and realize your moving 2TB of data and are upset you machine is taking forever to move said files and can't stop it. I know I over exaggerated some there, but things happen and keeping things simple in my humble opinion has always been astrong suit of unraids design.

lol, I don't think I've ever had to use a 12-Digit password before.

This is the latest bit of code I have from the guy who is keeping it up to date. I didn't write any of this so please don't consider me an expert on the topic. Lol It just works for me. #!/bin/bash # This script gets 19tives torrents list from Transmission, playing sessions from Plex, and tells rsync to copy them from array to cache drive. # It also cleans oldest modified files by rsyncing them back to array (in case of modification). # Hardlink are preserved # # By Reynald - 06 may 2020 - mailto:[email protected] # modified by quinto to work with plex v1.29 or later # v.0.5.19 # settings { #Plex PLEX_TOKEN=“xxxxxxxxxxxxx” PLEX_HOST="192.168.7.127:32400" PLEX_MAX_CACHED_SESSIONS=5 PLEX_CONTAINER_PATH="Media" #Rsync path STORAGE_PATH="/mnt/user0/Media/" CACHE_PATH="/mnt/cache/Media/" CACHE_MIN_FREE_SPACE_PCT="90" CACHE_MAX_FREE_SPACE_PCT="70" #Parameters LOG_MAX_SIZE=5000000 NOISY_HOUR_START=9 NOISY_HOUR_STOP=24 #if you're not using Youtube-dl agent for plex don't touch this line, it must only be changed when you're using yt-dl custom agent YOUTUBE_DL_AGENT="com.plexapp.agents.youtube-dl" #Options (set to true or false) DRY_RUN=false WRITE_ON_STORAGE=false #number of episodes to cache PLEX_CACHE_NB_EPISODES=6 PLEX_CACHE_SEASON_TILL_END=false VERBOSE=1 #0=Error; 1=Info; 2=More_Info; 3=Debug } ##### No modification below this line ##### CACHE_DISK=$(df -lah "$CACHE_PATH" | awk 'FNR == 2 {print $1}' | tr -d '\n') sys_checks() { # check if plex docker is running plex_running=`docker inspect -f '{{.State.Running}}' $PLEX_DOCKER` if [[ ! $plex_running ]] then echo "Error: Plex docker is not running" exit 1 fi # lock if [[ -f /var/lock/smart-cache_plex_transmission ]] then echo "Error: Script already running" exit 1 else touch /var/lock/smart-cache_plex_transmission [[ $VERBOSE -ge 2 ]] && echo "Welcome to $0" fi # check that path are mounted if [[ ! -d $STORAGE_PATH ]] || [[ ! -d $CACHE_PATH ]]; then echo "Error: Paths are not accessibles" rm /var/lock/smart-cache_plex_transmission exit 1 fi # cut log LOG_FILE=$(echo $0 | sed 's|\/script|\/log.txt|') echo "log= $LOG_FILE" LOG_SIZE=$(stat -c %s "$LOG_FILE") [[ $VERBOSE -ge 1 ]] && echo "Info: Log size is $LOG_SIZE" if [[ $LOG_SIZE -ge $LOG_MAX_SIZE ]] then [[ $VERBOSE -ge 1 ]] && echo "Info: Emptying log file" echo "" > "$LOG_FILE" fi [[ $VERBOSE -ge 1 ]] && echo "" } ####################### # Transfers functions # ####################### noisy_hours() { # return 0 if time in noisy hour range if [[ $(date '+%-H') -ge $NOISY_HOUR_START ]] && [[ $(date +%-H) -le $NOISY_HOUR_STOP ]] then return 0 else return 1 fi } rsync_transfer() { # get files and path SOURCE_FILE=$1 DEST_FILE=$2 SOURCE_PATH=$3 DEST_PATH=$4 RS_OPTIONS=$5 [[ $VERBOSE -ge 3 ]] && echo " --- Debug:Rsync_transfer function parameters:" [[ $VERBOSE -ge 3 ]] && echo " ---- Debug: Source file: $SOURCE_FILE" [[ $VERBOSE -ge 3 ]] && echo " ---- Debug: Dest. file: $DEST_FILE" [[ $VERBOSE -ge 3 ]] && echo " ---- Debug: Source path: $SOURCE_PATH" [[ $VERBOSE -ge 3 ]] && echo " ---- Debug: Dest. path: $DEST_PATH" [[ $VERBOSE -ge 3 ]] && echo " ---- Debug: Options : $RS_OPTIONS" # check if original file exist if [[ ! -f "${SOURCE_FILE}" ]] && [[ ! -f "${DEST_FILE}" ]] then echo " --- Error: Files:" echo " ${SOURCE_FILE}" echo " ${DEST_FILE}" echo " does not exist" return 1 elif [[ "${DEST_FILE}" = "${DEST_PATH}" ]] || [[ "${SOURCE_FILE}" = "${SOURCE_PATH}" ]] then echo " --- Error: Cannot sync root path!" return 1 elif [[ ! -f "${SOURCE_FILE}" ]] && [[ "${DEST_PATH}" = "${CACHE_PATH}" ]] && [[ -f "${DEST_FILE}" ]] then if noisy_hours then if [[ ! WRITE_ON_STORAGE ]] then [[ $VERBOSE -ge 2 ]] && echo " --- Info: File is on cache only. Inside noisy hours, sending to storage" rsync_transfer "${DEST_FILE}" "${SOURCE_FILE}" "${DEST_PATH}" "${SOURCE_PATH}" fi else [[ $VERBOSE -ge 2 ]] && echo " --- Warning: File is on cache only. Outside of noisy hours, doing nothing" fi return elif [[ -f "${DEST_FILE}" ]] && [[ "${DEST_PATH}" = "${CACHE_PATH}" ]] then [[ $VERBOSE -ge 2 ]] && echo " --- Info: File already cached" return fi # get dir SOURCE_DIR=$(dirname "${SOURCE_FILE}") DEST_DIR=$(dirname "${DEST_FILE}") if ! $DRY_RUN then # sync file mkdir -p "${DEST_DIR}" [[ $VERBOSE -ge 1 ]] && echo " --- Info: Syncing ${SOURCE_FILE} to ${DEST_FILE}" nice -n 13 ionice -c 3 /usr/bin/rsync --bwlimit=190000 -aHq "${SOURCE_FILE}" "${DEST_FILE}" if [[ ! $? -eq 0 ]] then echo " --- Error: cannot rsync ${SOURCE_FILE}" echo " to ${DEST_FILE}" return 1 fi # remove original file if requested if [[ "${RS_OPTIONS}" = "--remove-source-files" ]] && $RSYNC_RESULT then [[ $VERBOSE -ge 2 ]] && echo " --- Info: Delete ${SOURCE_FILE}" rm "${SOURCE_FILE}" fi else echo " --- Running in DRY RUN mode : Files aren't really copied, doing nothing" fi } ######## # Plex # ######## plex_cache() { # get Plex sessions STATUS_SESSIONS=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/status/sessions -H "X-Plex-Token: $PLEX_TOKEN") if [[ -z $STATUS_SESSIONS ]]; then echo "Error: Cannot connect to plex" return 1 fi NB_SESSIONS=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/@size)' -) echo "----------------------------" echo "$NB_SESSIONS active(s) plex session(s):" echo "----------------------------" # for each session if [[ $NB_SESSIONS -gt $PLEX_MAX_CACHED_SESSIONS ]] then NB_SESSIONS=$PLEX_MAX_CACHED_SESSIONS echo "Warning: Caching is limited to $PLEX_MAX_CACHED_SESSIONS plex sessions (user setting)" fi for i in `seq $NB_SESSIONS` do # get title ID=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@ratingKey)' -) TYPE=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@type)' -) MKEY=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@key)' -) # eventually get serie info if [[ $TYPE = "episode" ]] then TYPE="Serie" GRANDPARENTTITLE=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@grandparentTitle)' -) SEASON=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@parentIndex)' -) TITLE=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@title)' -) EPISODE=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@index)' -) PARENT_ID=$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@parentRatingKey)' -) PARENT_SESS=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/library/metadata/$PARENT_ID/children) PARENT_NB_EPISODES=$(echo $PARENT_SESS | xmllint --xpath 'string(//MediaContainer/@size)' -) PARENT_START_EPISODE=$(echo $PARENT_SESS | xmllint --xpath 'string(//MediaContainer/Video[1]/@index)' -) SECTION_ID=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/library/metadata/$PARENT_ID | xmllint --xpath 'string(//MediaContainer/@librarySectionID)' -) SECTION_AGENT=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/library/sections?X-Plex-Token= | xmllint --xpath 'string(//MediaContainer/Directory[@key='$SECTION_ID']/@agent)' -) if [[ ${SECTION_AGENT} = "com.plexapp.agents.youtube-dl" ]]; then TYPE="Youtube"; fi TITLE="$TYPE: ${GRANDPARENTTITLE} Season ${SEASON} - Episode ${EPISODE}/${PARENT_NB_EPISODES}: $TITLE" # update nb file to cache START_FILE="$EPISODE" if [[ ${SECTION_AGENT} == ${YOUTUBE_DL_AGENT} ]] then NB_FILES=$(( $EPISODE )) else if [[ $PLEX_CACHE_NB_EPISODES -gt 0 ]]; then NB_FILES=$(( ($EPISODE + $PLEX_CACHE_NB_EPISODES) < $PARENT_NB_EPISODES ? ($EPISODE + $PLEX_CACHE_NB_EPISODES) : $PARENT_NB_EPISODES )); fi $PLEX_CACHE_SEASON_TILL_END && NB_FILES=$(( $PARENT_NB_EPISODES )) fi elif [[ $TYPE = "movie" ]] then NB_SESS=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}${MKEY}?checkFiles=1?includeChildren=1?X-Plex-Token=) NB_MEDIAS=$(echo $NB_SESS | xmllint --xpath 'count(//Media)' -) NB_FILES=$(echo $NB_SESS | xmllint --xpath 'count(//Part)' -) TYPE="Movie" TITLE="$(echo $STATUS_SESSIONS | xmllint --xpath 'string(//MediaContainer/Video['$i']/@title)' -)" TITLE="$TYPE: $TITLE" START_FILE=1 else TYPE="Audio" TITLE="track caching not implemented" TITLE="$TYPE: $TITLE" START_FILE=0 NB_FILES=0 fi echo " - $i / $NB_SESSIONS: $TITLE" PLEX_FILE_SESS_MOVIE=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/library/metadata/$PARENT_ID/children) PLEX_FILE_SESS_TVSHOW=$(nice -n 13 ionice -c 3 curl --limit-rate 200k --silent http://${PLEX_HOST}/library/metadata/$ID) for j in `seq $START_FILE $NB_FILES` do # get file path if [[ $TYPE = "Audio" ]] then [[ $VERBOSE -ge 2 ]] && echo " -- Info: Skipping" else if [[ $TYPE != "Movie" ]] then if [[ $TYPE = "Youtube" ]] then # media is a youtube episode PLEX_FILE=$(echo "${PLEX_FILE_SESS_MOVIE}" | xmllint --xpath 'string(//MediaContainer/Video['$(($PARENT_START_EPISODE - $j + 1))']/Media/Part/@file)' -) else # media is a tv show PLEX_FILE=$(echo "${PLEX_FILE_SESS_MOVIE}" | xmllint --xpath 'string(//MediaContainer/Video['$(($j - $PARENT_START_EPISODE + 1))']/Media/Part/@file)' -) fi m=$PARENT_NB_EPISODES else # media is a movie if [[ $NB_FILES -gt 1 ]] then if [[ $NB_FILES -gt $NB_MEDIAS ]] then # multiple parts movie (part1, part2, etc) PLEX_FILE=$(echo "${PLEX_FILE_SESS_TVSHOW}" | xmllint --xpath 'string(//MediaContainer/Video/Media/Part['$j']/@file)' -) else # multiple medias movie (4k, 1080p, 720p, etc) PLEX_FILE=$(echo "${PLEX_FILE_SESS_TVSHOW}" | xmllint --xpath 'string(//MediaContainer/Video/Media['$j']/Part/@file)' -) fi else # movie in single part / format PLEX_FILE=$(echo "${PLEX_FILE_SESS_TVSHOW}" | xmllint --xpath 'string(//MediaContainer/Video/Media/Part/@file)' -) fi m=$NB_FILES fi FILE_TO_CACHE="$(sed -e 's|\"\"|\/|g' -e 's/%/\%/g' <<<${PLEX_FILE} | sed 's|\"||g' | sed 's|\/'${PLEX_CONTAINER_PATH}'\/||')" [[ $VERBOSE -ge 2 ]] && echo " -- Info: Streaming File $j/$m: ${FILE_TO_CACHE}" STORAGE_FILE="${STORAGE_PATH}${FILE_TO_CACHE}" CACHE_FILE="${CACHE_PATH}${FILE_TO_CACHE}" # and send to rsync rsync_transfer "${STORAGE_FILE}" "${CACHE_FILE}" "${STORAGE_PATH}" "${CACHE_PATH}" fi sleep .8 done sleep .8 done sleep .8 # [[ $NB_SESSIONS != 0 ]] && echo "" [[ $VERBOSE -ge 1 ]] && echo "" } #################### # Delete old files # #################### cleanup() { # get free space a=$(df -h | grep $CACHE_DISK | awk '{ printf "%d", $5 }') b=$CACHE_MIN_FREE_SPACE_PCT echo "---------------------" echo "Cache disk usage: ${a}%" echo "---------------------" if [[ "$a" -ge "$b" ]]; then echo "$a% space used, quota is $b%, cleaning" [[ $VERBOSE -ge 1 ]] && echo "Info: Scanning files..." # get oldest accessed files find "${CACHE_PATH}" -type f -printf "%[email protected] %p\n" | sort -n | sed "s|`echo ${CACHE_PATH}`|%|g" | cut -d'%' -f2 | while read FILE_TO_CLEAN do # loop start: get free space again a=$(df -h | grep $CACHE_DISK | awk '{ printf "%d", $5 }') b=$CACHE_MAX_FREE_SPACE_PCT # if free space not enough if [[ "$a" -ge "$b" ]]; then [[ $VERBOSE -ge 1 ]] && echo " - Info: $a% space used, target $b%, uncaching $FILE_TO_CLEAN" STORAGE_FILE="${STORAGE_PATH}${FILE_TO_CLEAN}" CACHE_FILE="${CACHE_PATH}${FILE_TO_CLEAN}" # sync back cache to storage rsync_transfer "${CACHE_FILE}" "${STORAGE_FILE}" "${CACHE_PATH}" "${STORAGE_PATH}" "--remove-source-files" fi # loop done fi a=$(df -h | grep $CACHE_DISK | awk '{ printf "%d", $5 }') b=$CACHE_MIN_FREE_SPACE_PCT [[ $VERBOSE -ge 1 ]] && echo " - Info: $a% space used, quota is $b%, nothing to do" # prune empty directories from source dir [[ $VERBOSE -ge 2 ]] && echo " -- Info: Cleaning empty directories..." find "${CACHE_PATH}" -type d -not -path '*/\.*' -empty -prune -exec rmdir --ignore-fail-on-non-empty {} \; [[ $VERBOSE -ge 1 ]] && echo "" } sys_checks plex_cache cleanup echo "" rm /var/lock/smart-cache_plex_transmission exit 0

Well things I'd wonder about. What time of day is your mover set to run? Are you using Plex and have the daily schedule set for that time period?

I personally use the Mover Tuning plugin and have it set to move after a week. That way i can access the files from the Cache and the array, but they are moved automatically after a period of time vs daily. Also the upside is I'm sure the files aren't open or being used unless one of my kids is up watching a video at 5AM. 😃

Same! Lol Litterally everything you said is the same. I was outside working with the younger crowd and I about died. lol

Yep Old Hoopster. I've been beaten to countless posts by this guy. Sometimes I wonder does he sleep with a laptop on his chest? 😃

I don't know of a solution for your sound, but you can schedule your conversions for times when your not using your Machine like when your sleeping or you can pin which cores are doing the conversion to leave some breathing room for your system.

My 2GB Sandisk Cruiser has been error free since 2009. It only boots up from it and then it just sits idle for days on end.

This is exactly what I did. Ran it as a NAS for a few days over the weekend while we was gone then went back to normal and turn on 1 thing at a time. We use Plex a lot so I turned on only Plex docker than each day I turned on another and another. Come to find out one of my dockers wasn't configured right and it was eating up resources and crashing. Sometimes the simplest things are the key to fixing things.

Same. I got lucky that my Motherboard has an onboard USB port. Just plug it in and forget about it. I know its no help, but every time I do any major changes to my machine I always store a backup on my windows PC. Add a new drive, upgrade OS...... on and on........ I would store it on Unraid.net, but it appears I only have a 2GB stick and I don't have enough room to make a backup. Lol

Everything is tied to your USB stick including My Servers.

Some of us have been using unraid since 2009 or earlier. Trust me it works. It might be just a slight misunderstanding of how things work or how they are to be setup. Looking at your Split Level you have it set to first level only. That means only the main folder will be split.

Not exactly the same, but I used a find command that checks a folder and it copies from A to B and moves to C. find /mnt/cache/uploads/incoming/* -iname "*.mkv" -type f -mmin +10 -exec cp {} /mnt/cache/uploads/2check/ \; -exec mv {} /mnt/cache/uploads/Tv-Incoming/ \; So it checks Incoming folder and copies to 2check then moves incoming to Tv-incoming So this kinda does what your asking, but not exactly. You could look for some code that utilizes inotifywait

I know at one time unless its still active Plex added a feature that would save your viewing in the cloud. I don't know if its for all users or just the main account. Maybe your experiencing the cloud watched status. I don't think I personally opted in for the cloud watched status, maybe you did?

I've never done it, but I probably would of copied my database from the VM install into the new docker which, should of captured your watched status, list of movies and tv shows. I've only really copied info from docker to docker which ideally is the same.

I honestly would recommend that as well. Why run Plex via a VM when you can loose the overhead and just run Plex directly as a docker. If you really like your VM you can still use it for other things when you need to.

Can't you use Direct Streaming? If the Playstation 4 is capable of playing 4K directly then it shouldn't need to transcode. I have 4 AppleTv's in my house and I use direct streaming so their is no need to Transcode anything. I don't use 4K since most of my content is ripped to 1080p so I can't truly comment on 4k content.

Please note. This thread is for Announcing the newest release and mentioning any issues you might have. Please make another post in General requiring any technical support and link to it. This is one of the only times we are asking you to cross post. By doing this it reduces the amount of clutter that might happen and direct others that might have the same problem to the discussion.

tdarr is nice if you need to convert some of your files

Lol No. I was hoping for something in unraid not my garage. All fixed now. Well the garage is. I haven't restarted MyPlugin. I'll tinker with that this weekend.

Lol, My 2GB ScanDisk Cruzer is still rocking since 2009. I've had a handful of oddities since I've had it thou. Not sure if it was the stick or the Mobo.

PFt..... I'm taking you out of my will then. lol Nah I just wanted to give you some context of my above frustration. I know sometimes text doesn't convey feelings/body language and I honestly didn't check to make sure I didn't come across as overly aggressive or annoyed in my Original post.

I'll look into it again. This weekend was a 4 day and if it could happen it did. Server was acting wonky, my kid backed out of my garage and wrecked the garage door, Thanksgiving everybody was late. Lol Thanks I'll look into this again. I really enjoy remote access and security. Not sure why I started having a few issues when its been running flawlessly for months. Didn't help that I have ssh and telnet disabled and at the time my monitor wasn't connected. Lol Ah you can temp disable via console. Nice I was hoping for a pseudo back door in.

This site can’t be reached Check if there is a typo in 192-168-7-127.4d4f6da4506b4508f3acafda8.myunraid.net. DNS_PROBE_FINISHED_NXDOMAIN I'm getting this for some odd reason. I can't admin my server locally and now I'm sitting at a black screen because I'm trying to use the GUI. Is there a way to disable all this temporary so I can admin my machine with a browser? When I type in my URL 192.168.7.127 I get the above even on my local network. Do I have a setting turned on or is there a way to disable that. I finally was able to login using GUI no plugins, but I can't disable the plugin since its not installed. I just want to get rid of that bizarre URL that isn't allowing me to manage my server until I figure out what is going on. Update turned off Strict so I can resolve the URL and connect internally. Was finally able to get in via no Plugins using GUI.