I have emby transcode to RAM (/dev/shm/) which works well except for garbage collection – temporary transcode files accumulate.

 

To solve that I wrote a script (run by the container) to delete temporary files and a user-script to launch it when the container's restarted or if it's not running for any other reason.

 

Assumptions:

 

/transcode

is the container's path to /dev/shm/ (or wherever you're transcoding to)

 

/transcode/transcoding-temp

is the container's path to the directory holding temp transcoding files (emby creates this subdirectory)

 

/system-share/transcoding-temp-fix.sh

is the container's path to the following script (make the script executable)

 

transcoding-temp-fix.sh

#!/bin/sh

TRANSCODE_DIR="/transcode/transcoding-temp"
# Delete old files when used space is above this %
PERCENT_LIMIT=50
# Delete this many files at a time
BATCH_SIZE=10

if [ -d "${TRANSCODE_DIR}" ]; then
    percent_full=$(df "${TRANSCODE_DIR}" | awk '{print $5}' | tail -1 | tr -dc '0-9')
    printf "Directory size: \t %3s%%\n" ${percent_full}
    printf "Directory limit:\t %3s%%\n" ${PERCENT_LIMIT}
    echo ""
    while [ $percent_full -gt $PERCENT_LIMIT ]; do
        if [ $(find ${TRANSCODE_DIR} -type f -name "*.ts" | wc -l) -gt 0 ]; then
            echo "(${percent_full}%) exceeds limit (${PERCENT_LIMIT}%), deleting oldest (${BATCH_SIZE}) files"
            find ${TRANSCODE_DIR} -type f -name "*.ts" -exec ls -1t "{}" + | tail -${BATCH_SIZE} | xargs rm
        else
            echo "*WARNING* (${percent_full}%) exceeds limit (${PERCENT_LIMIT}%) but files are not transcoding fragments"
            exit 1
        fi
        percent_full=$(df "${TRANSCODE_DIR}" | awk '{print $5}' | tail -1 | tr -dc '0-9')
    done
else
    echo "${TRANSCODE_DIR} (TRANSCODE_DIR): directory doesn't exist"
fi

 

Now the user script to launch it, set to run every 10 minutes (*/10 * * * *)

NOTE: Update EmbyServer name and system-share (if necessary) to match your system

#!/bin/bash
#arrayStarted=true
#clearLog=true

# Verify EmbyServer's running
running=$(docker container ls | grep EmbyServer | wc -l)
if [ "${running}" != "0" ]; then
  # verify watch command that calls clearing script is running
  watch_running=$(docker exec -i EmbyServer ps | grep 'watch ' | wc -l)

  # make sure the detection command ran properly otherwise
  # we might end up running multiple instances of the script
  if [ $? -eq 0 ]; then
    if [ "${watch_running}" == "0" ]; then
      echo "Clearing script is not running. Re-starting..."
      docker exec EmbyServer sh -c 'watch -n30 "/system-share/transcoding-temp-fix.sh 2>&1" > /transcode/transcoding-temp-fix.log &'
    fi
  else
    echo "ERROR: Command to detect script run status failed"
    /usr/local/emhttp/webGui/scripts/notify -e "emby-ClearTranscodingTmp" -s "Command to detect script status failed" -d "" -i "alert"
  fi
fi

 

Monitor the script's activity:

tail -f /dev/shm/transcoding-temp-fix.log

 

Sample output:

Every 30.0s: /system-share/transcoding-temp-fix.sh 2>&1     2022-10-10 14:45:19

Directory size: 	   5%
Directory limit:	   50%

 

NOTES:

• Script can probably be tweaked to work for Plex
• If a better solution exists let me know, this was quick and dirty.

Edited October 10, 20223 yr by CS01-HS

Hi @CS01-HS, thanks for this script.

 

Have you found a better solution or are you still using it?

On 10/4/2022 at 1:01 AM, Andrea3000 said:

Hi @CS01-HS, thanks for this script.

 

Have you found a better solution or are you still using it?

 

Still using them.

I haven't found a better solution but I haven't looked, this "just works."

It's possible Emby fixed their garbage collection problem so it's no longer necessary.

 

I've tweaked the scripts slightly for better error-handling (initial post edited with updated scripts)

Edited October 10, 20223 yr by CS01-HS

Thank you very much, I tried the script and it works great.

I had to make only a small adjustment because Emby on my system is creating a random alphanumeric subfolder inside transcoding-temp (one for every movie) where it stores the segments. 

You're right, mine too. They must have changed  it.

(My punishment for saying "just works")

 

I've replaced the simple ls:

ls ${TRANSCODE_DIR}/*.ts -1t | tail -${BATCH_SIZE} | xargs rm

 

with a find:

find ${TRANSCODE_DIR} -type f -name "*.ts" -exec ls -1t "{}" + | tail -${BATCH_SIZE} | xargs rm

to work around it.

 

I'll update my initial post unless you have something cleaner?

Initial post updated with fix and various improvements.

I have replaced this

ls ${TRANSCODE_DIR}/*.ts -1t | tail -${BATCH_SIZE} | xargs rm

with this

ls ${TRANSCODE_DIR}/*/*.ts -1t | tail -${BATCH_SIZE} | xargs rm

and it appears to be working fine.

 

Thanks for the update

That definitely works as long as they don't decide to change the directory structure again.

The find should be structure-agnostic (unless I screwed up.)

On 10/11/2022 at 6:03 AM, CS01-HS said:

That definitely works as long as they don't decide to change the directory structure again.

The find should be structure-agnostic (unless I screwed up.)

Have you tested this with live tv? Mine seems to transcode tv into one massive file, Ive had sporting events become 140gb on my ssd.

17 hours ago, lukeoslavia said:

Have you tested this with live tv? Mine seems to transcode tv into one massive file, Ive had sporting events become 140gb on my ssd.

 

No sorry, I don't use live TV so I've never tested it.

I'm guessing they do that for DVR functionality (serves as a reference file which in the standard case is the media itself.)

 

In the best case my script won't have any effect, in the worst case it'll delete it which could break things.

If active live TV files can be differentiated by name the script could be tweaked to ignore them, or if not we can infer by size (greater than X.)

 

What are you trying to solve exactly, because my guess is creation of these large files with live TV is unavoidable.

11 hours ago, CS01-HS said:

 

No sorry, I don't use live TV so I've never tested it.

I'm guessing they do that for DVR functionality (serves as a reference file which in the standard case is the media itself.)

 

In the best case my script won't have any effect, in the worst case it'll delete it which could break things.

If active live TV files can be differentiated by name the script could be tweaked to ignore them, or if not we can infer by size (greater than X.)

 

What are you trying to solve exactly, because my guess is creation of these large files with live TV is unavoidable.

I would love to have these large files eliminated so I can use ram as the transcode disk, but like you said, they seem to be intentionally created. I requested a feature to turn off "time-shift" on their forums to which they replied, we have part of the functionality now, and then closed it lol.

 

My main problem with the large files is that they never get deleted unless I restart the emby server (even when properly closed), which doesn't happen in windows.

In Live TV -> Advanced I see Default recording path

Does setting that change where emby stores tmp live TV files?

 

2 hours ago, lukeoslavia said:

My main problem with the large files is that they never get deleted unless I restart the emby server

 

Well that's easy enough to fix even with an unraid user script.

Something like this that runs every hour, deleting ts files more than an hour old:

find /host/path/to/transcode/dir/ -type f -name '*.ts' -mindepth 1 -mmin +60 -delete

 

You could even restrict it to files above a certain size with -size +100M

Edited October 23, 20223 yr by CS01-HS

On 10/22/2022 at 9:04 PM, CS01-HS said:

In Live TV -> Advanced I see Default recording path

Does setting that change where emby stores tmp live TV files?

 

it does not seem to, I have paths set for movies and tv separately, but the transcoding files are on a separate ssd.

 

On 10/22/2022 at 9:04 PM, CS01-HS said:

Well that's easy enough to fix even with an unraid user script.

Something like this that runs every hour, deleting ts files more than an hour old:

find /host/path/to/transcode/dir/ -type f -name '*.ts' -mindepth 1 -mmin +60 -delete

 

You could even restrict it to files above a certain size with -size +100M

 

I honestly never think to use a script for pretty much anything, but I will give this a shot. I will probably have to modify it a bit to accommodate sporting events.

Thanks for the tip!