CS01-HS Posted January 30, 2022 Share Posted January 30, 2022 (edited) I have emby transcode to RAM (/dev/shm/) which works well except for garbage collection – temporary transcode files accumulate. To solve that I wrote a script (run by the container) to delete temporary files and a user-script to launch it when the container's restarted or if it's not running for any other reason. Assumptions: /transcode is the container's path to /dev/shm/ (or wherever you're transcoding to) /transcode/transcoding-temp is the container's path to the directory holding temp transcoding files (emby creates this subdirectory) /system-share/transcoding-temp-fix.sh is the container's path to the following script (make the script executable) transcoding-temp-fix.sh #!/bin/sh TRANSCODE_DIR="/transcode/transcoding-temp" # Delete old files when used space is above this % PERCENT_LIMIT=50 # Delete this many files at a time BATCH_SIZE=10 if [ -d "${TRANSCODE_DIR}" ]; then percent_full=$(df "${TRANSCODE_DIR}" | awk '{print $5}' | tail -1 | tr -dc '0-9') printf "Directory size: \t %3s%%\n" ${percent_full} printf "Directory limit:\t %3s%%\n" ${PERCENT_LIMIT} echo "" while [ $percent_full -gt $PERCENT_LIMIT ]; do if [ $(find ${TRANSCODE_DIR} -type f -name "*.ts" | wc -l) -gt 0 ]; then echo "(${percent_full}%) exceeds limit (${PERCENT_LIMIT}%), deleting oldest (${BATCH_SIZE}) files" find ${TRANSCODE_DIR} -type f -name "*.ts" -exec ls -1t "{}" + | tail -${BATCH_SIZE} | xargs rm else echo "*WARNING* (${percent_full}%) exceeds limit (${PERCENT_LIMIT}%) but files are not transcoding fragments" exit 1 fi percent_full=$(df "${TRANSCODE_DIR}" | awk '{print $5}' | tail -1 | tr -dc '0-9') done else echo "${TRANSCODE_DIR} (TRANSCODE_DIR): directory doesn't exist" fi Now the user script to launch it, set to run every 10 minutes (*/10 * * * *) NOTE: Update EmbyServer name and system-share (if necessary) to match your system #!/bin/bash #arrayStarted=true #clearLog=true # Verify EmbyServer's running running=$(docker container ls | grep EmbyServer | wc -l) if [ "${running}" != "0" ]; then # verify watch command that calls clearing script is running watch_running=$(docker exec -i EmbyServer ps | grep 'watch ' | wc -l) # make sure the detection command ran properly otherwise # we might end up running multiple instances of the script if [ $? -eq 0 ]; then if [ "${watch_running}" == "0" ]; then echo "Clearing script is not running. Re-starting..." docker exec EmbyServer sh -c 'watch -n30 "/system-share/transcoding-temp-fix.sh 2>&1" > /transcode/transcoding-temp-fix.log &' fi else echo "ERROR: Command to detect script run status failed" /usr/local/emhttp/webGui/scripts/notify -e "emby-ClearTranscodingTmp" -s "Command to detect script status failed" -d "" -i "alert" fi fi Monitor the script's activity: tail -f /dev/shm/transcoding-temp-fix.log Sample output: Every 30.0s: /system-share/transcoding-temp-fix.sh 2>&1 2022-10-10 14:45:19 Directory size: 5% Directory limit: 50% NOTES: Script can probably be tweaked to work for Plex If a better solution exists let me know, this was quick and dirty. Edited October 10, 2022 by CS01-HS Quote Link to comment
Andrea3000 Posted October 4, 2022 Share Posted October 4, 2022 Hi @CS01-HS, thanks for this script. Have you found a better solution or are you still using it? Quote Link to comment
CS01-HS Posted October 6, 2022 Author Share Posted October 6, 2022 (edited) On 10/4/2022 at 1:01 AM, Andrea3000 said: Hi @CS01-HS, thanks for this script. Have you found a better solution or are you still using it? Still using them. I haven't found a better solution but I haven't looked, this "just works." It's possible Emby fixed their garbage collection problem so it's no longer necessary. I've tweaked the scripts slightly for better error-handling (initial post edited with updated scripts) Edited October 10, 2022 by CS01-HS Quote Link to comment
Andrea3000 Posted October 9, 2022 Share Posted October 9, 2022 Thank you very much, I tried the script and it works great. I had to make only a small adjustment because Emby on my system is creating a random alphanumeric subfolder inside transcoding-temp (one for every movie) where it stores the segments. 1 Quote Link to comment
CS01-HS Posted October 10, 2022 Author Share Posted October 10, 2022 You're right, mine too. They must have changed it. (My punishment for saying "just works") I've replaced the simple ls: ls ${TRANSCODE_DIR}/*.ts -1t | tail -${BATCH_SIZE} | xargs rm with a find: find ${TRANSCODE_DIR} -type f -name "*.ts" -exec ls -1t "{}" + | tail -${BATCH_SIZE} | xargs rm to work around it. I'll update my initial post unless you have something cleaner? Quote Link to comment
CS01-HS Posted October 10, 2022 Author Share Posted October 10, 2022 Initial post updated with fix and various improvements. Quote Link to comment
Andrea3000 Posted October 10, 2022 Share Posted October 10, 2022 I have replaced this ls ${TRANSCODE_DIR}/*.ts -1t | tail -${BATCH_SIZE} | xargs rm with this ls ${TRANSCODE_DIR}/*/*.ts -1t | tail -${BATCH_SIZE} | xargs rm and it appears to be working fine. Thanks for the update Quote Link to comment
CS01-HS Posted October 11, 2022 Author Share Posted October 11, 2022 That definitely works as long as they don't decide to change the directory structure again. The find should be structure-agnostic (unless I screwed up.) 1 Quote Link to comment
lukeoslavia Posted October 21, 2022 Share Posted October 21, 2022 On 10/11/2022 at 6:03 AM, CS01-HS said: That definitely works as long as they don't decide to change the directory structure again. The find should be structure-agnostic (unless I screwed up.) Have you tested this with live tv? Mine seems to transcode tv into one massive file, Ive had sporting events become 140gb on my ssd. Quote Link to comment
CS01-HS Posted October 22, 2022 Author Share Posted October 22, 2022 17 hours ago, lukeoslavia said: Have you tested this with live tv? Mine seems to transcode tv into one massive file, Ive had sporting events become 140gb on my ssd. No sorry, I don't use live TV so I've never tested it. I'm guessing they do that for DVR functionality (serves as a reference file which in the standard case is the media itself.) In the best case my script won't have any effect, in the worst case it'll delete it which could break things. If active live TV files can be differentiated by name the script could be tweaked to ignore them, or if not we can infer by size (greater than X.) What are you trying to solve exactly, because my guess is creation of these large files with live TV is unavoidable. Quote Link to comment
lukeoslavia Posted October 22, 2022 Share Posted October 22, 2022 11 hours ago, CS01-HS said: No sorry, I don't use live TV so I've never tested it. I'm guessing they do that for DVR functionality (serves as a reference file which in the standard case is the media itself.) In the best case my script won't have any effect, in the worst case it'll delete it which could break things. If active live TV files can be differentiated by name the script could be tweaked to ignore them, or if not we can infer by size (greater than X.) What are you trying to solve exactly, because my guess is creation of these large files with live TV is unavoidable. I would love to have these large files eliminated so I can use ram as the transcode disk, but like you said, they seem to be intentionally created. I requested a feature to turn off "time-shift" on their forums to which they replied, we have part of the functionality now, and then closed it lol. My main problem with the large files is that they never get deleted unless I restart the emby server (even when properly closed), which doesn't happen in windows. Quote Link to comment
CS01-HS Posted October 23, 2022 Author Share Posted October 23, 2022 (edited) In Live TV -> Advanced I see Default recording path Does setting that change where emby stores tmp live TV files? 2 hours ago, lukeoslavia said: My main problem with the large files is that they never get deleted unless I restart the emby server Well that's easy enough to fix even with an unraid user script. Something like this that runs every hour, deleting ts files more than an hour old: find /host/path/to/transcode/dir/ -type f -name '*.ts' -mindepth 1 -mmin +60 -delete You could even restrict it to files above a certain size with -size +100M Edited October 23, 2022 by CS01-HS Quote Link to comment
lukeoslavia Posted October 24, 2022 Share Posted October 24, 2022 On 10/22/2022 at 9:04 PM, CS01-HS said: In Live TV -> Advanced I see Default recording path Does setting that change where emby stores tmp live TV files? it does not seem to, I have paths set for movies and tv separately, but the transcoding files are on a separate ssd. On 10/22/2022 at 9:04 PM, CS01-HS said: Well that's easy enough to fix even with an unraid user script. Something like this that runs every hour, deleting ts files more than an hour old: find /host/path/to/transcode/dir/ -type f -name '*.ts' -mindepth 1 -mmin +60 -delete You could even restrict it to files above a certain size with -size +100M I honestly never think to use a script for pretty much anything, but I will give this a shot. I will probably have to modify it a bit to accommodate sporting events. Thanks for the tip! 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.