[Support] Djoss - HandBrake

[cbr600ds2]

this is the last few lines that its run - it just doesn't seem like it's running.  The watch folder and the output folder are shares on my cache that is an SSD due to the writes.   

 

 

The VNC desktop is: 56341221b657:0

0

******************************************************************************
Have you tried the x11vnc '-ncache' VNC client-side pixel caching feature yet?

The scheme stores pixel data offscreen on the VNC viewer side for faster
retrieval. It should work with any VNC viewer. Try it by running:

x11vnc -ncache 10 ...

One can also add -ncache_cr for smooth 'copyrect' window motion.
More info: http://www.karlrunge.com/x11vnc/faq.html#faq-client-caching

01/07/2022 19:53:13 Got connection from client 127.0.0.1
01/07/2022 19:53:13 other clients:
01/07/2022 19:53:13 Got 'ws' WebSockets handshake
01/07/2022 19:53:13 Got protocol: binary
01/07/2022 19:53:13 - webSocketsHandshake: using binary/raw encoding
01/07/2022 19:53:13 - WebSockets client version hybi-13
01/07/2022 19:53:13 Disabled X server key autorepeat.
01/07/2022 19:53:13 to force back on run: 'xset r on' (3 times)
01/07/2022 19:53:13 incr accepted_client=1 for 127.0.0.1:49006 sock=10
01/07/2022 19:53:13 Client Protocol Version 3.8
01/07/2022 19:53:13 Protocol version sent 3.8, using 3.8
01/07/2022 19:53:13 rfbProcessClientSecurityType: executing handler for type 1
01/07/2022 19:53:13 rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8
01/07/2022 19:53:13 Pixel format for client 127.0.0.1:
01/07/2022 19:53:13 32 bpp, depth 24, little endian
01/07/2022 19:53:13 true colour: max r 255 g 255 b 255, shift r 16 g 8 b 0
01/07/2022 19:53:13 no translation needed
01/07/2022 19:53:13 Enabling New) 

 

 

[Djoss]

By default it should run every 5 seconds, but maybe the interval has been changed.  You can check its value by looking at the "Automatic Video Converter: Check Interval" container setting.

[cbr600ds2]

5 minutes ago, Djoss said:

By default it should run every 5 seconds, but maybe the interval has been changed.  You can check its value by looking at the "Automatic Video Converter: Check Interval" container setting.

 

yeah its at 5 seconds but its not running.  Is there a way to pull my preset and save and delete the docker and reinstall?  Do you think that's a viable option

[Djoss]

2 minutes ago, cbr600ds2 said:

yeah its at 5 seconds but its not running.  Is there a way to pull my preset and save and delete the docker and reinstall?  Do you think that's a viable option

 

If you prefer it's probably better to revert to the previous version, by changing the "Repository" in settings to:

jlesage/handbrake:v1.24.2

 

But before doing so, can you please share the full log by running:

docker logs HandBrake

 

[cbr600ds2]

6 minutes ago, Djoss said:

 

If you prefer it's probably better to revert to the previous version, by changing the "Repository" in settings to:

jlesage/handbrake:v1.24.2

 

But before doing so, can you please share the full log by running:

docker logs HandBrake

 

 

https://pastebin.com/p4Zega8d

 

Let me know if you want me to try anything else or if you think I'm ok to try rolling it back.  I'll wait for your go ahead before doing anything.  

[Djoss]

Can you try to "touch" any file in the watch folder, or add a new one ?  This should kick the automatic video converter again.

 

BTW, the log from the web interface is incomplete.  You should run the "docker logs HandBrake" command to get all its content.

[cbr600ds2]

12 minutes ago, Djoss said:

Can you try to "touch" any file in the watch folder, or add a new one ?  This should kick the automatic video converter again.

 

BTW, the log from the web interface is incomplete.  You should run the "docker logs HandBrake" command to get all its content.

 

Changed one of the file names to try to kick it off.  it ran but it touched all of the files but gave the same error -

 

[autovideoconverter] Skipping '/watch/Better Nate Than Ever.mkv': not readable, check permissions

 

I've ran the docker command and included it here as a notepad file to show you that's what it gave me.  What should I be looking for? 

HandBrake logs.txt

[supawiz6991]

3 hours ago, cbr600ds2 said:

 

Changed one of the file names to try to kick it off.  it ran but it touched all of the files but gave the same error -

 

[autovideoconverter] Skipping '/watch/Better Nate Than Ever.mkv': not readable, check permissions

 

I've ran the docker command and included it here as a notepad file to show you that's what it gave me.  What should I be looking for? 

HandBrake logs.txt 33.79 kB · 0 downloads

Where is your Watch Directory docker setting pointing? What are the permissions for that directory set to? (if the directory is /mnt/user/hb then do ls -l /mnt/user and see what the permissions of the hb folder are. You should see something like   drwxrwxrwx 1 nobody users.

@Djoss once you have cbr600ds2, could you have a look into the issue I posted? The problem is still occurring and is slowing my progress through my collection, requiring a lot of re-encoding.  

 

[cbr600ds2]

11 hours ago, supawiz6991 said:

Where is your Watch Directory docker setting pointing? What are the permissions for that directory set to? (if the directory is /mnt/user/hb then do ls -l /mnt/user and see what the permissions of the hb folder are. You should see something like   drwxrwxrwx 1 nobody users.

@Djoss once you have cbr600ds2, could you have a look into the issue I posted? The problem is still occurring and is slowing my progress through my collection, requiring a lot of re-encoding.  

 

 

root@Skynet:~# ls -l /mnt/cache/TBC
total 86988276
-rw-rw-rw- 1 nobody users  2977871165 Mar  2 05:51 Against\ the\ Ice\ (2022).mkv
-rw-rw-rw- 1 nobody users  1932678059 Apr  5 14:40 Alice\ (2022).mkv
-rw-rw-rw- 1 nobody users  2253014544 Apr  8 18:24 All\ the\ Old\ Knives\ (2022).mp4
-rw-rw-rw- 1 nobody users  1684117449 Mar  9 23:36 Another\ Round\ (2020).mp4
-rw-rw-rw- 1 nobody users      107885 Jul  1 13:46 Another\ Round\ (2020).srt
-rw-rw-rw- 1 nobody users  1917646954 Apr  1 18:47 Apollo\ 10½\ A\ Space\ Age\ Childhood\ (2022).mkv
-rw-rw-rw- 1 desi   users  4103239659 Jun 27 15:21 Beautiful.Vampire.2018.1080p.AMZN.WEB-DL.DDP5.1.H.264-NTG.mkv
-rw-rw-rw- 1 nobody users  2009683618 Jun 23 19:36 Beavis\ and\ Butt-Head\ Do\ the\ Universe\ (2022).mp4
-rw-rw-rw- 1 nobody users  1774720019 Apr  1 18:46 Better\ Nate\ Than\ Ever.mkv
-rw-rw-rw- 1 nobody users  2030725176 Apr 17 00:44 Blacklight\ (2022).mkv
-rw-rw-rw- 1 nobody users  1856830754 Apr  1 18:52 Boon\ (2022).mkv
-rw-rw-rw- 1 nobody users  4702428798 Apr  5 06:12 Bull\ (2021).mkv
-rw-rw-rw- 1 nobody users  1798819622 Mar 31 20:38 Every\ Last\ Secret\ (2022).mkv
-rw-rw-rw- 1 nobody users  3096188723 Dec  7  2018 Hospitality\ (2018).mkv
-rw-rw-rw- 1 nobody users  1947161591 Apr  7 15:17 Let\ the\ Wrong\ One\ In\ (2021).mkv
-rw-rw-rw- 1 desi   users   774035374 Jun  1 23:28 Monstrous.S01E01.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 desi   users   638529908 Jun  1 23:28 Monstrous.S01E02.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 desi   users   665458171 Jun  1 23:29 Monstrous.S01E03.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 desi   users   638701094 Jun  1 23:28 Monstrous.S01E04.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 desi   users   616754486 Jun  1 23:28 Monstrous.S01E05.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 desi   users   671179719 Jun  1 23:28 Monstrous.S01E06.1080p.VIU.WEB-DL.AAC.H.264-iTsOK.mkv
-rw-rw-rw- 1 nobody users  2020270051 Mar 31 16:46 Moonshot\ (2022).mkv
-rw-rw-rw- 1 nobody users  2158237752 Jun 24 15:11 Rise\ (2022).mkv
-rw-rw-rw- 1 nobody users  7041924423 Mar  7  2018 Scorched\ Earth\ (2018).mkv
-rw-rw-rw- 1 nobody users  2434949532 Jun 29 08:32 Superman\ &\ Lois\ S02E15\ Waiting\ for\ Superman.mkv
-rw-rw-rw- 1 nobody users  3064887704 Jul  1 06:50 Terror\ on\ the\ Prairie\ (2022).mkv
-rw-rw-rw- 1 nobody users 10884860098 Jun 25 05:11 The\ Cellar\ (2022).mkv
-rw-rw-rw- 1 nobody users  2004281648 May 27 19:56 The\ Contractor\ (2022).mkv
-rw-rw-rw- 1 nobody users  1835233823 Jun 22 00:00 The\ Nan\ Movie\ (2022).mkv
-rw-rw-rw- 1 nobody users  1618415294 May 27 03:36 Umma\ (2022).mkv
-rw-rw-rw- 1 nobody users 12989743572 Apr 21 08:01 V\ H\ S\ 94\ (2021).mkv
-rw-rw-rw- 1 nobody users  2092243792 Apr  6 21:08 Wildhood\ (2022).mkv
-rw-rw-rw- 1 nobody users  2840989490 Jun 23 11:17 flash\ 8x19.mkv

 

I'm going to try rolling it back

 

[cbr600ds2]

Weird - the rollback immediate kicked off auto converting.  *shrug*   

 

Thanks @Djoss and @supawiz6991.

 

Sorry @supawiz6991 for cutting the line. 

Edited July 2, 2022 by cbr600ds2

[supawiz6991]

55 minutes ago, cbr600ds2 said:

Weird - the rollback immediate kicked off auto converting.  *shrug*   

 

Thanks @Djoss and @supawiz6991.

 

Sorry @supawiz6991 for cutting the line. 

No worries lol

Weird, since the directory has read and write permissions. I would imagine the docker container either didn't install correctly or something went wrong with it. I'm running the latest version and didn't get that error. I'm wondering if using /mnt/cache has anything to do with it since by default the docker wants to use /mnt/user. It shouldn't since permissions are correct but then again who knows.

I would definitely investigate further if you get a chance since staying rolled back isn't a permanent solution.

Idea! Install a second instance of handbrake docker and let it be at the latest version, duplicate your docker settings and see if the issue returns. If so, post your docker settings here and I'll setup a test instance and mimic your settings and see if I can reproduce the problem.

[cbr600ds2]

2 hours ago, supawiz6991 said:

Idea! Install a second instance of handbrake docker and let it be at the latest version, duplicate your docker settings and see if the issue returns. If so, post your docker settings here and I'll setup a test instance and mimic your settings and see if I can reproduce the problem.

 

if there's a second instance though and its pointing to the same folders in theory could that cause big issues if both work or...not really and I'm overthinking it hahaha

[supawiz6991]

23 minutes ago, cbr600ds2 said:

 

if there's a second instance though and its pointing to the same folders in theory could that cause big issues if both work or...not really and I'm overthinking it hahaha

So, possibly. They both could try to access and encode the same video at the same time but the worst that would happen is one would fail with a read error.

I have 4 instances on my unraid box. 2 for HD  and 2 for 4K and split the folders like so:

/mnt/user/MKHB/watch_hd_1
/mnt/user/MKHB/watch_hd_2
/mnt/user/MKHB/watch_uhd_1
/mnt/user/MKHB/watch_uhd_2

When you go to do the test if you stop the primary docker and only have the test one running you should be fine.

P.S. I only run two of those at a time and each is pinned to half the CPU so they won't ever step on each other.

Edited July 2, 2022 by supawiz6991

[cbr600ds2]

2 hours ago, supawiz6991 said:

So, possibly. They both could try to access and encode the same video at the same time but the worst that would happen is one would fail with a read error.

I have 4 instances on my unraid box. 2 for HD  and 2 for 4K and split the folders like so:

/mnt/user/MKHB/watch_hd_1
/mnt/user/MKHB/watch_hd_2
/mnt/user/MKHB/watch_uhd_1
/mnt/user/MKHB/watch_uhd_2

When you go to do the test if you stop the primary docker and only have the test one running you should be fine.

P.S. I only run two of those at a time and each is pinned to half the CPU so they won't ever step on each other.

Oh ok.  Gotcha .  Will try this week and get back with everyone

 

[danmartyn]

HI All,

 

I'm trying to setup a post_conversion script that moves converted files from the output directory to my array. I have it working to figure out if it's a TV show or Movie and what the destination path should be, but when I hit my mkdir command in the script, I get the following error in the log: mkdir: cannot create directory ‘/mnt/user/Media/TV Shows/Show/Season 1/’: Permission denied

 

This is the script (I left off the top part with the comment and variables being set from the example):

if [ "$CONVERSION_STATUS" -eq 0 ]; then #https://stackoverflow.com/a/24900088
	# Successful conversion.
	FILE="${CONVERTED_FILE##*/}"
	echo "✅  [$FILE]"
	
	# Check for show or movie
	ISTVSHOW=$(echo "$FILE" | egrep '(S[0-9]+E[0-9]+)')
	if [ -z "$ISTVSHOW" ]; then
		# MOVIE
		DESTINATION="/mnt/user/Media/Movies"
	else
		# Get Show Name
		REMOVE=$(echo "$FILE" | egrep -o ' - S[0-9]+E[0-9]+.*$')
		LENGTH=${#REMOVE}
		SHOW=$(echo "$FILE" | rev | cut -c $(($LENGTH+1))- | rev)
		
		# Get Season Number
		CLEANED="${REMOVE:4}"
		SEASON=$((10#$(echo "$CLEANED" | egrep -o '^[0-9]+')))
		DESTINATION="/mnt/user/Media/TV Shows/$SHOW/Season $SEASON"
	fi
	
	# Move the file to the destination, making any directories as needed
	echo "➡️  $DESTINATION/$FILE"
	mkdir -p "$DESTINATION/" && mv "$FILE" "$DESTINATION"
else
	# Failed conversion.
	echo "❌  $SOURCE_FILE"
fi

 

I'm hoping it's just a simple thing I don't know about to have the right permissions. Any help appreciated!

[supawiz6991]

4 hours ago, danmartyn said:

HI All,

 

I'm trying to setup a post_conversion script that moves converted files from the output directory to my array. I have it working to figure out if it's a TV show or Movie and what the destination path should be, but when I hit my mkdir command in the script, I get the following error in the log: mkdir: cannot create directory ‘/mnt/user/Media/TV Shows/Show/Season 1/’: Permission denied

 

This is the script (I left off the top part with the comment and variables being set from the example):

if [ "$CONVERSION_STATUS" -eq 0 ]; then #https://stackoverflow.com/a/24900088
	# Successful conversion.
	FILE="${CONVERTED_FILE##*/}"
	echo "✅  [$FILE]"
	
	# Check for show or movie
	ISTVSHOW=$(echo "$FILE" | egrep '(S[0-9]+E[0-9]+)')
	if [ -z "$ISTVSHOW" ]; then
		# MOVIE
		DESTINATION="/mnt/user/Media/Movies"
	else
		# Get Show Name
		REMOVE=$(echo "$FILE" | egrep -o ' - S[0-9]+E[0-9]+.*$')
		LENGTH=${#REMOVE}
		SHOW=$(echo "$FILE" | rev | cut -c $(($LENGTH+1))- | rev)
		
		# Get Season Number
		CLEANED="${REMOVE:4}"
		SEASON=$((10#$(echo "$CLEANED" | egrep -o '^[0-9]+')))
		DESTINATION="/mnt/user/Media/TV Shows/$SHOW/Season $SEASON"
	fi
	
	# Move the file to the destination, making any directories as needed
	echo "➡️  $DESTINATION/$FILE"
	mkdir -p "$DESTINATION/" && mv "$FILE" "$DESTINATION"
else
	# Failed conversion.
	echo "❌  $SOURCE_FILE"
fi

 

I'm hoping it's just a simple thing I don't know about to have the right permissions. Any help appreciated!

So a couple of things to check:
1. What user is the script running as?
2. What is the permissions for the destination directory? Do two checks here: ls -l "/mnt/user/media/  and then ls -l /mnt/user/ and make sure the user the script is running as has read/write permissions to media, TV Shows and Movies directories. Here is what my tv_shows directory looks like:
drwxrwxrwx 1 nobody users   299 Apr 16  2021 TV_Shows/

at a minimum it should be this and anything less or the script user not being in the users group and/or root will fail:

_rw_rw_rw_ 1 nobody users   299 Apr 16  2021 TV_Shows/

2b. Be careful allowing spaces in file/directory names in linux as it can cause headaches if you don't put them in commands/scripts properly. For example, in the ls -l command I posted, you'll notice I surrounded the path with "" as if I hadn't the ls command would treat tv as one directory and shows as another.  I think you are ok in your case as the error is showing the correct path.

3. Does the entire /mnt/user/media/TV Shows/ directory already exist?

[danmartyn]

4 hours ago, supawiz6991 said:

So a couple of things to check:
1. What user is the script running as?
2. What is the permissions for the destination directory? Do two checks here: ls -l "/mnt/user/media/  and then ls -l /mnt/user/ and make sure the user the script is running as has read/write permissions to media, TV Shows and Movies directories. Here is what my tv_shows directory looks like:
drwxrwxrwx 1 nobody users   299 Apr 16  2021 TV_Shows/

at a minimum it should be this and anything less or the script user not being in the users group and/or root will fail:

_rw_rw_rw_ 1 nobody users   299 Apr 16  2021 TV_Shows/

2b. Be careful allowing spaces in file/directory names in linux as it can cause headaches if you don't put them in commands/scripts properly. For example, in the ls -l command I posted, you'll notice I surrounded the path with "" as if I hadn't the ls command would treat tv as one directory and shows as another.  I think you are ok in your case as the error is showing the correct path.

3. Does the entire /mnt/user/media/TV Shows/ directory already exist?

I'm not super familiar with scripting, and sort of going off what I can find from Google, so I'm not sure how to find what user the script is running as. But here's the info from what you asked:

 

ls -l /mnt/user/:
drwxrwxrwx 1 nobody users 117 Nov 17  2019 Media/
drwxrwxrwx 1 nobody users 111 Jul 10 15:44 appdata/
drwxrwxrwx 1 nobody users   6 Jun 18 14:37 domains/
drwxrwxrwx 1 nobody users   6 Feb 22  2017 isos/
drwxrwxrwx 1 nobody users  35 Feb 22  2017 system/

 

ls -l /mnt/user/Media/:
drwxrwxrwx  1 nobody users    27 Apr 22 23:32 Kids\ Shows/
drwxr-sr-x  1 nobody users 28672 Jul 10 23:03 Movies/
drwxrwxr-x+ 1 nobody users   167 Jul 10  2021 Plex\ Posters/
drwxr-sr-x  1 nobody users 16384 Jun 24 23:34 TV\ Shows/

 

The directories all existed previously, although the Kids Shows is recent and I forget if I created it from my Mac or Krusader. The other ones were created like 5+ years ago. I've been manually moving things around in Krusader and relying on Hazel running on my Mac to move things, so was just trying to automate that so it can all happen on the server side. Looking at the Movies and TV Show directories, it looks like they don't have write permissions for not the owner, which I assume is what's causing my problem?

 

Do I just need to use chmod in the terminal on /mnt/user/Media/ to change the permissions, or do I need to do it for the Movies and TV Show directories directly?

 

Thanks for the help!

[alturismo]

@danmartyn may ahead, where are you running this script from

 

i dont use handbrake etc anymore, but when you run this script from inside this docker, you have to look at your path's from docker point of view and not from host point of view

 

lets say /output <> /mnt/user/

 

so something inside the docker is located at /output/Media/TVRIPS/blabla/saeson,,, and not /mnt/user/....

 

just in case your script is triggered inside the docker ... if it runs on the host by userscripts as sample, then forget what i wrote

[danmartyn]

I am running the script from inside the docker. Do I need to do something special to give it access to the array to be able to move the files for me?

[Djoss]

On 7/2/2022 at 2:30 AM, supawiz6991 said:

@Djoss once you have cbr600ds2, could you have a look into the issue I posted? The problem is still occurring and is slowing my progress through my collection, requiring a lot of re-encoding.  

 

You may want to try the debug version to see if more information can be retrieved from the conversion log.  Just use the tag "v1.25.1-debug".

[Djoss]

3 hours ago, danmartyn said:

I am running the script from inside the docker. Do I need to do something special to give it access to the array to be able to move the files for me?

 

Are you using default container settings?  Usually, the path /mnt/user/... doesn't exist inside the container.  By default, /mnt/user is mapped to /storage inside the container.  So you will find your Media directory at /storage/Media in the container.

 

Also, by default /storage is read-only.  You have to edit the container setting to change it read/write (Advanced view must be enabled for that).

[danmartyn]

On 7/11/2022 at 11:28 AM, Djoss said:

 

Are you using default container settings?  Usually, the path /mnt/user/... doesn't exist inside the container.  By default, /mnt/user is mapped to /storage inside the container.  So you will find your Media directory at /storage/Media in the container.

 

Also, by default /storage is read-only.  You have to edit the container setting to change it read/write (Advanced view must be enabled for that).

I'm overriding the Storage to pass in the Handbrake directory, which is on an nvme drive that isn't my cache drive, or part of the array that I use just for Handbrake. Is that going to be an issue for me?

 

[alturismo]

1 hour ago, danmartyn said:

Is that going to be an issue for me?

nope, just consider the docker point of view while using post processing scripts

[supawiz6991]

On 7/10/2022 at 11:25 PM, danmartyn said:

drwxr-sr-x  1 nobody users 28672 Jul 10 23:03 Movies/
drwxrwxr-x+ 1 nobody users   167 Jul 10  2021 Plex\ Posters/
drwxr-sr-x  1 nobody users 16384 Jun 24 23:34 TV\ Shows/

So there is a problem right here.

First, linux permissions are broken in to User, Group, Other so looking at movies:

d rwx r-s r-x

The d just means directory. 
First set: rwx signifies that the owner (nobody) has read, write and execute permisson.
Second set: r-s means the group (users) has read and the s means setuid which allows a script to be run with the owners or their groups permissions. Not sure why that is set that way...I've not had that happen on my setup.
Third set: r-x the other group has read and execute permissions.

Just a quick break down in case you are not familiar with linux permissions.

Second:
Neither the group (users) nor other have read permissions to those directories. From what I understand of s/setuid its possible to obtain those read permissions though that but that may not be working in this case.

doing chmod 755 Movies  may fix the problem BUT I strongly suggest investigating why s / setuid is there first to prevent breaking something...or you can perform what we in IT call the scream test.  Up to you.

As far as your script is concerned...if its running from inside a docker container then it will be run as that containers user. If its run outside then it may be root or hopefully a less privileged user.

P.S. Can anyone tell me why unraid likes to default to 777 permissions for user accessible directories instead of 770 or better yet 750? 

[danmartyn]

Thanks for the help everyone! I ended up needing to change my Docker settings to point at /mnt so that Handbrake could see my name drive for files to convert and my script could see /mnt/user/Media to move things. I also had to change the permissions on the directories as was mentioned. Thanks again for the help.