[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

Happy to see the project is out of Alpha and into Beta.

I just got the Docker image up and running a few hours ago and have been kicking the wheels around.

Kudos @Josh.5 for getting this project up and running! 🙇‍♂️ Let us know how we can support you and in what capacity.

 

Things I'd like to see:

  1. More finite control over video quality
  2. Define how audio will be encoded or passed through or both
  3. Flag folders to exclude/skip
    1. e.g., nightly/weekly news broadcast or variety show that are automatically deleted after viewing don't need to be encoded in h.265 (saves time & energy)
  4. Cancel current workers/encodes (See section 3.1 above) 😉
  5. Ignore "._" files
Edited by Zer0Nin3r
Link to comment
  • Was doing some more testing. I've noticed that a few TV shows weren't encoded due to some sort of errors similar listed below in this post
  • Unmanic is able to watch two folders/libraries which is good
  • Tried importing a movie this afternoon and found errors in my Docker logs
  • Zero reports inside the Unmanic GUI's "Failure" tab
2019-07-05T17:12:51:ERROR:Unmanic.Worker-1 - [FORMATTED] - Exception in processing job with Worker-1: - ['ffmpeg', '-y', '-i', '/library/Movies/ (2014)/ (2014) [-1080p].mkv', '-hide_banner', '-loglevel', 'info', '-strict', '-2', '-max_muxing_queue_size', '512', '-map', '0:0', '-map', '0:1', '-map', ' 0:1', '-map', '0:2', '-map', ' 0:2', '-map', '0:3', '-map', ' 0:3', '-map', '0:4', '-map', ' 0:4', '-map', '0:5', '-map', ' 0:5', '-map', '0:6', '-map', ' 0:6', '-map', '0:7', '-map', ' 0:7', '-map', '0:8', '-map', ' 0:8', '-map', ' 0:9', '-map', '0:10', '-map', '0:11', '-map', '0:12', '-map', '0:13', '-map', '0:14', '-map', '0:15', '-map', '0:16', '-map', '0:17', '-map', '0:18', '-map', '0:19', '-c:v', 'libx265', '-c:a:0', 'copy', '-c:a:1', 'aac', '-b:a:1', '128k', '-ac', '2', '-metadata:s:a:1', "title='Surround Stereo'", '-c:a:1', 'copy', '-c:a:2', 'aac', '-b:a:2', '128k', '-ac', '2', '-metadata:s:a:2', "title='Surround Stereo'", '-c:a:2', 'copy', '-c:a:3', 'aac', '-b:a:3', '128k', '-ac', '2', '-metadata:s:a:3', "title='Surround Stereo'", '-c:a:3', 'copy', '-c:a:4', 'aac', '-b:a:4', '128k', '-ac', '2', '-metadata:s:a:4', "title='Surround Stereo'", '-c:a:4', 'copy', '-c:a:5', 'aac', '-b:a:5', '128k', '-ac', '2', '-metadata:s:a:5', "title='Surround Stereo'", '-c:a:5', 'copy', '-c:a:6', 'aac', '-b:a:6', '128k', '-ac', '2', '-metadata:s:a:6', "title='Surround Stereo'", '-c:a:6', 'copy', '-c:a:7', 'aac', '-b:a:7', '128k', '-ac', '2', '-metadata:s:a:7', "title='Surround Stereo'", '-c:a:7', 'copy', '-c:a:8', 'aac', '-b:a:8', '128k', '-ac', '2', '-metadata:s:a:8', "title='Surround Stereo'", '-c:a:8', 'aac', '-b:a:8', '128k', '-ac', '2', '-c:s:8', 'copy', '-c:s:9', 'copy', '-c:s:10', 'copy', '-c:s:11', 'copy', '-c:s:12', 'copy', '-c:s:13', 'copy', '-c:s:14', 'copy', '-c:s:15', 'copy', '-c:s:16', 'copy', '-c:s:17', 'copy', '-y', '/tmp/unmanic/file_conversion-1562371936.667099/ (2014) [-1080p]-1562371936.667107.mkv']
Traceback (most recent call last):
File "/app/lib/worker.py", line 176, in run
self.process_task_queue_item()
File "/app/lib/worker.py", line 157, in process_task_queue_item
self.current_task.success = self.process_item()
File "/app/lib/worker.py", line 139, in process_item
ffmpeg_args)
File "/app/lib/ffmpeg.py", line 510, in convert_file_and_fetch_progress
raise FFMPEGHandleConversionError(command)
lib.ffmpeg.FFMPEGHandleConversionError: ['ffmpeg', '-y', '-i', '/library/Movies/ (2014)/ (2014) [-1080p].mkv', '-hide_banner', '-loglevel', 'info', '-strict', '-2', '-max_muxing_queue_size', '512', '-map', '0:0', '-map', '0:1', '-map', ' 0:1', '-map', '0:2', '-map', ' 0:2', '-map', '0:3', '-map', ' 0:3', '-map', '0:4', '-map', ' 0:4', '-map', '0:5', '-map', ' 0:5', '-map', '0:6', '-map', ' 0:6', '-map', '0:7', '-map', ' 0:7', '-map', '0:8', '-map', ' 0:8', '-map', ' 0:9', '-map', '0:10', '-map', '0:11', '-map', '0:12', '-map', '0:13', '-map', '0:14', '-map', '0:15', '-map', '0:16', '-map', '0:17', '-map', '0:18', '-map', '0:19', '-c:v', 'libx265', '-c:a:0', 'copy', '-c:a:1', 'aac', '-b:a:1', '128k', '-ac', '2', '-metadata:s:a:1', "title='Surround Stereo'", '-c:a:1', 'copy', '-c:a:2', 'aac', '-b:a:2', '128k', '-ac', '2', '-metadata:s:a:2', "title='Surround Stereo'", '-c:a:2', 'copy', '-c:a:3', 'aac', '-b:a:3', '128k', '-ac', '2', '-metadata:s:a:3', "title='Surround Stereo'", '-c:a:3', 'copy', '-c:a:4', 'aac', '-b:a:4', '128k', '-ac', '2', '-metadata:s:a:4', "title='Surround Stereo'", '-c:a:4', 'copy', '-c:a:5', 'aac', '-b:a:5', '128k', '-ac', '2', '-metadata:s:a:5', "title='Surround Stereo'", '-c:a:5', 'copy', '-c:a:6', 'aac', '-b:a:6', '128k', '-ac', '2', '-metadata:s:a:6', "title='Surround Stereo'", '-c:a:6', 'copy', '-c:a:7', 'aac', '-b:a:7', '128k', '-ac', '2', '-metadata:s:a:7', "title='Surround Stereo'", '-c:a:7', 'copy', '-c:a:8', 'aac', '-b:a:8', '128k', '-ac', '2', '-metadata:s:a:8', "title='Surround Stereo'", '-c:a:8', 'aac', '-b:a:8', '128k', '-ac', '2', '-c:s:8', 'copy', '-c:s:9', 'copy', '-c:s:10', 'copy', '-c:s:11', 'copy', '-c:s:12', 'copy', '-c:s:13', 'copy', '-c:s:14', 'copy', '-c:s:15', 'copy', '-c:s:16', 'copy', '-c:s:17', 'copy', '-y', '/tmp/unmanic/file_conversion-1562371936.667099/ (2014) [-1080p]-1562371936.667107.mkv']

 

Edited by Zer0Nin3r
Link to comment
  • 2 weeks later...

Shout out to @Josh.5 – I've had this running for about 3 weeks now. I only have 4TB of space on my server so space is precious. So far I've been able to reduce my media library by about 500GB – and I honestly don't think I've hit the half way mark as far as files processed. Over 1,300 successful conversions so far with no failures. 👍

Link to comment
  • 2 weeks later...
19 hours ago, jonathanm said:

Follow the directions in the first post of the thread.

Well I thought I figured it out but I don't have any of the options in the first post. I must be doing something wrong.

 

image.thumb.png.b67560164d9f3c4ef5e0ae5e56bd4e06.png

Edited by Geran
Link to comment

This docker has been incredible. The only problem I have is that I'm not pinning any cpu cores to it so it shares with my other dockers. I have the --cpu-shares=2 flag passed in, but constantly my Plex and Pihole dockers will just struggle to work so I am having issues with that.

 

My current solution has been to stop using Pihole and I have unmanic shutoff everyday at 3PM and start up again at 1AM. This isn't ideal since I'm losing encoding hours... Is there something else I could do? Maybe cpu pinning would be my best option? I just lose cores that way.

Link to comment
On 7/31/2019 at 9:43 AM, m0ngr31 said:

This docker has been incredible. The only problem I have is that I'm not pinning any cpu cores to it so it shares with my other dockers. I have the --cpu-shares=2 flag passed in, but constantly my Plex and Pihole dockers will just struggle to work so I am having issues with that.

 

My current solution has been to stop using Pihole and I have unmanic shutoff everyday at 3PM and start up again at 1AM. This isn't ideal since I'm losing encoding hours... Is there something else I could do? Maybe cpu pinning would be my best option? I just lose cores that way.

Yeah that will be a problem until you actually pin cores. --cpu-shares=2 just adjusts priority to using said core.

Link to comment
15 hours ago, Geran said:

What CPUs are people using and what is the average time it is taking you to convert files?

My unraid box is running on a 5930k and I just let unmaniac use it all. Avg times so far are around 1hr36min for a 1hr52min bluray rip with only one worker running.  A 43min tv show took around 50min with 3 workers running. I am seeing similar results in my tests so far but is going to most likely vary a lot depending on the original file.

Link to comment

Sorry if this has come up. Weird problem with unmanic. It's been working great until yesterday. It starts encoding a movie and gets to 2-3% then just disappears and starts on the next one. No failures. No completion. And the movie disappears from the pending list until they are all gone. When I reboot they show up and it starts all over again.

Sent from my Pixel 2 XL using Tapatalk

Link to comment
39 minutes ago, letrain said:

Sorry if this has come up. Weird problem with unmanic. It's been working great until yesterday. It starts encoding a movie and gets to 2-3% then just disappears and starts on the next one. No failures. No completion. And the movie disappears from the pending list until they are all gone. When I reboot they show up and it starts all over again.

Sent from my Pixel 2 XL using Tapatalk
 

I've been getting some weird stuff in my logs when it comes to movies.

 

I have the library pointed to '/library/'
And then the linking in Docker is as follows:

 

TV

/library/TV -> /mnt/user/Plex/TV

 

Movies

/library/Movies -> /mnt/user/Plex/Movies

 

Inside Unmanic via the GUI

/library/

 

TV shows get encoded just fine. For some reason with movies it spits out an error.

I haven't investigated too much into the matter, but should I come across anything else I'll be sure to update here.

On 8/3/2019 at 8:59 PM, Geran said:

What CPUs are people using and what is the average time it is taking you to convert files?

AMD Threadripper 1950x with 8 cores on die #2 pinned for Unmanic.

I can't remember the times off of my head, but it's about 1.5x in terms of speed.

 

With my previous Handbrake automation (see SpaceInvaderOne's videos) and with my custom h.265 settings, I was averaging about ~40-60 fps on movies.

Link to comment
2 hours ago, letrain said:

Sorry if this has come up. Weird problem with unmanic. It's been working great until yesterday. It starts encoding a movie and gets to 2-3% then just disappears and starts on the next one. No failures. No completion. And the movie disappears from the pending list until they are all gone. When I reboot they show up and it starts all over again.

Sent from my Pixel 2 XL using Tapatalk
 

I am seeing this as well. Originally I didn't notice because I just dumped a bunch of movies in a test directory to play with and when I checked it was still converting them. It wasn't until after  all pending tasks were gone did I see that most of the movies did not convert as there were no failed in the list. If I restart the docker it starts trying to convert again and all that are left either stop immediately or go a few % then stop, but still nothing in the failed list.

  • Upvote 1
Link to comment

This tool sounds fantastic... however I seem to not know enough about my unRaid and dockers set up to know how to get this working properly. 

 

Is there, besides the initial steps in the beginning of this thread, a walk through set up, specifically for the dependencies for this to run, which I think I am having an issue with. I can access the web gui for your container, but nothing is happening. 

 

If anyone can point me int he right direction I'd appreciate it very much. 

Link to comment
This tool sounds fantastic... however I seem to not know enough about my unRaid and dockers set up to know how to get this working properly. 
 
Is there, besides the initial steps in the beginning of this thread, a walk through set up, specifically for the dependencies for this to run, which I think I am having an issue with. I can access the web gui for your container, but nothing is happening. 
 
If anyone can point me int he right direction I'd appreciate it very much. 
Do you have the library setup correctly? That's all that really matters. Also is it set to scan at startup? Or every 60 minutes.

Sent from my Pixel 2 XL using Tapatalk

Link to comment

Ok... 

 

So I reconfigured with the suggestions from above... found the images and created the container correctly. At first it seemed nothing happened, but then I heard my server from upstairs.... 

 

See attached images.... 
 

How can I schedule this to not run 24/7 

 

I'm afraid my server is going to burn up... lol

It is working great though! 

unmanic_CPU.PNG

unmanic_SysOverview.PNG

Link to comment
20 hours ago, drawmonster said:

You need to pin cores to the container and not let it have all of them. I give it 4 cores / 8 threads.

 

Ok... So I've pinned some CPUs and threads.... See photo.
I've done the same for Plex, and I've removed my TV library from unmanic as the results have turned rather a poor quality video file. (probably because I've done something wrong)

 

Is there a way to tell unmanic to convert files that are larger than, say... 5GB? Some of my TV shoes are already only 1 to 2 GB in size, per episode, which is fine. I'd prefer the higher quality for TV shows. For movies though, a 56GB file encoded using this tool should still look and sound great - i hope... 

 

Appreciate any feedback 

unmanic_CPU_pinning.PNG

Edited by NLDer
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.