[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

How does one.... when wanting the original file for best quality available, exclude files from being re-encoded?

 

My system is humming away at encoding my entire library (movies only) to H.265 HEVC. I've tested a few files - before and after - and there isn't a terrible difference in quality as far as I can see / hear. 

 

Is there any reason not to re-encode? Other than space saving? 

Back to exclusions... there are TV shows I have i'd love to re-encode to save space, there are others.. daily show and such, that delete automatically after 5 days post viewing date. No need to re-encode those... how can I do this?

Link to comment

Got this up an running easily.  Great plugin. I just hope my i5-2500 doesn't self-combust.  Is there a way of limiting the load on the cores?  I know I can pin cores but I'd feel better knowing I can max out on 80%.  Probably more for emotional reasons than technical ones.

 

Came across an error today.  I dropped a few big files in, 17gb and a 10gg before work (~07:00)00.  Came home (~19:30) and the two threads were showing 30% and 39% complete.  That didn't make much sense 12 hours after kicking them.  Checked the log and I see ffmpeg threw an error early in the piece and stopped the conversion.  Every hour unmanic checked for files, saw they were already in the queue so did nothing after that.  It's been sitting there doing nothing.

 

Here's the log.  Not sure if it's a known thing or not.  I would have expected that the file gets tagged as failed, and the next in queue get processed.

 

The Log:

tdm-unmanic-20190849.txt

Link to comment
6 hours ago, ijuarez said:

This is a really stupid question but I am assuming that it has its internal DB so if the server shuts down it doesn't try to re-encode the same files, right?

Sent from my SM-N960U using Tapatalk
 

If you restart, yes the list of completed items is maintained. I would assume it then doesn't try to re-encode them.

Link to comment

Possible bug report. Not a show stopper, just unexpected behaviour. When I reduced the number of workers and applied the setting, the extraneous workers finished their job, but the file was not replaced in the library, nor was the temporary location cleared. I restarted the container, and the temporary work was deleted and the original files were untouched.

 

The files in question were subsequently converted.

 

Not a huge issue, just means that if you reduce the number of workers you will lose the time they spent on their current job.

Edited by jonathanm
Link to comment
On 8/17/2019 at 7:25 PM, ijuarez said:

This is a really stupid question but I am assuming that it has its internal DB so if the server shuts down it doesn't try to re-encode the same files, right?

Sent from my SM-N960U using Tapatalk
 

Well, it doesn't really need a DB, because the way the Unmanic was written, it scans your video files to check to see if they are already encoded in h.265. If they are, then those files are skipped.

 

Quote

First, Unmanic has a scheduler built in to scan your whole library for files that do not conform to your video presets. Videos found with incorrect formats are then queued for conversion.

source: Page 1 of this thread

Edited by Zer0Nin3r
Added some more insight.
Link to comment
On 8/14/2019 at 1:34 PM, NLDer said:

How does one.... when wanting the original file for best quality available, exclude files from being re-encoded?

At this point you don't. The soul purpose of Unmanic is to convert your library to h.265 as per the program's description.

Actually, I thought about this before and forgot about it. My workaround would be like this:

  1. Create a separate "archive" library in Plex.
  2. Direct your downloads to the "archive" folder.

The idea is to have your archive folder outside of the watch folder that Unmanic keeps an eye on.

 

On 8/14/2019 at 1:34 PM, NLDer said:

I've tested a few files - before and after - and there isn't a terrible difference in quality as far as I can see / hear. 

That again, is the point. h.265 is the newer codec that eventually we will all switch over to. The algorithms allow for the same quality of video at a higher compression. The downside at this time is that encoding is slower because the codec is more CPU intensive than h.264; wait for technology to catch up and h.265 encode times will drop.

 

What does this all mean? Think about the abolishment of Net Neutrality. ISP's are happy to charge Netflix more money for them to stream their content to you. (Which then those costs are passed onto you the consumer...another story for another time.) And if your ISP has data caps on your home internet, then larger files means less content you will be able to watch unless you want to pay for overage fees. In order to save both the consumer and the company (Netflix) itself money, Netflix will need to compress the video more without losing quality. That's what h.265 does great. example: Let's say a h.264 movie to stream is 8 GB (I'm just using a random number.) That same movie to stream on h.265 may now only "cost" ~1.2 GB. Same video quality, less space.

 

What h.265 doesn't do: Up-res. From your comment it can be inferred that one may have expectations (present company excluded) h.265 to improve the quality of the video that is being re-encoded. But like a copy machine — a copy of a copy of a copy results in degradation each time.

 

On 8/14/2019 at 1:34 PM, NLDer said:

Is there any reason not to re-encode? Other than space saving? 

If your Plex server is not on a powerful machine, then you may not want to re-encode into h.265 because currently, Plex will always transcode h.265 to h.264 on the fly when streaming on your devices. Well, that's not entirely true — I know that the Nvidia Shield TV on a LAN will direct stream the video without transcoding and the same with the Pixel 2 XL on a LAN. But accessing Plex from outside of your home — yes, Plex will transcode to h.264 every time (unless you change your remote streaming settings from your Plex client (notice I didn't say server.)

 

On 8/14/2019 at 1:34 PM, NLDer said:

here are TV shows I have i'd love to re-encode to save space, there are others.. daily show and such, that delete automatically after 5 days post viewing date. No need to re-encode those... how can I do this?

Not possible at the moment, although myself and others have requested this feature.

 

On 8/21/2019 at 2:01 PM, tronyx said:

Is it possible to only perform the audio conversion?

Magic 8-Ball says: My sources say no.

 

Maybe in future updates, but Unmanic is developed and maintained by a single developer at the moment. However, if you are coding savvy, I encourage you to make contributions to the GitHub to push development further. I would heed my own advice, but I haven't progressed past power user into programmer. But, if someone wants to take me under their wing and teach me to code or pay for me to go back to school — I won't turn down a free lunch. ^_^

Edited by Zer0Nin3r
The description doesn't explicity say h.265 | Added a possible workaround.
Link to comment
My unmanic docker just simply disappeared... 
 
Is there a known issue? 
Generally only reason for a container to "disappear" is an error in the template. Reading via apps, previous apps will show the docker run command and the error present

Sent from my NSA monitored device

Link to comment
5 minutes ago, Squid said:

Generally only reason for a container to "disappear" is an error in the template. Reading via apps, previous apps will show the docker run command and the error present

Sent from my NSA monitored device
 

I did play with the url settings for the webui as i have not been able to get that to work.... Is there a way to recover? 

Link to comment
13 hours ago, NLDer said:

I did play with the url settings for the webui as i have not been able to get that to work.... Is there a way to recover? 

So, by adding the container agian, the image was there and has restored my configuration. I'm still not able to get:
1. the logo to show
2. the WebUI option to show - I manually go to the address to open unmanic. 

Can anyone steer me int he right direction perhaps? 

 

Link to comment




 
@Josh.5 so it appears things have shit the bed to say (attached is a snib of the logs) how would you like a proper dump? debugging turned on?
image.thumb.png.8ca2f26170ced8bd532b4013f455f384.png
image.png.f666786b95f5cead87bdd042f7ef00d6.png


Actually that output is enough as far as unmanic goes. It looks like ffmpeg did not like the arguments it was passed or the file needed some kind of special configuration.
Can you please try running that command manually from a terminal inside the docker container. Ffmpeg will give you some output that will show what is wrong. Then can you give me that output?

Sent from my ONEPLUS A6000 using Tapatalk

Link to comment
I am seeing this as well. Originally I didn't notice because I just dumped a bunch of movies in a test directory to play with and when I checked it was still converting them. It wasn't until after  all pending tasks were gone did I see that most of the movies did not convert as there were no failed in the list. If I restart the docker it starts trying to convert again and all that are left either stop immediately or go a few % then stop, but still nothing in the failed list.
All tv shows seem to work fine. And some new movies that i download get converted. It's very strange. But I have heard nothing from the developer here or on GitHub.

Sent from my Pixel 2 XL using Tapatalk

  • Thanks 1
Link to comment
On 9/3/2019 at 9:38 PM, letrain said:

All tv shows seem to work fine. And some new movies that i download get converted. It's very strange. But I have heard nothing from the developer here or on GitHub.

Sent from my Pixel 2 XL using Tapatalk
 

FYI,  I went into the settings and check the remove Subtitles box and it seemed to fix majority of the issues.  I'm thinking ffmpeg is failing trying to convert embedded subtitles.

Link to comment
FYI,  I went into the settings and check the remove Subtitles box and it seemed to fix majority of the issues.  I'm thinking ffmpeg is failing trying to convert embedded subtitles.
No wonder it only processed some movies but not all. I'd like to keep the subtitles and the only option is to remove them. Most of my conversions are h264->h265. The container stays the same...is there any way to save them?

Sent from my Pixel 2 XL using Tapatalk

Link to comment
3 hours ago, letrain said:

No wonder it only processed some movies but not all. I'd like to keep the subtitles and the only option is to remove them. Most of my conversions are h264->h265. The container stays the same...is there any way to save them?

Sent from my Pixel 2 XL using Tapatalk
 

You could always get the subtitles from opensubititles.org and keep them separate from the movie container.   I think the issue is when they are inside of the container and in a format ffmpeg can't convert/understand.  I use Emby and Jellyfin for my media and they have the ability to download subtitles for you.  They are usually placed in the same directory as your video file.

Link to comment
No wonder it only processed some movies but not all. I'd like to keep the subtitles and the only option is to remove them. Most of my conversions are h264->h265. The container stays the same...is there any way to save them?

Sent from my Pixel 2 XL using Tapatalk

Subtitles are a little tricky. At the moment unmanic has real shitty support for the subtitle streams in your video files. The problem is that different containers have different implementations on how subtitles are stored and therefore ffmpeg requires different arguments depending on your combination of subtitles, src container and destination container. Removing embedded subtitle streams will fix a lot of errors when converting between containers until I expand on the settings capabilities of unmanic and how it generates tasks for ffmpeg.

Sent from my ONEPLUS A6000 using Tapatalk

  • Like 2
Link to comment
  • 2 weeks later...
On 9/6/2019 at 9:06 PM, Josh.5 said:

Subtitles are a little tricky. At the moment unmanic has real shitty support for the subtitle streams in your video files. The problem is that different containers have different implementations on how subtitles are stored and therefore ffmpeg requires different arguments depending on your combination of subtitles, src container and destination container. Removing embedded subtitle streams will fix a lot of errors when converting between containers until I expand on the settings capabilities of unmanic and how it generates tasks for ffmpeg.

Sent from my ONEPLUS A6000 using Tapatalk
 

Thanks. i have turned this off and seems to be working. Well encoding the problem files. Also i have noticed that many have asked for gpu support. i just purchased a p2000 and have passed through to plex docker for transcoding and easily handles everything i throw at it. i would like to throw my vote toward GPU implementation as well. would speed up the library conversion for sure. :)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.