[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

19 minutes ago, vcolombo said:

Quick update now that my transcodes have been running for over 12 hours on the ram disk... I'm still having some failures, but significantly fewer that before. Right now all the files I'm transcoding are files that had previously failed and most are working.

Curious, how much physical memory does your box have?

 

I'm cautious to use RAM drive with 32 GB installed

Link to comment
8 hours ago, vcolombo said:

Quick update now that my transcodes have been running for over 12 hours on the ram disk... I'm still having some failures, but significantly fewer than before. Right now all the files I'm transcoding are files that had previously failed and most are working.

Strange.  I'm transcoding to RAM and I've completed 397 transcodes and not one error.

Link to comment

Running beta7

 

Unmanic is processing files - I can see the workers doing their thing, and successful conversions in the log file.  However nothing it's done recently is getting moved back to the library, nor does it show up in the web interface as finished.  In my temp directory for the transcode, I've got lots of directories with the converted files still sitting there.  

 

https://imgur.com/a/t1Ran6n

 

Here are the logs related to item 1580179451.3117027 in the image above (sanitized for privacy)

 

Quote

unmanic.log:2020-02-04T13:53:25:INFO:Unmanic.TaskHandler - [FORMATTED] - Adding job to queue - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:09:INFO:Unmanic.Worker - [FORMATTED] - Processing item - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:13:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 picked up job - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:13:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 processing job - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T23:35:39:INFO:Unmanic.Worker-0 - [FORMATTED] - Successfully converted file '/library/movies/some_movie_folder/some_movie.mkv'
unmanic.log.2020-02-03:2020-02-03T23:35:39:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 finished job - /library/movies/some_movie_folder/some_movie.mkv

And here is how it's setup in Unraid:  https://imgur.com/a/AjM1toF
/mnt/disks/172.16.9.1_Media is a share from my other Unraid server where the library is.  This is a 10Gbps link between the 2 servers

 

Any thoughts on how to get this working correctly again?  It worked in the past but maybe I missed something when I moved the docker container to another server.  

Link to comment

Does anyone here happen to know the exact ffmpeg options Unmanic uses to perform its magic? I've been running tests between Unmanic and Tdarr and Unmanic outputs significantly smaller file sizes. I'd like to make sure I'm doing a one-to-one comparison with the same options between the two.

Thanks!

Link to comment
On 2/5/2020 at 8:30 AM, vcolombo said:

Does anyone here happen to know the exact ffmpeg options Unmanic uses to perform its magic? I've been running tests between Unmanic and Tdarr and Unmanic outputs significantly smaller file sizes. I'd like to make sure I'm doing a one-to-one comparison with the same options between the two.

Thanks!

Click on your Unmanic container (on the Docker tab in Unraid) and choose Console from the menu that appears.

Next, run the following command:

ps aux | cat

This will output the full command line of all running processes. You'll be able to see exactly the parameters passed to ffmpeg.

 

One thing though, the ffmpeg binary could be compiled with different options. If you still see different results, that may be why.

  • Thanks 1
Link to comment
13 hours ago, vcolombo said:

Ah, yeah, thanks, I hadn't even thought of checking the process in the container.

 

I did notice that the plugin I'm using with Tdarr is encoding to 10-bit, that's the only difference I've noticed from looking at the processed files so far.

Was just looking for the settings on Github before finding this page. Was looking to up the quality a little bit. There's comments in ffmpeg.py as below:

def generate_ffmpeg_args(self,):
        # ffmpeg -i /library/XXXXX.mkv \
        #     -c:v libx265 \
        #     -map 0:0 -map 0:1 -map 0:1 \
        #     -c:a:0 copy \
        #     -c:a:1 libmp3lame -b:a:0 192k -ac 2 \
        #     -y /cache/XXXXX.mkv
        
(...)

# Create command with infile, outfile and the arguments
        command = ['ffmpeg', '-y', '-i', infile] + args + ['-y', outfile]

On that note though, would be great to be able to change the ffmpeg convert settings, even if just adding x265 CRF params. Other than that, Unmanic has been working great for me so far. 👍🏼

Link to comment

So I'm continuing my comparison between Unmanic and Tdarr and am now looking at Unmanic's audio re-encoding. I'm wondering, besides file size, what advantage is there of re-encoding all the audio tracks as 2-channel AAC? It seems to me this would be fine if you're mostly watching your movies on your phone, or through your TV speakers, but if you have any halfway decent home audio setup that you'd want to keep the discrete audio channels. Am I missing something here? Is there some magic that Unmanic is performing that reduces the audio track size while still maintaining quality?

 

Thanks in advance!

Link to comment

So I finally had my first error - but I think perhaps it's due to the actual codec that the original file is encoded in more so than the program (I'm not re-encoding the audio, just the video).  The whole series had failed.  The only error is: No FFMPEG command log recorded for this task.   Which really doesn't say much as to why it failed.

 

Anyone have any ideas?

 

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 52 min 17 s
Bit rate mode                            : Variable
Bit rate                                 : 28.5 Mb/s
Maximum bit rate                         : 32.7 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.574
Stream size                              : 10.4 GiB (87%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : DTS XLL
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS-HD Master Audio
Codec ID                                 : A_DTS
Duration                                 : 52 min 17 s
Bit rate mode                            : Variable
Bit rate                                 : 4 134 kb/s
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Compression mode                         : Lossless
Stream size                              : 1.51 GiB (13%)
Title                                    : DTS-HD MA 5.1
Language                                 : English
Default                                  : Yes
Forced                                   : No

 

Link to comment

So after my Unmanic vs Tdarr testing, I've decided to use Tdarr for now. I like the simplicity of Unmanic, but Tdarr supports NVENC, so my encodes there are much faster. Initially I was using Unmanic because the resulting files were significantly smaller than Tdarr's output, but that was until I realized Unmanic was clobbering my audio. Once I disabled the audio transcoding in Unmanic the file sizes between the two were about the same. Besides the faster GPU encoding, I like the control Tdarr gives me through the use of plugins. It was more difficult to get set up and figure out the workflow and what plugins I wanted to use, but now that I've gone through that it seems to be working pretty well.

  • Thanks 1
Link to comment
On 2/15/2020 at 2:56 AM, vcolombo said:

So after my Unmanic vs Tdarr testing, I've decided to use Tdarr for now. I like the simplicity of Unmanic, but Tdarr supports NVENC, so my encodes there are much faster. <SNIP>

 

I was under the assumption that quality wise CPU Encoding by far trumps NVIDIA encoding.  Although the NVENC may be a ton faster, I read it is 'sloppy' as far as it's compression algorithm.  By using a CPU encode, the results, while slower, produce a far higher quality encode.

  • Like 1
Link to comment
2 hours ago, CoZ said:

 

I was under the assumption that quality wise CPU Encoding by far trumps NVIDIA encoding.  Although the NVENC may be a ton faster, I read it is 'sloppy' as far as it's compression algorithm.  By using a CPU encode, the results, while slower, produce a far higher quality encode.

That's what I've heard as well, but I honestly don't notice a difference. It's entirely possible that CPU encoding may result in a higher video quality at a smaller file size, but I have hundreds of DVDs and blu-rays I'm ripping, so I'd rather have faster encoding speeds if it means a negligible quality difference. I need to do a side-by-side comparison of a standard DVD, but the blu-rays I really can't see a difference.

 

Someday, when I get my Threadripper 3990x processor after winning the lottery, then I'll re-encode everything with CPU only. ;)

Edited by vcolombo
  • Haha 1
Link to comment
13 minutes ago, Bandit_King said:

This program raises my xeon 3450 temp from 40 degrees to 80 with one worker,  if I do 2 or 3 then the alarm goes off on the motherboard. 

 

Maybe this app can use less resources?

 

Then to be honest you don't have enough cooling in your system.  IMO, any computer (server / desktop / laptop/ phone) has to be able to handle a full CPU load at any given time for extended periods.

 

3 choices:

 

Increase the cooling

Downgrade the CPU to something that doesn't run as hot

Experiment with the docker run command to limit the maximum CPU that the app can use (google docker run reference)

  • Like 1
Link to comment
On 2/14/2020 at 11:56 PM, vcolombo said:

Initially I was using Unmanic because the resulting files were significantly smaller than Tdarr's output, but that was until I realized Unmanic was clobbering my audio.

I think this is a bug in Unmanic. On the audio encoding settings page it states:

"Modify the audio stream codec. Channel count and bitrate are kept the same as the source."

 

So it shouldn't be reducing the number of channels while re-encoding.

Link to comment

Except for that one TV series that failed to convert (and I've no idea why it failed) I'm now having the same issue when I point it to a specific movie folders.  The past four movies I've tried to encode to HEVC have failed.  No explanation given other than the generic:

 

" No FFMPEG command log recorded for this task. "

 

I've been (so far) only about to convert one movie on my array.  One has been successful, the other 4 have failed with that error.

 

Has anyone figured out a way to "clear" the failed jobs out of the history?  There are check boxes next to the completed/failed jobs but checking them does nothing.

Link to comment
Except for that one TV series that failed to convert (and I've no idea why it failed) I'm now having the same issue when I point it to a specific movie folders.  The past four movies I've tried to encode to HEVC have failed.  No explanation given other than the generic:
 
" No FFMPEG command log recorded for this task. "
 
I've been (so far) only about to convert one movie on my array.  One has been successful, the other 4 have failed with that error.
 
Has anyone figured out a way to "clear" the failed jobs out of the history?  There are check boxes next to the completed/failed jobs but checking them does nothing.
FFMPEG log recording is only enabled if you turn on debugging.

Sent from my ONEPLUS A5010 using Tapatalk

Link to comment
3 hours ago, Josh.5 said:

FFMPEG log recording is only enabled if you turn on debugging.

Sent from my ONEPLUS A5010 using Tapatalk
 

Thanks.  Enabled it, selected one of the failed movies and got this error:

 

[matroska,webm @ 0x152de0e5d180] Could not find codec parameters for stream 2 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options

 

So, it doesn't like movies with Subtitles in them or something?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.