Jump to content
Josh.5

[Support] Josh5 - Unmanic - Library Optimiser

454 posts in this topic Last Reply

Recommended Posts

19 minutes ago, vcolombo said:

Quick update now that my transcodes have been running for over 12 hours on the ram disk... I'm still having some failures, but significantly fewer that before. Right now all the files I'm transcoding are files that had previously failed and most are working.

Curious, how much physical memory does your box have?

 

I'm cautious to use RAM drive with 32 GB installed

Share this post


Link to post
8 hours ago, vcolombo said:

Quick update now that my transcodes have been running for over 12 hours on the ram disk... I'm still having some failures, but significantly fewer than before. Right now all the files I'm transcoding are files that had previously failed and most are working.

Strange.  I'm transcoding to RAM and I've completed 397 transcodes and not one error.

Share this post


Link to post
On 1/31/2020 at 11:33 AM, GeekMajic said:

Curious, how much physical memory does your box have?

 

I'm cautious to use RAM drive with 32 GB installed

@GeekMajic use /dev/shm/ instead of /tmp/

 

This is allocated to use only 50% of available ram as a safeguard.

Share this post


Link to post

Running beta7

 

Unmanic is processing files - I can see the workers doing their thing, and successful conversions in the log file.  However nothing it's done recently is getting moved back to the library, nor does it show up in the web interface as finished.  In my temp directory for the transcode, I've got lots of directories with the converted files still sitting there.  

 

https://imgur.com/a/t1Ran6n

 

Here are the logs related to item 1580179451.3117027 in the image above (sanitized for privacy)

 

Quote

unmanic.log:2020-02-04T13:53:25:INFO:Unmanic.TaskHandler - [FORMATTED] - Adding job to queue - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:09:INFO:Unmanic.Worker - [FORMATTED] - Processing item - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:13:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 picked up job - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T16:25:13:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 processing job - /library/movies/some_movie_folder/some_movie.mkv
unmanic.log.2020-02-03:2020-02-03T23:35:39:INFO:Unmanic.Worker-0 - [FORMATTED] - Successfully converted file '/library/movies/some_movie_folder/some_movie.mkv'
unmanic.log.2020-02-03:2020-02-03T23:35:39:INFO:Unmanic.Worker-0 - [FORMATTED] - Worker-0 finished job - /library/movies/some_movie_folder/some_movie.mkv

And here is how it's setup in Unraid:  https://imgur.com/a/AjM1toF
/mnt/disks/172.16.9.1_Media is a share from my other Unraid server where the library is.  This is a 10Gbps link between the 2 servers

 

Any thoughts on how to get this working correctly again?  It worked in the past but maybe I missed something when I moved the docker container to another server.  

Share this post


Link to post

Does anyone here happen to know the exact ffmpeg options Unmanic uses to perform its magic? I've been running tests between Unmanic and Tdarr and Unmanic outputs significantly smaller file sizes. I'd like to make sure I'm doing a one-to-one comparison with the same options between the two.

Thanks!

Share this post


Link to post
On 2/5/2020 at 8:30 AM, vcolombo said:

Does anyone here happen to know the exact ffmpeg options Unmanic uses to perform its magic? I've been running tests between Unmanic and Tdarr and Unmanic outputs significantly smaller file sizes. I'd like to make sure I'm doing a one-to-one comparison with the same options between the two.

Thanks!

Click on your Unmanic container (on the Docker tab in Unraid) and choose Console from the menu that appears.

Next, run the following command:

ps aux | cat

This will output the full command line of all running processes. You'll be able to see exactly the parameters passed to ffmpeg.

 

One thing though, the ffmpeg binary could be compiled with different options. If you still see different results, that may be why.

Share this post


Link to post

Ah, yeah, thanks, I hadn't even thought of checking the process in the container.

 

I did notice that the plugin I'm using with Tdarr is encoding to 10-bit, that's the only difference I've noticed from looking at the processed files so far.

Share this post


Link to post
13 hours ago, vcolombo said:

Ah, yeah, thanks, I hadn't even thought of checking the process in the container.

 

I did notice that the plugin I'm using with Tdarr is encoding to 10-bit, that's the only difference I've noticed from looking at the processed files so far.

Was just looking for the settings on Github before finding this page. Was looking to up the quality a little bit. There's comments in ffmpeg.py as below:

def generate_ffmpeg_args(self,):
        # ffmpeg -i /library/XXXXX.mkv \
        #     -c:v libx265 \
        #     -map 0:0 -map 0:1 -map 0:1 \
        #     -c:a:0 copy \
        #     -c:a:1 libmp3lame -b:a:0 192k -ac 2 \
        #     -y /cache/XXXXX.mkv
        
(...)

# Create command with infile, outfile and the arguments
        command = ['ffmpeg', '-y', '-i', infile] + args + ['-y', outfile]

On that note though, would be great to be able to change the ffmpeg convert settings, even if just adding x265 CRF params. Other than that, Unmanic has been working great for me so far. 👍🏼

Share this post


Link to post

So I'm continuing my comparison between Unmanic and Tdarr and am now looking at Unmanic's audio re-encoding. I'm wondering, besides file size, what advantage is there of re-encoding all the audio tracks as 2-channel AAC? It seems to me this would be fine if you're mostly watching your movies on your phone, or through your TV speakers, but if you have any halfway decent home audio setup that you'd want to keep the discrete audio channels. Am I missing something here? Is there some magic that Unmanic is performing that reduces the audio track size while still maintaining quality?

 

Thanks in advance!

Share this post


Link to post

You can choose to not have it re-encode.  I have the halfway decent audio system, and I'd never consider dropping down to 2.0

Share this post


Link to post

Do you just set all of these to false in your config?

"ENABLE_AUDIO_ENCODING": true,
"ENABLE_AUDIO_STREAM_STEREO_CLONING": true,
"ENABLE_AUDIO_STREAM_TRANSCODING": true,

Share this post


Link to post

So I finally had my first error - but I think perhaps it's due to the actual codec that the original file is encoded in more so than the program (I'm not re-encoding the audio, just the video).  The whole series had failed.  The only error is: No FFMPEG command log recorded for this task.   Which really doesn't say much as to why it failed.

 

Anyone have any ideas?

 

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 52 min 17 s
Bit rate mode                            : Variable
Bit rate                                 : 28.5 Mb/s
Maximum bit rate                         : 32.7 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.574
Stream size                              : 10.4 GiB (87%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : DTS XLL
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS-HD Master Audio
Codec ID                                 : A_DTS
Duration                                 : 52 min 17 s
Bit rate mode                            : Variable
Bit rate                                 : 4 134 kb/s
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Compression mode                         : Lossless
Stream size                              : 1.51 GiB (13%)
Title                                    : DTS-HD MA 5.1
Language                                 : English
Default                                  : Yes
Forced                                   : No

 

Share this post


Link to post

So after my Unmanic vs Tdarr testing, I've decided to use Tdarr for now. I like the simplicity of Unmanic, but Tdarr supports NVENC, so my encodes there are much faster. Initially I was using Unmanic because the resulting files were significantly smaller than Tdarr's output, but that was until I realized Unmanic was clobbering my audio. Once I disabled the audio transcoding in Unmanic the file sizes between the two were about the same. Besides the faster GPU encoding, I like the control Tdarr gives me through the use of plugins. It was more difficult to get set up and figure out the workflow and what plugins I wanted to use, but now that I've gone through that it seems to be working pretty well.

Share this post


Link to post

SO, doing a bit of a comparison between Tdarr and Unmanic.  If I wanted to replicate the video encoding settings that Unmanic uses in Tdarr, what would I use?

Share this post


Link to post
1 hour ago, JasonK said:

SO, doing a bit of a comparison between Tdarr and Unmanic.  If I wanted to replicate the video encoding settings that Unmanic uses in Tdarr, what would I use?

Scroll up to February 5th, Is that what you're looking for?

Share this post


Link to post
On 2/15/2020 at 2:56 AM, vcolombo said:

So after my Unmanic vs Tdarr testing, I've decided to use Tdarr for now. I like the simplicity of Unmanic, but Tdarr supports NVENC, so my encodes there are much faster. <SNIP>

 

I was under the assumption that quality wise CPU Encoding by far trumps NVIDIA encoding.  Although the NVENC may be a ton faster, I read it is 'sloppy' as far as it's compression algorithm.  By using a CPU encode, the results, while slower, produce a far higher quality encode.

Share this post


Link to post
2 hours ago, CoZ said:

 

I was under the assumption that quality wise CPU Encoding by far trumps NVIDIA encoding.  Although the NVENC may be a ton faster, I read it is 'sloppy' as far as it's compression algorithm.  By using a CPU encode, the results, while slower, produce a far higher quality encode.

That's what I've heard as well, but I honestly don't notice a difference. It's entirely possible that CPU encoding may result in a higher video quality at a smaller file size, but I have hundreds of DVDs and blu-rays I'm ripping, so I'd rather have faster encoding speeds if it means a negligible quality difference. I need to do a side-by-side comparison of a standard DVD, but the blu-rays I really can't see a difference.

 

Someday, when I get my Threadripper 3990x processor after winning the lottery, then I'll re-encode everything with CPU only. ;)

Edited by vcolombo

Share this post


Link to post

This program raises my xeon 3450 temp from 40 degrees to 80 with one worker,  if I do 2 or 3 then the alarm goes off on the motherboard. 

 

Maybe this app can use less resources?

Share this post


Link to post
13 minutes ago, Bandit_King said:

This program raises my xeon 3450 temp from 40 degrees to 80 with one worker,  if I do 2 or 3 then the alarm goes off on the motherboard. 

 

Maybe this app can use less resources?

 

Then to be honest you don't have enough cooling in your system.  IMO, any computer (server / desktop / laptop/ phone) has to be able to handle a full CPU load at any given time for extended periods.

 

3 choices:

 

Increase the cooling

Downgrade the CPU to something that doesn't run as hot

Experiment with the docker run command to limit the maximum CPU that the app can use (google docker run reference)

Share this post


Link to post
On 2/14/2020 at 11:56 PM, vcolombo said:

Initially I was using Unmanic because the resulting files were significantly smaller than Tdarr's output, but that was until I realized Unmanic was clobbering my audio.

I think this is a bug in Unmanic. On the audio encoding settings page it states:

"Modify the audio stream codec. Channel count and bitrate are kept the same as the source."

 

So it shouldn't be reducing the number of channels while re-encoding.

Share this post


Link to post

Except for that one TV series that failed to convert (and I've no idea why it failed) I'm now having the same issue when I point it to a specific movie folders.  The past four movies I've tried to encode to HEVC have failed.  No explanation given other than the generic:

 

" No FFMPEG command log recorded for this task. "

 

I've been (so far) only about to convert one movie on my array.  One has been successful, the other 4 have failed with that error.

 

Has anyone figured out a way to "clear" the failed jobs out of the history?  There are check boxes next to the completed/failed jobs but checking them does nothing.

Share this post


Link to post
Except for that one TV series that failed to convert (and I've no idea why it failed) I'm now having the same issue when I point it to a specific movie folders.  The past four movies I've tried to encode to HEVC have failed.  No explanation given other than the generic:
 
" No FFMPEG command log recorded for this task. "
 
I've been (so far) only about to convert one movie on my array.  One has been successful, the other 4 have failed with that error.
 
Has anyone figured out a way to "clear" the failed jobs out of the history?  There are check boxes next to the completed/failed jobs but checking them does nothing.
FFMPEG log recording is only enabled if you turn on debugging.

Sent from my ONEPLUS A5010 using Tapatalk

Share this post


Link to post
3 hours ago, Josh.5 said:

FFMPEG log recording is only enabled if you turn on debugging.

Sent from my ONEPLUS A5010 using Tapatalk
 

Thanks.  Enabled it, selected one of the failed movies and got this error:

 

[matroska,webm @ 0x152de0e5d180] Could not find codec parameters for stream 2 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options

 

So, it doesn't like movies with Subtitles in them or something?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.