[Support] Josh5 - Unmanic - Library Optimiser


1072 posts in this topic Last Reply

Recommended Posts

3 hours ago, Squid said:

Personally, instead of pinning this I prioritize every other docker app to take precedence over this.

 

https://forums.unraid.net/topic/57181-real-docker-faq/page/2/#comment-566087

 

so do you leave any cpu pins in place when you're using this prioritization? or just give unmanic full cpu access with low priority and let her rip?

Edited by Cpt. Chaz
typo
Link to post
  • Replies 1.1k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Application Name: UNMANIC - Library Optimiser Application Site: https://github.com/Unmanic/unmanic/ Docker Hub: https://hub.docker.com/r/josh5/unmanic/ Github: https://github.com/Unmani

@Ashe @trekkiedj @itimpi   Please see update first post also for an example of how I have it setup. I have also added a temporary solution to adding multiple library locations if anyone need

There should be a new update available. This has quite a number of changes to how ffmpeg is executed. It should resolve some issues with inotify and library scanning. For those people who created

Posted Images

I'm seeing these errors on a few of my files.

[matroska,webm @ 0x14d1e69b6180] Could not find codec parameters for stream 2 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, matroska,webm, from '/library/movies/Sherlock Holmes (2009)/Sherlock.Holmes.2009.Bluray-1080p.mkv':
  Metadata:
    title           : 
    creation_time   : 2019-02-09T21:18:11.000000Z
    ENCODER         : Lavf58.20.100
  Duration: 02:08:24.75, start: 0.000000, bitrate: 3852 kb/s
    Chapter #0:0: start 0.000000, end 406.406000
    Metadata:
      title           : Chapter 1
    Chapter #0:1: start 406.406000, end 690.148000
    Metadata:
      title           : Chapter 2
    Chapter #0:2: start 690.148000, end 941.649000
    Metadata:
      title           : Chapter 3
    Chapter #0:3: start 941.649000, end 1148.856000
    Metadata:
      title           : Chapter 4
    Chapter #0:4: start 1148.856000, end 1408.824000
    Metadata:
      title           : Chapter 5
    Chapter #0:5: start 1408.824000, end 1669.835000
    Metadata:
      title           : Chapter 6
    Chapter #0:6: start 1669.835000, end 1921.211000
    Metadata:
      title           : Chapter 7
    Chapter #0:7: start 1921.211000, end 2228.268000
    Metadata:
      title           : Chapter 8
    Chapter #0:8: start 2228.268000, end 2450.281000
    Metadata:
      title           : Chapter 9
    Chapter #0:9: start 2450.281000, end 2677.133000
    Metadata:
      title           : Chapter 10
    Chapter #0:10: start 2677.133000, end 2859.607000
    Metadata:
      title           : Chapter 11
    Chapter #0:11: start 2859.607000, end 3051.924000
    Metadata:
      title           : Chapter 12
    Chapter #0:12: start 3051.924000, end 3317.523000
    Metadata:
      title           : Chapter 13
    Chapter #0:13: start 3317.523000, end 3544.416000
    Metadata:
      title           : Chapter 14
    Chapter #0:14: start 3544.416000, end 3784.531000
    Metadata:
      title           : Chapter 15
    Chapter #0:15: start 3784.531000, end 3967.714000
    Metadata:
      title           : Chapter 16
    Chapter #0:16: start 3967.714000, end 4273.895000
    Metadata:
      title           : Chapter 17
    Chapter #0:17: start 4273.895000, end 4479.225000
    Metadata:
      title           : Chapter 18
    Chapter #0:18: start 4479.225000, end 4656.235000
    Metadata:
      title           : Chapter 19
    Chapter #0:19: start 4656.235000, end 5015.010000
    Metadata:
      title           : Chapter 20
    Chapter #0:20: start 5015.010000, end 5166.912000
    Metadata:
      title           : Chapter 21
    Chapter #0:21: start 5166.912000, end 5457.035000
    Metadata:
      title           : Chapter 22
    Chapter #0:22: start 5457.035000, end 5715.293000
    Metadata:
      title           : Chapter 23
    Chapter #0:23: start 5715.293000, end 5985.897000
    Metadata:
      title           : Chapter 24
    Chapter #0:24: start 5985.897000, end 6257.835000
    Metadata:
      title           : Chapter 25
    Chapter #0:25: start 6257.835000, end 6501.912000
    Metadata:
      title           : Chapter 26
    Chapter #0:26: start 6501.912000, end 6842.669000
    Metadata:
      title           : Chapter 27
    Chapter #0:27: start 6842.669000, end 6967.419000
    Metadata:
      title           : Chapter 28
    Chapter #0:28: start 6967.419000, end 7226.094000
    Metadata:
      title           : Chapter 29
    Chapter #0:29: start 7226.094000, end 7704.697000
    Metadata:
      title           : Chapter 30
    Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc (default)
    Metadata:
      DURATION        : 02:08:24.739000000
    Stream #0:1(eng): Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s (default)
    Metadata:
      title           : Surround
      DURATION        : 02:08:24.747000000
    Stream #0:2(eng): Subtitle: hdmv_pgs_subtitle
    Metadata:
      DURATION        : 02:08:23.133000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (dts (dca) -> aac (native))
  Stream #0:1 -> #0:2 (dts (dca) -> aac (native))
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
[libx264 @ 0x55f350758680] using SAR=1/1
[libx264 @ 0x55f350758680] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0x55f350758680] profile High, level 4.0
[libx264 @ 0x55f350758680] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=21 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/tmp/unmanic/file_conversion-1570601247.879251/Sherlock.Holmes.2009.Bluray-1080p-1570601247.879268.mp4':
  Metadata:
    title           : 
    encoder         : Lavf58.20.100
    Chapter #0:0: start 0.000000, end 406.406000
    Metadata:
      title           : Chapter 1
    Chapter #0:1: start 406.406000, end 690.148000
    Metadata:
      title           : Chapter 2
    Chapter #0:2: start 690.148000, end 941.649000
    Metadata:
      title           : Chapter 3
    Chapter #0:3: start 941.649000, end 1148.856000
    Metadata:
      title           : Chapter 4
    Chapter #0:4: start 1148.856000, end 1408.824000
    Metadata:
      title           : Chapter 5
    Chapter #0:5: start 1408.824000, end 1669.835000
    Metadata:
      title           : Chapter 6
    Chapter #0:6: start 1669.835000, end 1921.211000
    Metadata:
      title           : Chapter 7
    Chapter #0:7: start 1921.211000, end 2228.268000
    Metadata:
      title           : Chapter 8
    Chapter #0:8: start 2228.268000, end 2450.281000
    Metadata:
      title           : Chapter 9
    Chapter #0:9: start 2450.281000, end 2677.133000
    Metadata:
      title           : Chapter 10
    Chapter #0:10: start 2677.133000, end 2859.607000
    Metadata:
      title           : Chapter 11
    Chapter #0:11: start 2859.607000, end 3051.924000
    Metadata:
      title           : Chapter 12
    Chapter #0:12: start 3051.924000, end 3317.523000
    Metadata:
      title           : Chapter 13
    Chapter #0:13: start 3317.523000, end 3544.416000
    Metadata:
      title           : Chapter 14
    Chapter #0:14: start 3544.416000, end 3784.531000
    Metadata:
      title           : Chapter 15
    Chapter #0:15: start 3784.531000, end 3967.714000
    Metadata:
      title           : Chapter 16
    Chapter #0:16: start 3967.714000, end 4273.895000
    Metadata:
      title           : Chapter 17
    Chapter #0:17: start 4273.895000, end 4479.225000
    Metadata:
      title           : Chapter 18
    Chapter #0:18: start 4479.225000, end 4656.235000
    Metadata:
      title           : Chapter 19
    Chapter #0:19: start 4656.235000, end 5015.010000
    Metadata:
      title           : Chapter 20
    Chapter #0:20: start 5015.010000, end 5166.912000
    Metadata:
      title           : Chapter 21
    Chapter #0:21: start 5166.912000, end 5457.035000
    Metadata:
      title           : Chapter 22
    Chapter #0:22: start 5457.035000, end 5715.293000
    Metadata:
      title           : Chapter 23
    Chapter #0:23: start 5715.293000, end 5985.897000
    Metadata:
      title           : Chapter 24
    Chapter #0:24: start 5985.897000, end 6257.835000
    Metadata:
      title           : Chapter 25
    Chapter #0:25: start 6257.835000, end 6501.912000
    Metadata:
      title           : Chapter 26
    Chapter #0:26: start 6501.912000, end 6842.669000
    Metadata:
      title           : Chapter 27
    Chapter #0:27: start 6842.669000, end 6967.419000
    Metadata:
      title           : Chapter 28
    Chapter #0:28: start 6967.419000, end 7226.094000
    Metadata:
      title           : Chapter 29
    Chapter #0:29: start 7226.094000, end 7704.697000
    Metadata:
      title           : Chapter 30
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 23.98 fps, 24k tbn, 23.98 tbc (default)
    Metadata:
      DURATION        : 02:08:24.739000000
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      title           : Surround
      DURATION        : 02:08:24.747000000
      encoder         : Lavc58.35.100 aac
    Stream #0:2(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      DURATION        : 02:08:24.747000000
      title           : 'SurroundStereo'
      encoder         : Lavc58.35.100 aac
frame=    1 fps=0.7 q=0.0 size=       1kB time=00:00:00.55 bitrate=  10.9kbits/s speed= 0.4x    
frame=    8 fps=3.9 q=0.0 size=       1kB time=00:00:00.85 bitrate=   7.1kbits/s speed=0.419x    
frame=   11 fps=4.3 q=0.0 size=       1kB time=00:00:00.98 bitrate=   6.2kbits/s speed=0.384x    
frame=   22 fps=7.1 q=0.0 size=       1kB time=00:00:01.42 bitrate=   4.2kbits/s speed=0.464x    
frame=   38 fps= 11 q=0.0 size=       1kB time=00:00:02.09 bitrate=   2.9kbits/s speed=0.581x    
frame=   46 fps= 11 q=0.0 size=       1kB time=00:00:02.43 bitrate=   2.5kbits/s speed=0.561x    
frame=   52 fps= 10 q=0.0 size=       1kB time=00:00:02.68 bitrate=   2.3kbits/s speed=0.519x    
frame=   54 fps=9.5 q=0.0 size=       1kB time=00:00:02.75 bitrate=   2.2kbits/s speed=0.484x    
frame=   55 fps=7.9 q=0.0 size=       1kB time=00:00:02.79 bitrate=   2.2kbits/s speed=0.404x    
frame=   58 fps=7.8 q=0.0 size=       1kB time=00:00:02.92 bitrate=   2.1kbits/s speed=0.393x    
frame=   61 fps=7.7 q=0.0 size=       1kB time=00:00:03.05 bitrate=   2.0kbits/s speed=0.385x    
frame=   64 fps=7.5 q=0.0 size=       1kB time=00:00:03.17 bitrate=   1.9kbits/s speed=0.373x    
frame=   66 fps=7.1 q=28.0 size=       1kB time=00:00:03.26 bitrate=   1.9kbits/s speed=0.349x    
frame=   68 fps=6.8 q=28.0 size=       1kB time=00:00:03.34 bitrate=   1.8kbits/s speed=0.337x    
frame=   77 fps=7.4 q=28.0 size=       1kB time=00:00:03.71 bitrate=   1.6kbits/s speed=0.355x    
frame=   85 fps=7.8 q=28.0 size=       1kB time=00:00:04.05 bitrate=   1.5kbits/s speed=0.37x    
frame=   91 fps=7.8 q=28.0 size=       1kB time=00:00:04.30 bitrate=   1.4kbits/s speed=0.37x    
frame=   96 fps=7.6 q=28.0 size=       1kB time=00:00:04.52 bitrate=   1.3kbits/s speed=0.357x    
frame=  104 fps=7.9 q=28.0 size=       1kB time=00:00:04.84 bitrate=   1.3kbits/s speed=0.367x    
frame=  111 fps=5.9 q=28.0 size=       1kB time=00:00:05.14 bitrate=   1.2kbits/s speed=0.275x    
frame=  119 fps=6.2 q=28.0 size=       1kB time=00:00:05.48 bitrate=   1.1kbits/s speed=0.285x    
frame=  120 fps=5.9 q=28.0 size=       1kB time=00:00:05.50 bitrate=   1.1kbits/s speed=0.269x    
frame=  121 fps=5.5 q=28.0 size=       1kB time=00:00:05.54 bitrate=   1.1kbits/s speed=0.254x    
frame=  124 fps=5.5 q=28.0 size=       1kB time=00:00:05.67 bitrate=   1.1kbits/s speed=0.253x    
frame=  126 fps=5.4 q=28.0 size=     257kB time=00:00:05.76 bitrate= 365.1kbits/s speed=0.246x    
frame=  129 fps=5.3 q=28.0 size=     257kB time=00:00:05.88 bitrate= 357.2kbits/s speed=0.244x    
frame=  131 fps=5.2 q=28.0 size=     257kB time=00:00:05.97 bitrate= 352.1kbits/s speed=0.239x    
frame=  132 fps=4.0 q=28.0 size=     257kB time=00:00:06.01 bitrate= 349.6kbits/s speed=0.181x    
frame=  136 fps=4.0 q=28.0 size=     257kB time=00:00:06.18 bitrate= 340.0kbits/s speed=0.182x    
frame=  137 fps=3.9 q=28.0 size=     257kB time=00:00:06.22 bitrate= 337.6kbits/s speed=0.177x    
frame=  141 fps=3.9 q=28.0 size=     513kB time=00:00:06.40 bitrate= 656.3kbits/s speed=0.179x    
frame=  143 fps=3.9 q=28.0 size=     513kB time=00:00:06.46 bitrate= 649.8kbits/s speed=0.176x    
frame=  147 fps=3.9 q=28.0 size=     513kB time=00:00:06.63 bitrate= 633.1kbits/s speed=0.176x    
frame=  152 fps=4.0 q=28.0 size=     769kB time=00:00:06.84 bitrate= 919.6kbits/s speed=0.178x    
frame=  154 fps=3.9 q=28.0 size=     769kB time=00:00:06.93 bitrate= 908.3kbits/s speed=0.174x    
frame=  157 fps=3.9 q=28.0 size=     769kB time=00:00:07.06 bitrate= 891.8kbits/s speed=0.175x    
frame=  158 fps=3.8 q=28.0 size=     769kB time=00:00:07.10 bitrate= 886.5kbits/s speed=0.173x    
frame=  161 fps=3.8 q=28.0 size=     769kB time=00:00:07.23 bitrate= 870.8kbits/s speed=0.172x    
frame=  164 fps=3.8 q=28.0 size=    1025kB time=00:00:07.36 bitrate=1140.6kbits/s speed=0.172x    
frame=  169 fps=3.9 q=28.0 size=    1025kB time=00:00:07.55 bitrate=1111.6kbits/s speed=0.174x    
frame=  171 fps=3.9 q=28.0 size=    1025kB time=00:00:07.63 bitrate=1099.2kbits/s speed=0.174x    
frame=  173 fps=3.9 q=28.0 size=    1025kB time=00:00:07.72 bitrate=1087.0kbits/s speed=0.173x    
frame=  178 fps=3.9 q=28.0 size=    1281kB time=00:00:07.93 bitrate=1322.1kbits/s speed=0.176x    
frame=  184 fps=4.0 q=28.0 size=    1281kB time=00:00:08.21 bitrate=1277.4kbits/s speed=0.18x    
frame=  188 fps=4.1 q=28.0 size=    1281kB time=00:00:08.34 bitrate=1257.8kbits/s speed=0.181x    
frame=  195 fps=4.2 q=28.0 size=    1537kB time=00:00:08.64 bitrate=1457.1kbits/s speed=0.185x    
frame=  201 fps=4.2 q=28.0 size=    1537kB time=00:00:08.89 bitrate=1415.1kbits/s speed=0.188x    
frame=  213 fps=4.4 q=28.0 size=    1537kB time=00:00:09.38 bitrate=1341.2kbits/s speed=0.195x    
frame=  219 fps=4.5 q=28.0 size=    1793kB time=00:00:09.64 bitrate=1523.0kbits/s speed=0.197x    
frame=  226 fps=4.6 q=28.0 size=    1793kB time=00:00:09.94 bitrate=1477.3kbits/s speed=0.201x    
frame=  234 fps=4.7 q=28.0 size=    1793kB time=00:00:10.26 bitrate=1431.2kbits/s speed=0.205x    
Version - v0.0.1-beta7~cb17eca
2018 © Unmanic by Josh Sunnex.

 

Link to post
21 hours ago, Cpt. Chaz said:

so do you leave any cpu pins in place when you're using this prioritization? or just give unmanic full cpu access with low priority and let her rip?

 

Do both. pin to where you want then change priorities 

Link to post

May be a little off topic, but i'm not sure where else to post this... hoping since my original unmanic instance is running on unraid, that'll buy me a little grace...

 

i've got an imac on the same LAN as my unraid. i keep the media library and cache directory in unraid. wanting to speed up library process time, i setup unmanic in docker on my mac, so i can use its processor too.

 

got the install done, made it to the web ui, but can't figure out how to set the library and cache directories from unmanic on the mac to the unraid library and cache. was thinking maybe i could bash into the container and set /library path and /cache path, but don't know enough about docker to do this, especially for remote file paths. 

 

anytime i try putting in some variable of [unraid volume]/media/plex/tv shows in the web ui's library field, i get "WD=-1, Errno=No such file or directory (ENOENT)" in the log and no change in the UI. Can anyone help me out?

Link to post

I'm having as issue where unmanic is attempting to convert files that have failed continuously.  Can we get an option for number of attempts to convert or maybe a black list.  I'm thinking a blacklist would be best.  At this point I can't use it anymore as it is trying to convert files that always fail.

Link to post
2 hours ago, discojon said:

I'm having as issue where unmanic is attempting to convert files that have failed continuously.  Can we get an option for number of attempts to convert or maybe a black list.  I'm thinking a blacklist would be best.  At this point I can't use it anymore as it is trying to convert files that always fail.

Do a force update. The latest version solved that issue for me.

Link to post
On 10/4/2019 at 9:51 PM, Josh.5 said:

Feature requests are welcome

FFMPEG Stripping the metadata using the -1 command , If you could add that to the settings im sure many people would find that useful as far as i have found so far no other batch re-encoding app has that option. im not coder so im not sure how hard it would to be to implement. but i know it save many people the time of striping metadata one by one.

Link to post
FFMPEG Stripping the metadata using the -1 command , If you could add that to the settings im sure many people would find that useful as far as i have found so far no other batch re-encoding app has that option. im not coder so im not sure how hard it would to be to implement. but i know it save many people the time of striping metadata one by one.
Feature requests are welcome. You should raise an issue for it on GitHub. Issues posted here in the forum are easy to lose track of and forget about.

Sent from my ONEPLUS A6000 using Tapatalk

Link to post

One issue I've noticed after point unmanic to my entire tv shows library is that the dashboard page becomes unresponsive. Tested on both firefox and chrome. I've queued up several hundred jobs, all of which are trying to be displayed on the dashboard which the browser cannot handle. The conversions continue on just fine and the work around is to check progress on the history page.

Link to post

Think this might be my first post here.. but I want to say thank you for building this docker. I've been running it the last couple of days and it works just fine.

 

Is there any way you can build notifications into this in the future? A handful of my apps have webhooks into Slack where I get all my notifications. I wouldn't mind getting notifications on when a video file is finished, and how much % of space was saved.

Link to post
On 10/21/2019 at 2:17 PM, SRB said:

One issue I've noticed after point unmanic to my entire tv shows library is that the dashboard page becomes unresponsive. Tested on both firefox and chrome. I've queued up several hundred jobs, all of which are trying to be displayed on the dashboard which the browser cannot handle. The conversions continue on just fine and the work around is to check progress on the history page.

I'm having this problem too. Reducing number of pinned cores doesn't seem to affect this, nor does reducing priority. I've tried Safari and Firefox.

Link to post
  • 2 weeks later...

Feature request: I know you are busy getting this ready for prime time. I see we now have failures appearing in the failure tab. Fantastic as I can now see subs have been causing some fails because ffmpeg can't transcode them. It would be nice if after something fails we could change the settings for an individual file without changing for every files or having to change around folders to not affect other files. Hope this makes sense and is possible thank you for your great work!

 

Sent from my Pixel 2 XL using Tapatalk

 

 

 

 

Link to post

Hey, so i just discovered this docker and I had a few questions about it.  It has a feature that I really like (being able to scan my library and identify which files are in 264 and automatically queue them for transcode to 265).

But I'm curious as to how well it does of the transcode job.  Right now, I'm using handbrake.  I'm in the process of converting a lot of my 4k movies into x265, but I have things such as my RF set to specific settings depending if I'm doing 4k movies or tv shows, etc.  I see the ability to set it (Unmanic) to use x265 for transcoding and keep my items mp4, but how well does it do in terms of quality, bitrate, variable framerate, etc?  Is it a really good alternative, compared to just manually checking every media file I have, and adding it to my handbrake queue?  (Keep in mind, I have about 25,000 pieces of media between my movies and tv shows).

 

Also, when it scans my tv shows, once it identifies a file that needs to be queued for transcode, once the file is transcoded, does it delete the old file and replace it with the new freshly transcoded file?

 

Lastly, I know I can simply add as many /library/***whateverfolder*** and it'll scan all subfolders.  But if my plex is set up with the following structure /media/(tv shows, movies, 4k movies, 4k tv shows), couldn't I just set my library path to /media  and it would automatically scan all subfolders or do I actually need to create a library path for every subfolder?

Edited by Meller
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.