[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

1 minute ago, xlucero1 said:

To hopefully wrap this up here then; I'm sure everyone has GPU passed thru to their VM. The Nvidia Driver plugin says dont install it if you pass thru GPU to VM. So how do I do it? Download the plugin anyway? VM and Unmatic should still work or no? 

 

What they mean by don't install the drivers if you are using VM passthrough, is that it will not help. If you passthrough your GPU to a VM, you CANNOT use your GPU with any Docker containers.

 

If you need both, you will need to have two GPU's installed in your unRAID server. This is a hardware limitation and has nothing to do with unRAID on any of the app developers.

 

I have an inexpensive GPU for my WIndows 10 VM and a better GPU for my transcoding. So, one is passed through and the other is used in two Docker containers.

 

Note: If you do have two and one of them is passed through, only the GPU that is not passed through will appear in your Nvidia drivers list as you can see from my screenshots.

 

image.thumb.png.d537ac8107d88fdadae6ef1135f4054d.png

 

image.png.6c59f73e0a5948768b69db32c44873cf.png

  • Upvote 2
Link to comment

Seems like you pretty much have to have 2 GPUs to use Unmatic then; assuming, most people using Unraid pass through their GPU to VM. Otherwise what you said, makes perfect sense. Thank you for explaining that. 
 

I was looking for alternatives to get this going. 

- Either install a 2nd GPU, 

- Or Maybe I can host Unraid on another computer (with another GPU) and encode all videos remotely; unless I can use a windows desktop and this?  

Thank you

Link to comment
44 minutes ago, xlucero1 said:

Seems like you pretty much have to have 2 GPUs to use Unmatic then; assuming, most people using Unraid pass through their GPU to VM

or using their intel igpu for unraid transcodings instead ... plex, unmanic, ... probably most common

 

you could also use unmanic with nvidia while your VM is turned off as note, but that needs a system (hardware) which can handle it

 

- no need to bind GPU in vfio bind and VM is starting, stopping properly (so unraid can run really headless, not always working)

- time what you do, your system can crash while starting a vm while a docker is using the needed GPU or vice versa

...

Link to comment

You do not need a GPU to use Unmanic. That is a pretty outrageous statement to make.

 

Using a GPU inside the official Unmanic Docker container is an optional thing that you can do if you wish. You can also use Unmanic on the VM that you have passed your GPU into if you like.

 

Unmanic is a tool with a lot of capabilities of which only a very small sub-set would even have the option to take advantage of a GPU.

  • Like 2
Link to comment
On 11/13/2022 at 5:27 PM, xlucero1 said:

I'm sure everyone has GPU passed thru to their VM. The Nvidia Driver plugin says dont install it if you pass thru GPU to VM. So how do I do it? Download the plugin anyway? VM and Unmatic should still work or no? 

 

What you can do is stop the Unmanic container prior to launching your VM and vice versa — stopping your VM prior to launching Unmanic. This way you don't crash anything. If you pause Unmanic during an encoding process and then launch your VM, the GPU may not be released to the VM resulting in issues.

 

Like @Josh.5 said, you don't have to passthrough your GPU to Unmanic; Unmanic can still encode using CPU.

 

@Eddie Seelke It is possible to run a single GPU in Unraid and also pass it through to your VM. You would have to pull the firmware from the GPU card and then load it into your VM. I had very limited success with it a few year ago with a GTX 1060. In the end, I spung for a cheap GT 710 to make Unraid happy even as a headless server.

Edited by Zer0Nin3r
Typos & clarity
Link to comment
  • 2 weeks later...

I've been a Tdarr user for a while now and have modified a few plug-ins for it to make Tdarr do what I want.  Now, I've come across unmanic and I like the interface design much better.  

 

I have a library of mostly hevc files most are 10 bit, but many are still 8 bit.  I'm trying to convert all my media to hevc 10 bit to support direct streaming for Apple Tvs.  I'm currently using the Video Encoder H265/HEVC - libx265 (CPU) encoder.  It does fine encoding anything that is not HEVC into HEVC 10 bit.  I am using custom settings of the encoder with these settings "-map 0 -dn -c:v libx265 -preset veryfast  -pix_fmt p010le -a53cc 0 -c:a copy -c:s copy -max_muxing_queue_size 999".  

 

The problem I have is that it does not re-encode the hevc 8 bit to 10 bit.  Is there anyway to force it to?

Link to comment

Is persistent load from the unmanic-service process expected?

 

Pausing workers (which were already inactive) and disabling file monitors didn't affect it, and there are no entries in the log (even with debug enabled) indicating activity.

 

top within the container:

783633949_Screenshot2022-12-05at8_52_34AM.thumb.png.9891feca92ef90e1704b75f7e582731a.png

 

top in unraid:

917695839_Screenshot2022-12-05at8_53_16AM.thumb.png.53255399a6d6e6ebb2e3f93a760b554a.png

 

 

 

Link to comment
  • 2 weeks later...
  • 3 weeks later...

There is a lot of info in this post and I must have missed it.

 

What is the CRF that is the most standard to change from 264 to 265? I tried a setting, and it made a 1.7 GB video 211 MB.  I don't want it that small, so I think I need to adjust the CRF some more. When watching the film by Josh, he had it at 29 or so, That seems just too much.

Is 23 the default setting that will change the size, but not as much as the last experiment did?

Thanks.

Link to comment
  • 4 weeks later...
6 hours ago, TexasDave said:

Is there a "compromise" (or ideas) on how I can increase my bitrate? Thanks!!

 

increasing the bitrate, may just look at the plugin settings you are using, slider or value, as lower as higher ...

 

looks really like bad bonding on the upper sample picture ... while looking at your picture, may also check your TV settings as the black bars are pretty light too (could be now also only the photo) .... may helps a little vs. the bonding.

 

in general, a 3,8kb hevc should be more then enough for a 1920x800 stream, so overall ... are you using nvenc (nvidia) or vaapi / qsv (intel), may show your settings from there here so others rather can help ...

 

and also as note, surfaces with transitions like your space picture up there, also sky (blue) from light to lighter or little darker are really hard to encode ... and alot of TV's needs a good setup to avoid it ...

 

and some players may even dont like hevc with 8bit bt709 color range ... may even try it on another device just to make sure.

 

im not using this container (only testing purposes sometimes) as i have some more preferences which this app cant handle, but as its more or less simple ffmpeg based you should see the settings in the webui and when i remember correctly in the logs you even see the exact ffmpeg cmd string. this will help more then the results from plex mediainfo.

  • Like 1
Link to comment

I am using Video Encoder H265/HEVC - hevc_nvenc (NVIDIA GPU)

Command is the following:
 

ffmpeg -hide_banner -loglevel info -i /library/tv/Star Trek - Discovery/Season 4/Star Trek - Discovery - S04E12 - Species Ten-C.mkv -strict -2 -max_muxing_queue_size 2048 -threads 4 -map 0:v:0 -map 0:a:0 -map 0:s:0 -map 0:s:1 -map 0:s:2 -map 0:s:3 -map 0:s:4 -map 0:s:5 -map 0:s:6 -map 0:s:7 -map 0:s:8 -map 0:s:9 -map 0:s:10 -map 0:s:11 -map 0:s:12 -c:v:0 hevc_nvenc -profile:v:0 main -preset medium -c:a:0 copy -c:s:0 copy -c:s:1 copy -c:s:2 copy -c:s:3 copy -c:s:4 copy -c:s:5 copy -c:s:6 copy -c:s:7 copy -c:s:8 copy -c:s:9 copy -c:s:10 copy -c:s:11 copy -c:s:12 copy -y /tmp/unmanic/unmanic_file_conversion-jotzv-1675293877/Star Trek - Discovery - S04E12 - Species Ten-C-jotzv-1675293877-WORKING-1-1.mkv


I think everything is working fine. I am assuming that if you compress, you lose bitrate, I am just wanting to try to see if I can get it a bit better. I do not mind some increased size if I can get rid of the black pixelization.

Link to comment
35 minutes ago, TexasDave said:

Command is the following:

ok, so nvenc, my personal experience is also that nv is a little worse then intel vaapi / qsv

 

you are currently only using preset medium (3rd line here now, extracted from your lineup)

ffmpeg -hide_banner -loglevel info -i /library/tv/Star Trek - Discovery/Season 4/Star Trek - Discovery - S04E12 - Species Ten-C.mkv -strict -2 -max_muxing_queue_size 2048 -threads 4 \
-map 0:v:0 -map 0:a:0 -map 0:s:0 -map 0:s:1 -map 0:s:2 -map 0:s:3 -map 0:s:4 -map 0:s:5 -map 0:s:6 -map 0:s:7 -map 0:s:8 -map 0:s:9 -map 0:s:10 -map 0:s:11 -map 0:s:12 \
-c:v:0 hevc_nvenc -profile:v:0 main -preset medium \
-c:a:0 copy -c:s:0 copy -c:s:1 copy -c:s:2 copy -c:s:3 copy -c:s:4 copy -c:s:5 copy -c:s:6 copy -c:s:7 copy -c:s:8 copy -c:s:9 copy -c:s:10 copy -c:s:11 copy -c:s:12 copy \
-y /tmp/unmanic/unmanic_file_conversion-jotzv-1675293877/Star Trek - Discovery - S04E12 - Species Ten-C-jotzv-1675293877-WORKING-1-1.mkv

 

how i get some better results with nvenc with some more settings

image.thumb.png.219c354d11d61cdfd56ac178c23392fd.png

 

may take a look when you expand unmanic and see where to put what ...

 

rest, others may can give you some hints howto use nvenc best ... personally i switched back to qsv, slower but the results are slightly better here in my terms ...

 

here a speed comparision, 1st encode was nvenc, 2nd qsv, 3 min vs. 11 min ;)

Sat Jan 28 07:59:18 CET 2023 Interview With the Vampire - S01E07 - Das Ding lag still.mkv is h264 formatted, transcoding and crop to 960, P_F_25 now .. Nr. 1
Sat Jan 28 08:02:33 CET 2023 Interview With the Vampire - S01E07 - Das Ding lag still.mkv replaced, from 1713 mb to 797 mb, cropped to 960, P_F_25, spared 916 mb
Wed Feb 1 23:15:08 CET 2023 Navy CIS - S20E01 - Fluchtpunkt Hawaii, Part.mkv is h264 formatted, transcoding and crop to 1080, P now .. Nr. 1
Wed Feb 1 23:26:45 CET 2023 Navy CIS - S20E01 - Fluchtpunkt Hawaii, Part.mkv replaced, from 2836 mb to 1361 mb, cropped to 1080, P, spared 1475 mb

 

this all just as startpoint, when you want to go deeper, look online for nvenc ffmpeg extra and howto ...

 

something to read ...

https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/

  • Like 1
Link to comment

OK - So I know it is far from scientific but this is what I am seeing.

I ran four samples on Plex (Original, GPU-Medium, GPU-Slow, CPU-Medium). GPU is a 1050Ti..... GPU is definitely faster than CPU. About 7-8x (5 minutes v 40 minutes).

Quality on GPU is about the same between SLOW and MEDIUM.

GPU produces a better bit rate which seems to be my issue.

I attach a few screen shots. Like I say, I know it is far from scientific.

I am running Plex on unRAID, through Shield TV and on to a Hisense Roku TV which is decent but certainly not high end.

I know there is a bitrate flag and I may try bumping that up.

Even on the original there is some minor pixelation. The photos make it look worse than it is. But I can tell a difference between the original and the ones that have been processed.

Changing the brightness on the TV from Brightest to Dark made the most difference. 🙂

Besides bumping the bitrate up - any other ideas?

I think I will just live with it. I am on an average TV in average viewing conditions. Friends that access Plex do not complain so...

Summary.jpg

TV Compare.jpg

Link to comment
  • 2 weeks later...

I'm using unmanic to remove unwanted audio streams and the issue is that it copies everything to the temp cache and then back to the array, regardless of whether any work needs to be done. So even for files that have no audio tracks to be removed. So if I let it rip through my library it will essentially copy-paste like 50TB of data which is obviously not desirable.

 

Did I misconfigure something or is this the expected behavior?

 

Is there some way we can analyse files without doing this copy+write action? Just read the files in their current location?

Link to comment
  • 3 weeks later...

I am getting the following error

 

python3[17996]: segfault at 0 ip 0000000000000000 sp 000014622acb1898 error 14

 

This error is showing when unmanic is running. Don't know what is causing it. All was working perfectly. Then one day this error stared to appear in my log. Error is there only when unmanic is running.

 

After a while it crashes my server

 

I stopped the container and the error is no longer there. A shame cause I really like unmanic.

 

Any idea on how to fix this ?

 

Link to comment

I had a h264 file which size was about 2 GB, after reencode to h265 it was about 25GB big. What's wrong?

 

These were my settings:

 

image.thumb.png.5046e9bc12db32e53906e97359ebbd76.png

image.thumb.png.8a69b7b7199f7045ac4fea6b48fa5c0b.png

 

I have an Nvidia GPU RTX A2000.

 

I just wanna convert h264 into h265 without big quality loss.

Edited by enJOyIT
Link to comment
53 minutes ago, alturismo said:

image.png.f7cd371e330c002bbcd37dfec14ae85c.png

 

there is your reason ...

 

im pretty confident with 19 compared originnal h264 > hevc, on RTX i would go to ~ 17 and test some Videos and check yourself.

 

as lower as higher the quality, lossless ... is not the way to go ;)

 

Do I have to use my own parameters? Because there are only 4 options here:

image.thumb.png.53ddc7e98ea9e43d8def610dbffa8520.png

 

May you can share your settings? That would be great 🙂

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.