[Support] Josh5 - Unmanic - Library Optimiser


867 posts in this topic Last Reply

Recommended Posts

1 minute ago, rmeaux said:


One worker, one file. One file, one worker. Also multiple workers do not speed up your productivity IMO. Stick with one.

Quality is great for me. Been a while since I’ve used handbrake. I could never get the same file reduction. I also don’t notice a diff between CPU and GPU converted quality.

Try a few files. I have 10GB movies that dropped to 800MB. I also have 10GB movies that dropped to 9GB. There are a few factors that come into play with how much can be done to a file. unManic took my “Supernatural” folder from 1.1TB to 86GB. But it actually was increasing my “Friends” folder while I monitored it.


Sent from my iPhone using Tapatalk

I'm transcoding everything into h264.

 

There's 5 video encoder options for h264. Is one better then the other? Which one would be the better one to use?

 

Thanks

Link to post
  • Replies 866
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Application Name: UNMANIC - Library Optimiser Application Site: https://github.com/Josh5/unmanic/ Docker Hub: https://hub.docker.com/r/josh5/unmanic/ Github: https://github.com/Josh5/un

There should be a new update available. This has quite a number of changes to how ffmpeg is executed. It should resolve some issues with inotify and library scanning. For those people who created

@Ashe @trekkiedj @itimpi   Please see update first post also for an example of how I have it setup. I have also added a temporary solution to adding multiple library locations if anyone need

Posted Images

I'm transcoding everything into h264.
 
There's 5 video encoder options for h264. Is one better then the other? Which one would be the better one to use?
 
Thanks

Well I may be wrong but the ones that have “nvenc” in their their title definitely need a Nvidia GPU to function. (?? or QuickSync CPU ??)

libx264rgb is the same thing as libx264 but has RGB pixel formats as input instead of YUV. That’s from Google. Can’t give you more info than that. Not my field of expertise.


Sent from my iPhone using Tapatalk
Link to post
6 minutes ago, rmeaux said:


Well I may be wrong but the ones that have “nvenc” in their their title definitely need a Nvidia GPU to function. (?? or QuickSync CPU ??)

libx264rgb is the same thing as libx264 but has RGB pixel formats as input instead of YUV. That’s from Google. Can’t give you more info than that. Not my field of expertise.


Sent from my iPhone using Tapatalk

 

A few posts back I asked if QuickSync is supported, because it is mentioned in the settings, but no one answered. 

 

Also if Unmanic is based on FFMPEG, it should also support QS. 

 

Nvenc is for Nvidia and HEVC is for x265. 

Link to post

So despite the lack of ability to actually configure any encoder settings / resolutions / etc I've been running this between my Unraid docker and Windows with Docker Desktop (does not work 100% accurately, but it works well enough to make use of it) and feeding it a ton of media files that I wanted to see the results on.   I'm fairly impressed with the results.  I've converted around 300GB+ of media files to a fraction of that and the quality appears to be very good.

 

I have not figured out a way to force a conversion of media files that Unmanic thinks is already in the correct format.  Just because it might be in the same format does not mean that I don't want to try to convert it for space saving reasons.

 

@Josh.5 option to force possible?

Link to post
16 hours ago, Energen said:

So despite the lack of ability to actually configure any encoder settings / resolutions / etc I've been running this between my Unraid docker and Windows with Docker Desktop (does not work 100% accurately, but it works well enough to make use of it) and feeding it a ton of media files that I wanted to see the results on.   I'm fairly impressed with the results.  I've converted around 300GB+ of media files to a fraction of that and the quality appears to be very good.

 

I have not figured out a way to force a conversion of media files that Unmanic thinks is already in the correct format.  Just because it might be in the same format does not mean that I don't want to try to convert it for space saving reasons.

 

@Josh.5 option to force possible?

Maybe I'm wrong but this isn't trying to change the video quality, just converting from x to y codec and possible file format. Meaning even if you ran through something that's already in the right file format and codec, it wouldn't change size.

Link to post
On 9/28/2020 at 12:47 PM, chiefo said:

Maybe I'm wrong but this isn't trying to change the video quality, just converting from x to y codec and possible file format. Meaning even if you ran through something that's already in the right file format and codec, it wouldn't change size.

So this goes back to what I originally said about not knowing what quality settings Unmanic is hard-coded with.  Doesn't matter what codec or format is used if the quality settings are different.

 

If you take two original videos and encode one at x265 4K and one at x265 1080p, and then use an automated process like Unmanic to convert those videos.. the logic that Unmanic uses says that both of those files are already in the correct format and are ignored.  Do you get what I mean by quality settings?  In the mean time that 30GB 4K video could have been reduced to 2GB.

 

Quote

 


DEBUG:Unmanic.FFMPEGHandle - [FORMATTED] - File already in container format mkv - /library/B (2008).mkv
DEBUG:Unmanic.FFMPEGHandle - [FORMATTED] - File already has hevc codec video stream - /library/B (2008).mkv
INFO:Unmanic.LibraryScanner - [FORMATTED] - File does not need to be processed - /library/B (2008).mkv
INFO:Unmanic.LibraryScanner - [FORMATTED] - Ignoring file due to already correct format - 'B (2008).mkv'
 

 

 

In the case of Unmanic -- this B (2008).mkv file is ignored..  it's a 1.6GB file.  If Unmanic actually processed it at the quality settings it's configured for........ would the ending file be smaller?  That's what I'm driving at...  the question still remains what 'quality' setting Unmanic uses for the codec it uses.  Or are the files skipped because Unmanic actually compares the encoder settings and sees that it's the same? 

 

Another example, in Handbrake if you select x265 1080p but one with an avaerage bitrate of 3000 and another with 500, they are technically the same format/codec but obviously different quality.  The comparison of codec only would make both of these files the same.

Edited by Energen
Link to post
  • 2 weeks later...

EDIT: Never mind i got it ...to everyone else running into this; don't add the variable via SpaceInvaderOne style just read and carefully follow the instructions at the top 

 

Hello all i am trying to install Unmanic and i am running into the following error when i click apply.

"usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused "process_linux.go:432: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: device error: unknown device id: false\\n\""": unknown. "

 

I am running Nvidia-unRAID version 6.9.0 beta_22 and i have an older Nvidia GeForce GTX980TI i am declaring the extra parameters and the NVIDIA_VISIBLE_DEVICES with the one Gpu UUID that i have but i get that error still. I use that gpu with handbrake and plex and even once with tdarr with no errors or problems everything works fine for those containers.

what am i doing wrong ?? is my video card even capable of doing a decent job of converting or helping to h265 so i can save some space or not really, the cpu will be the one to do the work.

 

thank you 

nvidiaUnmanic.png

Edited by richland007
Link to post

Now that i got it installed how do you get it working LOL :)

I do not see anything on the setting where the scheduler may be??... and again how do i add files to encode??

the folders are assigned the right way i am sure of it...does it pick up the movies automatically??

 

thank you

Link to post
@Energen did you ever find out the encoding settings Unmanic uses? Your questions/concerns are mine too; what's the output quality?
I'm converting the first file so I'm yet to compare the output, it sounds like (from your comments) the quality is very good (hopefully indistinguishable in a blind test?)
 
 
I don't know the specific quality settings it uses. I suppose if I really wanted to figure it out I could dig through the source code but I'm not sure if that would really gain me anything, other than just knowing.

My experience is that 95% of my conversions the output file was of a smaller size with no perceivable visual differences upon a quick check on my 24" 1080p LCD using Potplayer on my PC. The other 5% resulted in a larger file then the original, but no quality loss.

When I refer to quality loss I mean that I don't see any artifacts or blurring that would indicate a low bitrate encode. I do not mean that I'm taking a 4K source and converting it to a fraction of the size and it still looks like 4K, know what I mean?



Sent from my SM-G981U1 using Tapatalk

Link to post
26 minutes ago, Energen said:

I don't know the specific quality settings it uses. I suppose if I really wanted to figure it out I could dig through the source code but I'm not sure if that would really gain me anything, other than just knowing.

My experience is that 95% of my conversions the output file was of a smaller size with no perceivable visual differences upon a quick check on my 24" 1080p LCD using Potplayer on my PC. The other 5% resulted in a larger file then the original, but no quality loss.

When I refer to quality loss I mean that I don't see any artifacts or blurring that would indicate a low bitrate encode. I do not mean that I'm taking a 4K source and converting it to a fraction of the size and it still looks like 4K, know what I mean?



Sent from my SM-G981U1 using Tapatalk
 

 

Awesome, I just checked the first file it converted for me and sadly wasn't happy with the result, there was a lot of smearing around moving heads and such. The file size did come down considerably, from 8gb or 1gb but the loss in quality for me was a little too much. I'd prefer a 4gb file with much higher quality. I'll wait for an update where we can set the quality!

Link to post
  • 2 weeks later...

I love this tool. it's a lot cheaper to do this (long term) than to keep buying ever more HDD's. 

 

On thing I've run into is errors and videos not transcoding properly and how difficult it is to re-queue a failed object. I've had to go an remove the container, delete the database and redownload.

 

I've love to see an option to re-queue all failed object or to "initialize" or wipe the database on next restart.

Link to post
I love this tool. it's a lot cheaper to do this (long term) than to keep buying ever more HDD's. 
 
On thing I've run into is errors and videos not transcoding properly and how difficult it is to re-queue a failed object. I've had to go an remove the container, delete the database and redownload.
 
I've love to see an option to re-queue all failed object or to "initialize" or wipe the database on next restart.
You only need to delete the contents of the appdata folder for unmanic to clear out any history.

Sent from my SM-G981U1 using Tapatalk

Link to post

Started running into this since last night: 

 

[matroska,webm @ 0x558601613300] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options [matroska,webm @ 0x558601613300] 

 

Everything is failing.

Link to post
2 hours ago, Energen said:

You only need to delete the contents of the appdata folder for unmanic to clear out any history.

Sent from my SM-G981U1 using Tapatalk
 

Well yes, that is one option, but then I have to go set up all my settings again. My method preserves settings.

Link to post
Started running into this since last night: 
 
[matroska,webm @ 0x558601613300] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options [matroska,webm @ 0x558601613300] 
 
Everything is failing.
You have to turn off subtitles encoding.

Sent from my SM-G981U1 using Tapatalk

Link to post
Well yes, that is one option, but then I have to go set up all my settings again. My method preserves settings.
So if that's what you'd rather do then ok, but it's certainly more hassle then setting the very few options there are.

Also you can use your imagination and only delete the unmanic.db file and retain the settings.json file so you accomplish both deleting history and retaining your settings. I'm sure you could have figured that out had you looked at the directory first.

Sent from my SM-G981U1 using Tapatalk

Link to post
19 minutes ago, Energen said:

So if that's what you'd rather do then ok, but it's certainly more hassle then setting the very few options there are.

Also you can use your imagination and only delete the unmanic.db file and retain the settings.json file so you accomplish both deleting history and retaining your settings. I'm sure you could have figured that out had you looked at the directory first.

Sent from my SM-G981U1 using Tapatalk
 

 

I am with eagle, would be nice and easier for all to have this settings. 

 

If the settings should be easy to implement, I don't understand why you should not do it, but okay, maybe it is not easy. 

 

That was just meant as a suggestion for improvement.

Link to post
2 hours ago, aurevo said:

 

I am with eagle, would be nice and easier for all to have this settings. 

 

If the settings should be easy to implement, I don't understand why you should not do it, but okay, maybe it is not easy. 

 

That was just meant as a suggestion for improvement.

I agree, but there hasn't been any development on this in over a year so we have to make do with what we have...

Link to post
2 hours ago, Energen said:

Also you can use your imagination and only delete the unmanic.db file and retain the settings.json file so you accomplish both deleting history and retaining your settings. I'm sure you could have figured that out had you looked at the directory first.
 

This is exactly what I said I am doing.

Link to post
29 minutes ago, eagle470 said:

This is exactly what I said I am doing.

I see this is going to be never ending... just take my point and accept it and move on.  You said "I've had to go an remove the container, delete the database and redownload." ... when you ONLY need to delete the database, you don't have to remove or redownload the container.  Stop container, delete database, start.  Finished.   So no, not "exactly" what you said you were doing.

 

And my mistake, I don't look at github just to prove a point.  The last time I looked at the github page nothing had been updated in quite some time.  You certainly wouldn't know of any developmental changes by the lack of communication or responses by the dev regarding the multitiudes of bugs/suggestions in this thread.   However, the last docker update, which is the only thing that actually matters, was 6+ months ago.  Let me rephrase for accuracy so you don't point it out to try and prove me wrong, the "latest" tag is still 6 months old.

 

Anyways, enjoy.

Edited by Energen
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.