[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

6 hours ago, Aerodb said:

I assumed this issue was the same for me. I implemented the strip images plugin as you advised(first in the worker queue). I had the same issue when i re-added the job that had previously failed. log attached. 

 

Please let me know if you have any troubleshooting steps that i should take. 

Failed Job 9.24.21.txt 6.56 kB · 1 download

it is the subtitle stream in that file that is failing. It this point in time the only available solution is removing those subtitiles. You can extract them to external srt files if you really want to keep them.

I may look into creating an all-in-one plugin for mp4 files due to the container's limited support for streams.

  • Thanks 1
Link to comment
5 hours ago, CyrixDX4 said:

wait is THAT all it does? I thought there was some images put in to specific movies that display subtitles or other images on the screen (John Wick movie when they go into talking about Keana being Baba Yaga, etc.).

 

 

Yea. There is another plugin that you are confusing it with that removes image type subtitles. Those are "subtitle" streams. The plugin that I mentioned removes image "video" streams which are things like embedded thumbnails - which you really dont need if you are using Plex/Kodi/Emby/Jellyfin.

Link to comment
35 minutes ago, reggierat said:

I've been testing this out on a new series folders and wondering before i let it loose on my entire tv folder, is it possible to exclude folders ? there's a few tv series i'd prefer to leave as is for the time being

 

Also for general tv stuff, how do people feel about H265/HEVC media, slow and lossless settings? Should also mention i'm viewing on a 120" screen

Install the "Path Ignore" plugin. That can be configured to ignore any number of paths/files containing matching regex.

The plugin has a few regex examples to get you started.

Link to comment
3 hours ago, Josh.5 said:

it is the subtitle stream in that file that is failing. It this point in time the only available solution is removing those subtitiles. You can extract them to external srt files if you really want to keep them.

I may look into creating an all-in-one plugin for mp4 files due to the container's limited support for streams.

to be sure i understand your meaning... would this be an issue if I just switched to MKV containers rather than MP4?

Link to comment
3 hours ago, Aerodb said:

to be sure i understand your meaning... would this be an issue if I just switched to MKV containers rather than MP4?

Yes and no. MKV has better support for a wider range of stream codecs (as far as I am aware). But you will still have the exact same issue if you, for example, then had to convert a MP4 with mov_text subtitles into a MKV (which does not support mov_text subtitles).

 

I think there are a few fringe cases where a plugin that handles all 3 - video, audio and subtitle - streams would be worth having. For example, someone mentioned this week that WMV is not supported in the new version of Unmanic. And they are right. WMV has it's own set of codecs not compatible with the real world. So to transcode and remux that, we need to do audio and video at the same time.
I think that having a "AIO - MP4 (X264 | X265 - AAC)" Plugin (with some config to limit by file extension) is worth creating for exactly these cases. Because it is specific to remux to MP4, it can be written to encode all text subs to mov_text if it detects them.

 

Link to comment
54 minutes ago, Josh.5 said:

Yes and no. MKV has better support for a wider range of stream codecs (as far as I am aware). But you will still have the exact same issue if you, for example, then had to convert a MP4 with mov_text subtitles into a MKV (which does not support mov_text subtitles).

 

I think there are a few fringe cases where a plugin that handles all 3 - video, audio and subtitle - streams would be worth having. For example, someone mentioned this week that WMV is not supported in the new version of Unmanic. And they are right. WMV has it's own set of codecs not compatible with the real world. So to transcode and remux that, we need to do audio and video at the same time.
I think that having a "AIO - MP4 (X264 | X265 - AAC)" Plugin (with some config to limit by file extension) is worth creating for exactly these cases. Because it is specific to remux to MP4, it can be written to encode all text subs to mov_text if it detects them.

 

I see your point and it makes sense. I'm not really attached to one container or another, the same for subs being within the container or standalone files. I was mostly using unmanic to have a uniform library in 265 for added storage compression. anything else is an added extra, but I'm excited to see what else I can use unmanic for. 

Link to comment
4 minutes ago, Aerodb said:

I see your point and it makes sense. I'm not really attached to one container or another, the same for subs being within the container or standalone files. I was mostly using unmanic to have a uniform library in 265 for added storage compression. anything else is an added extra, but I'm excited to see what else I can use unmanic for. 

Honestly then I would recommend purging the subs. They will cause you issues when switching between containers. You are better off DL your own subs.

  • Thanks 1
Link to comment

Thnx , I’ll be testing later today.

Other question: does unmanic skip m4v files?
I have a lot of those and the old version of unmanic picked them up and converted them. The new version seems to skip them entirely.
Am I missing something?
I don't have any m4v files. So can't test anything for you, but I can say that I'm confident you are missing something :).

Unmanic does not skip any files. It scans everything unless configured not to.

What plugins are you running?
Link to comment
I don't have any m4v files. So can't test anything for you, but I can say that I'm confident you are missing something smile.png.

Unmanic does not skip any files. It scans everything unless configured not to.

What plugins are you running?

Just the basics
5bd56e4dbccf50e2317d96af65f6bdaf.jpg
And yes, m4v is included in the file extensions.
I will try to run with out the extension plugin later today.
Any log files a can look at to see why m4v does not get picked up? Have debug turned on, but no log entry on m4v.
Link to comment

Just the basics
5bd56e4dbccf50e2317d96af65f6bdaf.jpg
And yes, m4v is included in the file extensions.
I will try to run with out the extension plugin later today.
Any log files a can look at to see why m4v does not get picked up? Have debug turned on, but no log entry on m4v.
Yea, there is log entries for each file with debugging enabled.

If the m4v file is already h265, it will be ignored with that config.
Link to comment
Yea, there is log entries for each file with debugging enabled.

If the m4v file is already h265, it will be ignored with that config.

Omg. Ashamed now. Completely missed the m4v error. Sorry about that.
4b25f7773e8ddaed56f5d5056a3b153f.jpg
I am pretty sure the same container files did get processed by the old unmanic.
Not sure why it has difficulty fetching the mime type.
I’ll check later if cpu encoding plug-in gives the same error.
Link to comment

Omg. Ashamed now. Completely missed the m4v error. Sorry about that.
4b25f7773e8ddaed56f5d5056a3b153f.jpg
I am pretty sure the same container files did get processed by the old unmanic.
Not sure why it has difficulty fetching the mime type.
I’ll check later if cpu encoding plug-in gives the same error.
The old version of unmanic only picked up files my extension.
This one, the plugins detect the mimetype from the extension to decide what it is.

It may be worh looking into why a m4v file is not detectable and not considered a video by python... Is it officially a video file?
Link to comment
The old version of unmanic only picked up files my extension.
This one, the plugins detect the mimetype from the extension to decide what it is.

It may be worh looking into why a m4v file is not detectable and not considered a video by python... Is it officially a video file?

Yeps. I have many like these. Way back I used iFlicks to repack mkv to m4v. So iTunes could handle them.
If I run ffmpeg -i '{}' -vcodec copy -acodec copy '{}.mkv' to change from m4v to mkv. Then unmanic has no issue.
Manual ffmpeg detects meta data fine.
Output for ffmpeg -i Ant-Man.m4v -f ffmetadata in.txt is here https://pastebin.com/mvpCCMqW
Link to comment

Yeps. I have many like these. Way back I used iFlicks to repack mkv to m4v. So iTunes could handle them.
If I run ffmpeg -i '{}' -vcodec copy -acodec copy '{}.mkv' to change from m4v to mkv. Then unmanic has no issue.
Manual ffmpeg detects meta data fine.
Output for ffmpeg -i Ant-Man.m4v -f ffmetadata in.txt is here https://pastebin.com/mvpCCMqW
I'll look into is. Really not sure why mimetype is not detected.
Link to comment

Earlier versions had a mapping to library within the docker template.

 

Recent versions seem to have added a tv and movies mapping, which I need to keep deleting as they keep coming back. By default they map to places I don’t have (or want) which keeps creating them on my drive.

 

Is this a bug or an intended feature?

Edited by page3
Link to comment
1 minute ago, page3 said:

Earlier versions had a mapping to library within the docker template.

 

Recent versions seem to have added a tv and movies mapping, which I need to keep deleting as they keep coming back. By default they map to places I don’t have (or want) which keeps creating them on my drive.

 

Is this a bug or an intended feature?

Can you show me your docker template config?

Link to comment
On 9/20/2021 at 9:48 AM, JonathanM said:

It's certainly worked well for me so far, I guess I'll stay the course.

 

Totally unrelated, could you possibly post an example of what to put in the extra parameters of the audio encoder plugin to duplicate the "Create a stereo clone of streams" that the old version had?

 

I'll understand if you say RTFM. Just hopeful it's a quick addition to the "write your own custom audio options" section.

 

This plugin is read for testing in my development plugin repo...
https://github.com/Josh5/unmanic-plugins

image.thumb.png.b2534b551834d1fdab902a7b7a9abf3a.png

 

It works by reading the tags to detect if a stream is a stereo clone of a another stream. The tags generated will be "[Stereo]" if there is not title for that stream, or "English [Stereo]" if there was a title for the stream.
This plugin should also be compatible with detecting if a file was already processed by the legacy version of Unmanic.

 

You can configure options like bitrate etc in the advanced options (just like all the other ffmpeg plugins). I have added an example of how to do this.

 

Let me know if you have any issues.

Edited by Josh.5
  • Like 1
Link to comment
2 hours ago, Josh.5 said:

This plugin is read for testing in my development plugin repo...
https://github.com/Josh5/unmanic-plugins

image.thumb.png.b2534b551834d1fdab902a7b7a9abf3a.png

 

It works by reading the tags to detect if a stream is a stereo clone of a another stream. The tags generated will be "[Stereo]" if there is not title for that stream, or "English [Stereo]" if there was a title for the stream.
This plugin should also be compatible with detecting if a file was already processed by the legacy version of Unmanic.

 

You can configure options like bitrate etc in the advanced options (just like all the other ffmpeg plugins). I have added an example of how to do this.

 

Let me know if you have any issues.

 

Perhaps I'm doing something wrong, but I'm failing to add repositories whenever I attempt to use that button. I tried with the full link to the repository, as well as just Josh5/unmanic-plugins

fail to save.png

Link to comment
8 hours ago, Josh.5 said:

Can you show me your docker template config?

I currently have this, which is working great.

 

1760566029_Screenshot2021-09-28at16_32_17.thumb.jpg.ef115f0a214a2ee3979f0da98f81920e.jpg

 

Whenever the docker updates it appears to add two further library paths - one for tv and one for movies. Neither of which I need so I just delete them. They do seem to keep coming back though.

 

I could be missing something obvious!

 

Edited by page3
Link to comment
4 hours ago, DaClownie said:
 
Perhaps I'm doing something wrong, but I'm failing to add repositories whenever I attempt to use that button. I tried with the full link to the repository, as well as just Josh5/unmanic-plugins
1869129897_failtosave.png.2aa92c90b3b2b27452add2e8df3c10af.png

Yes, you are doing that wrong.

Did you go to the link and read the README.md file as it explains how to install the repo.

The README links to these instructions: https://docs.unmanic.app/docs/plugins/adding_a_custom_plugin_repo/

Edited by Josh.5
Link to comment
2 hours ago, page3 said:

I currently have this, which is working great.

 

1760566029_Screenshot2021-09-28at16_32_17.thumb.jpg.ef115f0a214a2ee3979f0da98f81920e.jpg

 

Whenever the docker updates it appears to add two further library paths - one for tv and one for movies. Neither of which I need so I just delete them. They do seem to keep coming back though.

 

I could be missing something obvious!

 

The lower-case 'tv' and 'movies' directories are created by the docker template. It has nothing to do with Unmanic itself. Unmanic has no code that is specific to "Movies" or "TVShows" - it is a library optimiser and is completely agnostic of what type of file library you have.

So the only way this is possibly happening is because your template is not correct or another container is creating these folders.

Take note of the docker create/run command that is executed when you apply the template:

uraid-unmanic-template1.thumb.png.45e8b0ee7320f807c36cd238cfbb2142.pnguraid-unmanic-template2.thumb.png.3ff3fa8bfea5b8db7bcd7db1f5f3fbcd.png

 

Make sure your template has the mount removed in the "Container Path:" section - not just the description...

uraid-unmanic-template3.png.8612c2de0579e4ae7acd30b63a33021b.png

 

 

 

Link to comment

I've pushed an update to the "Remux Video Files" plugin - this is now improved to handle transcoding streams that are not compatible with the destination container.

For example, if you have a wmv and you want it converted to an mp4, this will need to be transcoded. It will default to h264/aac. If the stream type is not compatible, then it will just be removed. This will only affect people coming from unique video containers, if you have a mkv file in h265/ac3 and you run it with this plugin to remux to mp4, it will not transcode the video stream as the plugin knows that mp4 files can handle h265, so that stream will be simply copied. 

For people running into issues with a codec not being compatible with the container they are wanting to move to, can you add this plugin in your plugin flow BEFORE any other FFmpeg tasks - eg, before audio sorting, h265 encoding, etc.

cc: @Aerodb @EdwinZelf

  • Like 1
  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.