[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

20 hours ago, jonfive said:

Is it at all possible to add a "custom" library to watch?
If i were to add another path in the docker config would i be able to select it within unmanic as an additional library - or would that have to be intended in the software to begin with?

yeah just create new paths. all the container needs to see is /library then you can sub folder that out.

 

image.png.7c90b0ee4ce0396ec49ef813258e8375.png

  • Like 1
Link to comment

Hey All,

 

How can we use Plex Live/DVR with Unmanic? As I record a show, I notice that it tries to convert while it is still recording, possibly causing problems with the show/movie. Would love to use Unmanic instead of Handbrake and keep the files appropriately in the right folders.

Link to comment

This app is terrific, i've converted some of my tvshows and I've already saved 160Gb.

 

I've got a problem with some subtitles, logs say:


 

Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> hevc (libx265))
Stream #0:1 -> #0:1 (copy)
Stream #0:2 -> #0:2 (copy)
Stream #0:3 -> #0:3 (dvd_subtitle (dvdsub) -> subrip (srt))
Stream #0:4 -> #0:4 (dvd_subtitle (dvdsub) -> subrip (srt))
Subtitle encoding currently only possible from text to text or bitmap to bitmap

 

Selecting "Remove subtitles" solves the problem, but I want to keep them.

Any suggestion?

Link to comment
This app is terrific, i've converted some of my tvshows and I've already saved 160Gb.
 
I've got a problem with some subtitles, logs say:

 
Stream mapping:Stream #0:0 -> #0:0 (h264 (native) -> hevc (libx265))Stream #0:1 -> #0:1 (copy)Stream #0:2 -> #0:2 (copy)Stream #0:3 -> #0:3 (dvd_subtitle (dvdsub) -> subrip (srt))Stream #0:4 -> #0:4 (dvd_subtitle (dvdsub) -> subrip (srt))Subtitle encoding currently only possible from text to text or bitmap to bitmap

 
Selecting "Remove subtitles" solves the problem, but I want to keep them.
Any suggestion?

https://lmgtfy.app/?q=Subtitle+encoding+currently+only+possible+from+text+to+text+or+bitmap+to+bitmap


Wait... I found a better link. Looks like LMGTFY has changed a bit this year.

https://googlethatforyou.com?q=Subtitle%20encoding%20currently%20only%20possible%20from%20text%20to%20text%20or%20bitmap%20to%20bitmap

Link to comment
36 minutes ago, Josh.5 said:

 

I've tried searching a bit before posting. I'm not running any custom FFMPEG command, so I was wondering if Unmanic is trying to alter subtitle format by itself or what (I've already used it for different files with subtitles without any problem).

Edited by melagodo
Link to comment
 
I've tried searching a bit before posting. I'm not running any custom FFMPEG command, so I was wondering if Unmanic is trying to alter subtitle format by itself or what (I've already used it for different files with subtitles without any problem).
You are trying to convert pictures into txt. This is not possible with FFMPEG.
Link to comment

@melagodo
Just to be clear on that...
This is not exactly an issue with Unmanic. If you are converting a video in a container such as MP4 which contains bitmap subs to a container like mkv that doesn't support them, then it will fail.

If you like, I can write you a plugin that will strip subtitles if they are bitmap type, but if they are txt type, leave them and copy them to the new mkv.

Let me know.

Link to comment
20 hours ago, Josh.5 said:

@melagodo
Just to be clear on that...
This is not exactly an issue with Unmanic. If you are converting a video in a container such as MP4 which contains bitmap subs to a container like mkv that doesn't support them, then it will fail.

If you like, I can write you a plugin....

 

That would be awesome. I keep trying digging into FFMPEG secrets, but not really going anywhere.

Strange thing is that source file is already a mkv. Here's full FFMPEG log. I know this goes way beyond unmanic support, but if you can help me or give me a hint...

I might as well go with Handbrake route, just trying to understand what's happening under the hood for future similar problems.

Thanks again mate

 

Input #0, matroska,webm, from '/library/movies/Sailor Moon.1x01.(1) [DVDRip AC3 ITA JAP Sub ITA MKV By FaTTi BeNe TeAM].mkv':
  Metadata:
    title           : By FaTTi BeNe TeAM
    encoder         : libebml v1.3.3 + libmatroska v1.4.4
    creation_time   : 2016-06-01T16:10:36.000000Z
  Duration: 00:24:28.28, start: 0.080000, bitrate: 1952 kb/s
    Chapter #0:0: start 0.000000, end 92.840000
    Metadata:
      title           : Chapter 1
    Chapter #0:1: start 92.920000, end 694.200000
    Metadata:
      title           : Chapter 2
    Chapter #0:2: start 694.280000, end 1326.160000
    Metadata:
      title           : Chapter 3
    Chapter #0:3: start 1326.240000, end 1417.000000
    Metadata:
      title           : Chapter 4
    Chapter #0:4: start 1417.080000, end 1447.720000
    Metadata:
      title           : Chapter 5
    Chapter #0:5: start 1447.800000, end 1468.280000
    Metadata:
      title           : Chapter 6
    Stream #0:0(jpn): Video: h264, yuv420p(tv, smpte170m/bt470bg/bt709, progressive), 640x480, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Metadata:
      title           : By FaTTi BeNe TeAM
      BPS             : 1499865
      BPS-eng         : 1499865
      DURATION        : 00:24:28.280000000
      DURATION-eng    : 00:24:28.280000000
      NUMBER_OF_FRAMES: 36707
      NUMBER_OF_FRAMES-eng: 36707
      NUMBER_OF_BYTES : 275277827
      NUMBER_OF_BYTES-eng: 275277827
      _STATISTICS_WRITING_APP: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-06-01 16:10:36
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-06-01 16:10:36
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(ita): Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s (default)
    Metadata:
      title           : Italiano
      BPS             : 224000
      BPS-eng         : 224000
      DURATION        : 00:24:28.192000000
      DURATION-eng    : 00:24:28.192000000
      NUMBER_OF_FRAMES: 45881
      NUMBER_OF_FRAMES-eng: 45881
      NUMBER_OF_BYTES : 41109376
      NUMBER_OF_BYTES-eng: 41109376
      _STATISTICS_WRITING_APP: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-06-01 16:10:36
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-06-01 16:10:36
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:2(jpn): Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s
    Metadata:
      title           : Giapponese
      BPS             : 224000
      BPS-eng         : 224000
      DURATION        : 00:24:28.192000000
      DURATION-eng    : 00:24:28.192000000
      NUMBER_OF_FRAMES: 45881
      NUMBER_OF_FRAMES-eng: 45881
      NUMBER_OF_BYTES : 41109376
      NUMBER_OF_BYTES-eng: 41109376
      _STATISTICS_WRITING_APP: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-06-01 16:10:36
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-06-01 16:10:36
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:3(ita): Subtitle: dvd_subtitle, 720x576
    Metadata:
      title           : Italiano (Dialoghi)
      BPS             : 2305
      BPS-eng         : 2305
      DURATION        : 00:24:27.103000000
      DURATION-eng    : 00:24:27.103000000
      NUMBER_OF_FRAMES: 281
      NUMBER_OF_FRAMES-eng: 281
      NUMBER_OF_BYTES : 422712
      NUMBER_OF_BYTES-eng: 422712
      _STATISTICS_WRITING_APP: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-06-01 16:10:36
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-06-01 16:10:36
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:4(ita): Subtitle: dvd_subtitle, 720x576 (default)
    Metadata:
      title           : Italiano (Cartelli)
      BPS             : 262
      BPS-eng         : 262
      DURATION        : 00:24:27.103000000
      DURATION-eng    : 00:24:27.103000000
      NUMBER_OF_FRAMES: 28
      NUMBER_OF_FRAMES-eng: 28
      NUMBER_OF_BYTES : 48102
      NUMBER_OF_BYTES-eng: 48102
      _STATISTICS_WRITING_APP: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.2.0 ('Photograph') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-06-01 16:10:36
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-06-01 16:10:36
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (libx265))
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (dvd_subtitle (dvdsub) -> subrip (srt))
  Stream #0:4 -> #0:4 (dvd_subtitle (dvdsub) -> subrip (srt))
Subtitle encoding currently only possible from text to text or bitmap to bitmap

 

Link to comment

Hello friends,

 

Love the docker, works wonderfully.

I was wondering if it's possible to transcode say transcode 4k movies down to 1080p and (if possible) not touch any lower resolution content?

 

On 4/10/2021 at 3:24 AM, Josh.5 said:

If you like, I can write you a plugin that will strip subtitles if they are bitmap type, but if they are txt type, leave them and copy them to the new mkv.

And as an answer to this; yes that'd be amazing!

 

Link to comment
8 hours ago, melagodo said:

 

That would be awesome. I keep trying digging into FFMPEG secrets, but not really going anywhere.

Strange thing is that source file is already a mkv. Here's full FFMPEG log. I know this goes way beyond unmanic support, but if you can help me or give me a hint...

I might as well go with Handbrake route, just trying to understand what's happening under the hood for future similar problems.

Thanks again mate

 

 

Before I do write the plugin, can you please update your unmanic to the latest 'staging' build and test that failing file again with the

"Remove subtitles" option unchecked.

I am by no means an FFMPEG expert. So I could be wrong about what the MKV container supports as far as subtitle codecs. The only way I can fix this is if people with these kinds of videos can do the testing for me...

Link to comment
On 1/5/2019 at 10:40 PM, Josh.5 said:

 

Nvidia GPU hardware encoding: [DEPRECATED]

(Only use the below instructions if you are not running the latest version of Unraid that ships with NVIDIA drivers already installed)

 

Is GPU hardware encoding supported on the newest 6.9.x?  I'm getting ffmpeg errors like "Cannot load libcuda.so.1"

 

  • Like 1
Link to comment

Hi all,

 

I have Unmanic running on two machines.

 

1. My NAS which is what runs everything. It is an i5 with 16GB of RAM. I started Unmanic with the flag to not use more than 50% CPU and only enabled two workers. When I woke this morning, my server was completely unresponsive and none of the apps were accessible. I was able to access it via terminal and killall ffmpeg processes. The weird thing is that even though I only had two workers enabled in Unmanic, there were 17 running ffmpeg processes running. Once I killed them the server and other apps came back up. Since I had to kill this, I have no logs for it.

 

2. I have a Dell Poweredge server with a Xeon CPU and 16GB of ram. This server does not belong to me, so I am just using it to convert my media. It is running a trial of Unraid with only two apps running. Resilio Sync and Unmanic. I have synced my entire 10TB media library to it. I have Unmanic pointed to the library. It has been running a few days without issue and then syncing the changed files to server 1 above. Yesterday morning I noticed that I could no longer access the webui. In Unraid, I could see CPU was at 100% until I stopped Unmanic. After restarting I could access the webui again. I noticed all 5 workers start working and could see progress. After a short while the webui was no longer accessible. I let it run overnight and event though the CPU stayed at 100%, it seems no media files were updated.

 

I did disable watcher on both systems, but it didn't make any difference.I tried reducing workers to 1, but still having same issues.

 

Here is the log from server 2.

Running Unmanic from installed module
Starting migrations
There is nothing to migrate
UnmanicLogger - SETUP LOGGER
Clearing cache path - /tmp/unmanic/unmanic_file_conversion-1618327767.4300964
Traceback (most recent call last):
File "/usr/local/bin/unmanic", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 396, in main
service.run()
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 359, in run
self.start_threads(settings)
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 306, in start_threads
common.clean_files_in_dir(settings.get_cache_path())
File "/usr/local/lib/python3.8/dist-packages/unmanic/libs/common.py", line 119, in clean_files_in_dir
shutil.rmtree(root)
File "/usr/lib/python3.8/shutil.py", line 719, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/lib/python3.8/shutil.py", line 717, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/tmp/unmanic/unmanic_file_conversion-1618327767.4300964'
Running Unmanic from installed module
Starting migrations
There is nothing to migrate
UnmanicLogger - SETUP LOGGER
Clearing cache path - /tmp/unmanic/unmanic_file_conversion-1618327767.4300964
Traceback (most recent call last):
File "/usr/local/bin/unmanic", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 396, in main
service.run()
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 359, in run
self.start_threads(settings)
File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 306, in start_threads
common.clean_files_in_dir(settings.get_cache_path())
File "/usr/local/lib/python3.8/dist-packages/unmanic/libs/common.py", line 119, in clean_files_in_dir
shutil.rmtree(root)
File "/usr/lib/python3.8/shutil.py", line 719, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/lib/python3.8/shutil.py", line 717, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/tmp/unmanic/unmanic_file_conversion-1618327767.4300964'

 

I also tried changing the repository to staging, but still having same issues. Not sure what to check for next.

Link to comment
Hi all,
 
I have Unmanic running on two machines.
 
1. My NAS which is what runs everything. It is an i5 with 16GB of RAM. I started Unmanic with the flag to not use more than 50% CPU and only enabled two workers. When I woke this morning, my server was completely unresponsive and none of the apps were accessible. I was able to access it via terminal and killall ffmpeg processes. The weird thing is that even though I only had two workers enabled in Unmanic, there were 17 running ffmpeg processes running. Once I killed them the server and other apps came back up. Since I had to kill this, I have no logs for it.
 
2. I have a Dell Poweredge server with a Xeon CPU and 16GB of ram. This server does not belong to me, so I am just using it to convert my media. It is running a trial of Unraid with only two apps running. Resilio Sync and Unmanic. I have synced my entire 10TB media library to it. I have Unmanic pointed to the library. It has been running a few days without issue and then syncing the changed files to server 1 above. Yesterday morning I noticed that I could no longer access the webui. In Unraid, I could see CPU was at 100% until I stopped Unmanic. After restarting I could access the webui again. I noticed all 5 workers start working and could see progress. After a short while the webui was no longer accessible. I let it run overnight and event though the CPU stayed at 100%, it seems no media files were updated.
 
I did disable watcher on both systems, but it didn't make any difference.I tried reducing workers to 1, but still having same issues.
 
Here is the log from server 2.
Running Unmanic from installed moduleStarting migrationsThere is nothing to migrateUnmanicLogger - SETUP LOGGERClearing cache path - /tmp/unmanic/unmanic_file_conversion-1618327767.4300964Traceback (most recent call last):File "/usr/local/bin/unmanic", line 8, in sys.exit(main())File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 396, in mainservice.run()File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 359, in runself.start_threads(settings)File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 306, in start_threadscommon.clean_files_in_dir(settings.get_cache_path())File "/usr/local/lib/python3.8/dist-packages/unmanic/libs/common.py", line 119, in clean_files_in_dirshutil.rmtree(root)File "/usr/lib/python3.8/shutil.py", line 719, in rmtreeonerror(os.rmdir, path, sys.exc_info())File "/usr/lib/python3.8/shutil.py", line 717, in rmtreeos.rmdir(path)OSError: [Errno 39] Directory not empty: '/tmp/unmanic/unmanic_file_conversion-1618327767.4300964'Running Unmanic from installed moduleStarting migrationsThere is nothing to migrateUnmanicLogger - SETUP LOGGERClearing cache path - /tmp/unmanic/unmanic_file_conversion-1618327767.4300964Traceback (most recent call last):File "/usr/local/bin/unmanic", line 8, in sys.exit(main())File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 396, in mainservice.run()File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 359, in runself.start_threads(settings)File "/usr/local/lib/python3.8/dist-packages/unmanic/service.py", line 306, in start_threadscommon.clean_files_in_dir(settings.get_cache_path())File "/usr/local/lib/python3.8/dist-packages/unmanic/libs/common.py", line 119, in clean_files_in_dirshutil.rmtree(root)File "/usr/lib/python3.8/shutil.py", line 719, in rmtreeonerror(os.rmdir, path, sys.exc_info())File "/usr/lib/python3.8/shutil.py", line 717, in rmtreeos.rmdir(path)OSError: [Errno 39] Directory not empty: '/tmp/unmanic/unmanic_file_conversion-1618327767.4300964'

 
I also tried changing the repository to staging, but still having same issues. Not sure what to check for next.

Can you post some screenshots of your configs used to run Unmanic
Link to comment
On 4/8/2021 at 2:09 PM, BomB191 said:

yeah just create new paths. all the container needs to see is /library then you can sub folder that out.

 

image.png.7c90b0ee4ce0396ec49ef813258e8375.png


 

You should really remove user0 and make it /mnt/user/

 

The method you have has been deprecated and could be removed at any time and then you will have problems not to mention it does not touch your cache cool using user0

Link to comment
2 hours ago, Josh.5 said:

Can you try mounting /tmp/Unmanic to /mnt/user/.unmanic/cache

 

Ok, I made that change on my main server and everything became unresponsive after 10 minutes. Even Plex stopped working until I stopped Unamnic. I also changed this servers extra parameters to "--cpus='.25'", but it still crashed.

 

I made your suggested change on the temp server as well. After about 5 minutes I could no longer access the webui. I then added the extra parameter of "--cpus='.9'" to this server and restarted. I still lost access to the webui after a few minutes.

 

Am I doing something wrong? This worked for several days straight and then all of a sudden stopped working a couple of days ago. I may have added more apps to my main server, but I made no changes whatsoever on the temp server other than what was asked for here today.

Link to comment
 
Ok, I made that change on my main server and everything became unresponsive after 10 minutes. Even Plex stopped working until I stopped Unamnic. I also changed this servers extra parameters to "--cpus='.25'", but it still crashed.
 
I made your suggested change on the temp server as well. After about 5 minutes I could no longer access the webui. I then added the extra parameter of "--cpus='.9'" to this server and restarted. I still lost access to the webui after a few minutes.
 
Am I doing something wrong? This worked for several days straight and then all of a sudden stopped working a couple of days ago. I may have added more apps to my main server, but I made no changes whatsoever on the temp server other than what was asked for here today.
So just to be clear... On unraid you have a share named ".unmanic"?
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.