[Support] Josh5 - Unmanic - Library Optimiser


1072 posts in this topic Last Reply

Recommended Posts

@Josh.5

 

FYI,

 

Someone at Datahoarding has created a template for unManic and published this on CA.  I believe earlier you had stated that you did not wish unManic to be published as of yet.  Is this still the case?  Should the template be removed from CA?  Ideally though if you would like it in CA it would be better for it to be published within your own repository.

Link to post
  • Replies 1.1k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Application Name: UNMANIC - Library Optimiser Application Site: https://github.com/Unmanic/unmanic/ Docker Hub: https://hub.docker.com/r/josh5/unmanic/ Github: https://github.com/Unmani

@Ashe @trekkiedj @itimpi   Please see update first post also for an example of how I have it setup. I have also added a temporary solution to adding multiple library locations if anyone need

There should be a new update available. This has quite a number of changes to how ffmpeg is executed. It should resolve some issues with inotify and library scanning. For those people who created

Posted Images

@Josh.5
 
FYI,
 
Someone at Datahoarding has created a template for unManic and published this on CA.  I believe earlier you had stated that you did not wish unManic to be published as of yet.  Is this still the case?  Should the template be removed from CA?  Ideally though if you would like it in CA it would be better for it to be published within your own repository.
Yea. It's gonna suck for average people when I merge the dev-settings branch and they have to go and save their settings again because it has changed. The application is still in beta as it is still at a point where it could change and an update could mean a fresh install. Untill that is not the case, I was holding out on it being put in CA.
But if they have published the template, then they can provide support to the people using it right.

Sent from my ONEPLUS A6000 using Tapatalk

Link to post
3 minutes ago, Josh.5 said:

But if they have published the template, then they can provide support to the people using it right.

lol

<Support>https://forums.unraid.net/topic/76878-support-josh5-unmanic-library-optimiser/</Support>

 

Link to post

First of all I found this by looking at setting up Spaceinvaders Handbrake/Sonarr/Radarr Video but luckily I found this as with a 50+ TB library Handbraking current items will take a long time and be one hell of a tedious manual process so thank you Josh.5 for making this.

 

Just finished my first folder as you can see from the attachments(24 series) the before and after results are amazing with video quality perfect.

 

I do however have some comments/Recommendation about what this app has done for me not sure if anyone else has also mentioned this before so sorry if I am repeating stuff.

 

  • My settings appear to not be persistent on a app restart ? After every restart of the app(changing template settings by changing series folder and allocating more cores) the settings go to default.
  •  On my unRAID dashboard the UNMATIC container does not have the image I just have a ? logo
  • When clicking on the container can you add the function to go to "Webui" like other containers as I also do not have this ?
  • When adding the container I did not have any variables to fill in I had to manually add this ? Might have just been the way I did it? (Copy the full URL from docker site and pasted it into the "Add container" button on unRAID).
  • Some way to warn or tell people to allocate cores as it will just spank everything to 100%
  • After changing folder from 24(Which is now complete) to the next series my "Completed tasks" are now blank again ?
  • Add option for pending tasks to be moved up and down (Higher and lower priority)?
  • Add a Fu**ing donation button I owe you a beer!

Keep up the good work.

 

1.24 before folder size.jpg

4.after.jpg

5.Ripping the threads.jpg

Link to post
On 9/4/2019 at 6:17 AM, Josh.5 said:


 

 


Actually that output is enough as far as unmanic goes. It looks like ffmpeg did not like the arguments it was passed or the file needed some kind of special configuration.
Can you please try running that command manually from a terminal inside the docker container. Ffmpeg will give you some output that will show what is wrong. Then can you give me that output?

Sent from my ONEPLUS A6000 using Tapatalk
 

 

I have no idea how to do this 🤣🤣

Link to post

Hi @Josh.5 

 

Im noticing that after a successful number of conversions its now stopped converting even though there are files in the wrong formats. Im getting a lot of errors in the log. 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/lib/ffmpeg.py", line 193, in check_file_to_be_processed
container = unffmpeg.containers.grab_module(self.settings.OUT_CONTAINER)
File "/app/lib/unffmpeg/containers/__init__.py", line 66, in grab_module
raise ImportError('{} is not part of our collection!'.format(module_name))
ImportError: mkv is not part of our collection!
2019-09-30T15:07:45:ERROR:Unmanic.FFMPEGHandle - [FORMATTED] - Exception - check_file_to_be_processed check file container: mkv is not part of our collection!

Traceback (most recent call last):
File "/app/lib/unffmpeg/containers/__init__.py", line 59, in grab_module
module = import_module('.' + module_name, package=__name__)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'lib.unffmpeg.containers.mkv'

Thanks for the help and a great app.

Link to post
25 minutes ago, lordvaderxiii said:

Hi @Josh.5 

 

Im noticing that after a successful number of conversions its now stopped converting even though there are files in the wrong formats. Im getting a lot of errors in the log. 


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/lib/ffmpeg.py", line 193, in check_file_to_be_processed
container = unffmpeg.containers.grab_module(self.settings.OUT_CONTAINER)
File "/app/lib/unffmpeg/containers/__init__.py", line 66, in grab_module
raise ImportError('{} is not part of our collection!'.format(module_name))
ImportError: mkv is not part of our collection!
2019-09-30T15:07:45:ERROR:Unmanic.FFMPEGHandle - [FORMATTED] - Exception - check_file_to_be_processed check file container: mkv is not part of our collection!

Traceback (most recent call last):
File "/app/lib/unffmpeg/containers/__init__.py", line 59, in grab_module
module = import_module('.' + module_name, package=__name__)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'lib.unffmpeg.containers.mkv'

Thanks for the help and a great app.

Ah, so this is 100% my fault. I should have posted something here before moving onto the next feature...

 

Last night I merged in the applications settings changes that I've been working on for the past wee while. These new changes moves all the settings to an SQLite database. There were also changes made to how Unmanic loads data pertaining to containers/audio codecs/video codecs/subtitles. These changes will allow things like employing HW encoders in the future (slowly getting there...). For now, the big takeaway is that anyone who updates Unmanic with last nights changes will need to go and save their settings again. The error you have posted is that 'mkv' container (how it was configured) is now 'matroska' as that is the correct name of the FFMPEG. So the container settings need to be saved again.

 

TL;DR

Everyone who updates Unmanic needs to save their settings again or it will stop working for them.

EDIT:

If anyone is interested in what the update changed:

https://github.com/Josh5/unmanic/releases/tag/0.0.1-beta6

Edited by Josh.5
Added link to GitHub release info
Link to post

I just saw this on the applist for unraid and installed. Its been running for 3 hours (3 workers) on my i7-8086k with full access to all cores/threads. All 3 workers are still showing 0%. I'm using default settings. CPU usage has been 100% for the full 3 hours

 

I did just make sure that remove subtitles was checked, turned down to only 2 workers, and restarted the container. Its been running for 20 minutes still 0% completed in docker, still 100% CPU usage on all cores. I've also turned off all other dockers so only this one is running.

 

(I did set it to a share using cache only, minimum available space is 100GB)

 

Edit: I changed the codec from HEVC to 264 and let it run for a few minutes, it did complete some files but again showed no % updates at all. It would do one every 10-15 minutes this way, but will not do anything with the default HEVC.

Edited by Remamian
Link to post

Minor issue

 

If you change the number of workers (lower them) after transcoding has begun, the system will still continue on its merry way (and reflect the new number of workers after the current job is finished), but will no longer move the files back to their final resting place but leave them in the temp directory

Link to post
7 hours ago, Remamian said:

I just saw this on the applist for unraid and installed. Its been running for 3 hours (3 workers) on my i7-8086k with full access to all cores/threads. All 3 workers are still showing 0%. I'm using default settings. CPU usage has been 100% for the full 3 hours

 

I did just make sure that remove subtitles was checked, turned down to only 2 workers, and restarted the container. Its been running for 20 minutes still 0% completed in docker, still 100% CPU usage on all cores. I've also turned off all other dockers so only this one is running.

 

(I did set it to a share using cache only, minimum available space is 100GB)

 

Edit: I changed the codec from HEVC to 264 and let it run for a few minutes, it did complete some files but again showed no % updates at all. It would do one every 10-15 minutes this way, but will not do anything with the default HEVC.

i'm having the exact same issue. looking in the temp/cache folder, the new HEVC files appear to be being built, but no progress is showing at all in the web UI for the respective workers - stuck at 0%. 

Link to post
7 hours ago, Squid said:

Minor issue

 

If you change the number of workers (lower them) after transcoding has begun, the system will still continue on its merry way (and reflect the new number of workers after the current job is finished), but will no longer move the files back to their final resting place but leave them in the temp directory

https://forums.unraid.net/topic/76878-support-josh5-unmanic-library-optimiser/?do=findComment&comment=768903

 

Are you saying subsequent transcodes are lost for you until you restart the container? If so, that is new behaviour.

Link to post
3 hours ago, Cpt. Chaz said:

i'm having the exact same issue. looking in the temp/cache folder, the new HEVC files appear to be being built, but no progress is showing at all in the web UI for the respective workers - stuck at 0%. 

Wanted to post an update to this - even though the worker progress is stuck at 0% in the UI, looks like the jobs are still completing and executing correctly, including showing finished files in the "completed tasks" window.

Link to post
20 hours ago, Josh.5 said:

Ah, so this is 100% my fault. I should have posted something here before moving onto the next feature...

 

Last night I merged in the applications settings changes that I've been working on for the past wee while. These new changes moves all the settings to an SQLite database. There were also changes made to how Unmanic loads data pertaining to containers/audio codecs/video codecs/subtitles. These changes will allow things like employing HW encoders in the future (slowly getting there...). For now, the big takeaway is that anyone who updates Unmanic with last nights changes will need to go and save their settings again. The error you have posted is that 'mkv' container (how it was configured) is now 'matroska' as that is the correct name of the FFMPEG. So the container settings need to be saved again.

 

TL;DR

Everyone who updates Unmanic needs to save their settings again or it will stop working for them.

EDIT:

If anyone is interested in what the update changed:

https://github.com/Josh5/unmanic/releases/tag/0.0.1-beta6

Thanks @Josh.5

 

I stumbled on that by messing with the settings. 

Link to post
1 hour ago, Cpt. Chaz said:

Wanted to post an update to this - even though the worker progress is stuck at 0% in the UI, looks like the jobs are still completing and executing correctly, including showing finished files in the "completed tasks" window.

Im getting this issue too. Workers at 0% but jobs definitely completing. 

Link to post
10 hours ago, Josh.5 said:

 A fix for the 0% issue that everyone is having is being built and pushed now. ETA 10 mins.

 

@Squid I've logged your issue on Github https://github.com/Josh5/unmanic/issues/73. Cheers.

worked like a charm - thanks bud! i've seen on the forums in the past you don't take donations, would love to "buy you a pint" for your work. Is this still the case?

Link to post
worked like a charm - thanks bud! i've seen on the forums in the past you don't take donations, would love to "buy you a pint" for your work. Is this still the case?
I second that. I've decreased my library size by over 3TBs. You need put a donate button in the app or something.

Sent from my Pixel 3 XL using Tapatalk

Link to post

Awesome.
I appreciate the gesture, but I won't be taking donations for this application any time soon. I would prefer that if you feel a need to contribute, you do so by reporting bugs (as you have) and by helping others here in the forums. And of course, if you can code, pull requests on GitHub are also welcome.


Cheers.

Sent from my ONEPLUS A6000 using Tapatalk

Link to post
11 hours ago, Josh.5 said:

Awesome.
I appreciate the gesture, but I won't be taking donations for this application any time soon. I would prefer that if you feel a need to contribute, you do so by reporting bugs (as you have) and by helping others here in the forums. And of course, if you can code, pull requests on GitHub are also welcome.


Cheers.

Sent from my ONEPLUS A6000 using Tapatalk
 

@Josh.5 Youre a legend! Thanks. 

 

Will continue to report bugs. 

Link to post

I have updated to the latest version and just 2 things to report.

 

1. I do my conversions folder by folder at the moment (/Mnt/usr/series/Show to convert) and then "edit" the container to the next show when it is done (/Mnt/usr/series/Show to convert). Each time the container starts up it goes back to the default settings it is not persistent.

2. When changing something in the settings and then clicking "Submit" my browser goes to http://192.168.0.4:8899/settings/?step=general with the page below. I then manually need to remove the "/?step=general" to get back to the settings GUI

 

image.thumb.png.5a0535ae5f3a39f504451fb57379d285.png

 

Apart from that my poor Threadripper has been crying for a week while my disk free disk space keeps going up 🙂 

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.