[Support] Josh5 - Unmanic - Library Optimiser


Recommended Posts

I am also getting no GUI access with the latest staging built from scratch. I will downgrade my Nvidia version and report back. I attached my unmanic log. It keeps repeating the same thing over and over. Lots of peewee whatever that is. 
a9bf91935af9.log.gz
Yea, I've fixed this. Could you please pull the latest release and try again.
Link to comment
9 minutes ago, rmeaux said:

I am also getting no GUI access with the latest staging built from scratch. I will downgrade my Nvidia version and report back. I attached my unmanic log. It keeps repeating the same thing over and over. Lots of peewee whatever that is. 

a9bf91935af9.log.gzFetching info...

Sorry, just double checked. I only pushed the fix to the master branch. I've just applied it to the staging branch also. Give it 10 mins and then there should be an update for you to pull.

 

Sorry again.

Link to comment
11 minutes ago, Josh.5 said:

Sorry, just double checked. I only pushed the fix to the master branch. I've just applied it to the staging branch also. Give it 10 mins and then there should be an update for you to pull.

 

Sorry again.

 

No apologies, Thanks!!

Link to comment
14 minutes ago, Josh.5 said:

Sorry, just double checked. I only pushed the fix to the master branch. I've just applied it to the staging branch also. Give it 10 mins and then there should be an update for you to pull.

 

Sorry again.

 

Many thanks will test

Link to comment

I just got Unmanic up and running and found two things that I wanted to check on...

 

First, when I had the network set to Custom: br0 and left the IP blank so it could pull a DHCP address, it never pulls an IP or spins up the web interface.  The Docker DHCP scope is defined and works on other containers.  When I set the network to Bridged, it brings up the web interface without issues.   Is there something that I'm missing?

 

Second, I have everything set up and it is detecting videos that need optimized correctly, but the jobs are running as CPU only jobs and not using the GPU.  I have the following lines in the log:

 

[cont-init.d] 30-patch-nvidia: executing...
Detected nvidia driver version: 460.67
Patch for this (460.67) nvidia driver not found.

Is that why Unmanic isn't using the GPU?

 

Thanks for any assistance you might be able to render.

Edited by matt.shepker
Link to comment
5 hours ago, Josh.5 said:

Sorry, just double checked. I only pushed the fix to the master branch. I've just applied it to the staging branch also. Give it 10 mins and then there should be an update for you to pull.

 

Sorry again.

Many thanks Josh. I’ve managed to now install but still loosing access to the UI soon as it starts encoding. 
 

log attached. 

F210E78F-A23D-483E-A737-C2A807C90B13.jpeg

Link to comment
On 3/29/2021 at 6:03 PM, Yekul said:

Having a quick play with this, and the foundations look real solid and almost just what i'm after. I am curious though, any plans to add a manual scan button to the front page, and perhaps a 'start paused' type option? So we could select a single folder from the scan and start it before bed for example. I prefer to be more in control of what is happening is all, so being able to scan and encode a folder at a time without -everything- being encoded automatically would be great.

 

@Yekul There is also a way to do this manually. disable the "scan on start" option in the general settings, save and submit, then restart the container. then go back to the settings, and point the library path at whatever specific folder you want. go back to the dashboard, and at your discretion, hit the options tab and "rescan library now". 

 

this will keep the entire library path from being scanned automatically, and offer a little more control over specific folders to be scanned and transcoded manually, along with any cron schedules you apply. 

Link to comment
15 hours ago, matt.shepker said:

I just got Unmanic up and running and found two things that I wanted to check on...

 

First, when I had the network set to Custom: br0 and left the IP blank so it could pull a DHCP address, it never pulls an IP or spins up the web interface.  The Docker DHCP scope is defined and works on other containers.  When I set the network to Bridged, it brings up the web interface without issues.   Is there something that I'm missing?

 

Second, I have everything set up and it is detecting videos that need optimized correctly, but the jobs are running as CPU only jobs and not using the GPU.  I have the following lines in the log:

 


[cont-init.d] 30-patch-nvidia: executing...
Detected nvidia driver version: 460.67
Patch for this (460.67) nvidia driver not found.

Is that why Unmanic isn't using the GPU?

 

Thanks for any assistance you might be able to render.

I'm not sure if that would affect the NVIDIA use unless you already had more than 3 GPU encoder jobs running from another container.

At any rate, I've just updated this in the last staging build.

Link to comment
10 hours ago, mickh18 said:

Many thanks Josh. I’ve managed to now install but still loosing access to the UI soon as it starts encoding. 
 

log attached. 

F210E78F-A23D-483E-A737-C2A807C90B13.jpeg

This error is likely because of issues with the permissions of you cache directory. Can you make sure the cache path is `chmod -R a+rw ...`

Link to comment

I don't know much about ffmepg commands.  By much, I mean nothing at all.  How do I increase -max_muxing_queue_size from whatever it's defaulted too in unmanic to something like 1024?  Every once in a while I have a few files that fail and everything I google says to increase the mux queue size to at least 1024.  I see the example for the ffmepg under the advance tab... but I don't want to change anything else in my settings (like output file location, etc...) I'm scared I'll mess something up.

Link to comment
8 minutes ago, Meller said:

I don't know much about ffmepg commands.  By much, I mean nothing at all.  How do I increase -max_muxing_queue_size from whatever it's defaulted too in unmanic to something like 1024?  Every once in a while I have a few files that fail and everything I google says to increase the mux queue size to at least 1024.  I see the example for the ffmepg under the advance tab... but I don't want to change anything else in my settings (like output file location, etc...) I'm scared I'll mess something up.

image.thumb.png.ec0cb30149759c6af691d89f335a4d8d.png

 

This attached image show exactly what you want to do.

See how adding these have overwritten the additional options.

 

The "Custom FFMPEG Options" input does not overwrite all of the other Unmanic settings.

Link to comment
23 minutes ago, Josh.5 said:

image.thumb.png.ec0cb30149759c6af691d89f335a4d8d.png

 

This attached image show exactly what you want to do.

See how adding these have overwritten the additional options.

 

The "Custom FFMPEG Options" input does not overwrite all of the other Unmanic settings.

Appreciate it, sir.

Link to comment

Hi @Josh.5, really impressive work!

 

I have a strange behaviour. If I set Library scan schedule to "0" (which should disable it, I spin down my disks...) docker start consuming 100% of each core, alternating between the 3 cores pinned to the docker (right after clicking submit in the GUI).

I've been able to reproduce the issue multiple time.

Any others?

Edited by melagodo
Link to comment
Hi @Josh.5, really impressive work!
 
I have a strange behaviour. If I set Library scan schedule to "0" (which should disable it, I spin down my disks...) docker start consuming 100% of each core, alternating between the 3 cores pinned to the docker (right after clicking submit in the GUI).
I've been able to reproduce the issue multiple time.
Any others?
Can you give me details on your config and post your logs?
Link to comment

Tried running version 0.0.3+2bff016 this morning with Nvidia Driver v465.19.01 and v460.39. The first file it finds stops at about the mid-40% status and I get the following error in the screen shot and I lose the unManic GUI. The error says check internet connection but that's not the case..  

 

 

Screenshot (2).png

f8a91e3b2271.log.gz

Link to comment

@Josh.5 - I was able to get everything up and running and it processed through around 1000 files without issue.  Now it is sitting with a bunch of stuff in the queue and it won't kick off.  Looking at the log, I see this:
 

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 6807, in get
return clone.execute(database)[0]
File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 4226, in __getitem__
return self.row_cache[item]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/foreman.py", line 423, in run
next_item_to_process = self.task_queue.get_next_pending_tasks()
File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/taskqueue.py", line 213, in get_next_pending_tasks
task_item = self.fetch_next_task_filtered('pending', self.sort_by, self.sort_order)
File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/taskqueue.py", line 155, in fetch_next_task_filtered
next_task.read_and_set_task_by_absolute_path(task_item.abspath)
File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/task.py", line 250, in read_and_set_task_by_absolute_path
self.read_task_settings_from_db()
File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/task.py", line 226, in read_task_settings_from_db
self.settings = self.task.settings.limit(1).get()
File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 6812, in get
(clone.model, sql, params))
unmanic.libs.unmodels.tasksettings.TaskSettingsDoesNotExist: <Model: TaskSettings> instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."task_id", "t1"."audio_codec", "t1"."audio_stream_encoder", "t1"."audio_codec_cloning", "t1"."audio_stream_encoder_cloning", "t1"."audio_stereo_stream_bitrate", "t1"."cache_path", "t1"."config_path", "t1"."keep_filename_history", "t1"."debugging", "t1"."enable_audio_encoding", "t1"."enable_audio_stream_transcoding", "t1"."enable_audio_stream_stereo_cloning", "t1"."enable_inotify", "t1"."enable_video_encoding", "t1"."library_path", "t1"."log_path", "t1"."number_of_workers", "t1"."out_container", "t1"."remove_subtitle_streams", "t1"."run_full_scan_on_start", "t1"."schedule_full_scan_minutes", "t1"."search_extensions", "t1"."video_codec", "t1"."video_stream_encoder", "t1"."overwrite_additional_ffmpeg_options", "t1"."additional_ffmpeg_options", "t1"."enable_hardware_accelerated_decoding" FROM "tasksettings" AS "t1" WHERE ("t1"."task_id" = ?) LIMIT ? OFFSET ?
Params: [1, 1, 0]

[2021-04-06 06:09:10,842 pyinotify WARNING] Event queue overflowed.

[W 210406 06:09:10 pyinotify:929] Event queue overflowed.

 

Any ideas on what I should do to resolve this?

Link to comment
@Josh.5 - I was able to get everything up and running and it processed through around 1000 files without issue.  Now it is sitting with a bunch of stuff in the queue and it won't kick off.  Looking at the log, I see this:
 
Traceback (most recent call last):File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 6807, in getreturn clone.execute(database)[0]File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 4226, in __getitem__return self.row_cache[item]IndexError: list index out of rangeDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_innerself.run()File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/foreman.py", line 423, in runnext_item_to_process = self.task_queue.get_next_pending_tasks()File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/taskqueue.py", line 213, in get_next_pending_taskstask_item = self.fetch_next_task_filtered('pending', self.sort_by, self.sort_order)File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/taskqueue.py", line 155, in fetch_next_task_filterednext_task.read_and_set_task_by_absolute_path(task_item.abspath)File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/task.py", line 250, in read_and_set_task_by_absolute_pathself.read_task_settings_from_db()File "/usr/local/lib/python3.6/dist-packages/unmanic/libs/task.py", line 226, in read_task_settings_from_dbself.settings = self.task.settings.limit(1).get()File "/usr/local/lib/python3.6/dist-packages/peewee.py", line 6812, in get(clone.model, sql, params))unmanic.libs.unmodels.tasksettings.TaskSettingsDoesNotExist:  instance matching query does not exist:SQL: SELECT "t1"."id", "t1"."task_id", "t1"."audio_codec", "t1"."audio_stream_encoder", "t1"."audio_codec_cloning", "t1"."audio_stream_encoder_cloning", "t1"."audio_stereo_stream_bitrate", "t1"."cache_path", "t1"."config_path", "t1"."keep_filename_history", "t1"."debugging", "t1"."enable_audio_encoding", "t1"."enable_audio_stream_transcoding", "t1"."enable_audio_stream_stereo_cloning", "t1"."enable_inotify", "t1"."enable_video_encoding", "t1"."library_path", "t1"."log_path", "t1"."number_of_workers", "t1"."out_container", "t1"."remove_subtitle_streams", "t1"."run_full_scan_on_start", "t1"."schedule_full_scan_minutes", "t1"."search_extensions", "t1"."video_codec", "t1"."video_stream_encoder", "t1"."overwrite_additional_ffmpeg_options", "t1"."additional_ffmpeg_options", "t1"."enable_hardware_accelerated_decoding" FROM "tasksettings" AS "t1" WHERE ("t1"."task_id" = ?) LIMIT ? OFFSET ?Params: [1, 1, 0][2021-04-06 06:09:10,842 pyinotify WARNING] Event queue overflowed.[W 210406 06:09:10 pyinotify:929] Event queue overflowed.

 
Any ideas on what I should do to resolve this?

Can you please send me your complete unmanic.log file collection and the version of unmanic that you are running?
Link to comment
22 hours ago, matt.shepker said:

@Josh.5 - I was able to get everything up and running and it processed through around 1000 files without issue.  Now it is sitting with a bunch of stuff in the queue and it won't kick off.  Looking at the log, I see this:
 


[2021-04-06 06:09:10,842 pyinotify WARNING] Event queue overflowed.

[W 210406 06:09:10 pyinotify:929] Event queue overflowed.

 

Any ideas on what I should do to resolve this?

I recently had a similar issue, josh will be able to confirm but it looks like the watcher overloaded. Try disabling and restarting. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.