Jump to content
dee31797

Virt-Manager, Intel-GPU-Tools and more Dockers

380 posts in this topic Last Reply

Recommended Posts

1 minute ago, page3 said:

I've been trying a number of options to shrink .ts recorded stuff and all have been too complicated, slow or clunky.

 

Recordings-autoconverter is working beautifully. Thanks :)

Very happy to hear! That's the exact reason I made it for myself and it's good news that it helped others as well. 😁

Share this post


Link to post

having errors. Not sure what it is even pointing at.

 

ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-9ubuntu2)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-vaapi --enable-nvenc --enable-cuvid --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-postproc --enable-small --enable-version3 --extra-cflags='-I/opt/ffmpeg/include -I/opt/ffmpeg/include/ffnvcodec' --extra-ldflags=-L/opt/ffmpeg/lib --extra-libs=-ldl --prefix=/opt/ffmpeg
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mpegts, from '/watch/watch/Live PD Presents  PD Cam S04E04.ts':
  Duration: 00:25:09.63, start: 1.400000, bitrate: 3721 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuvj420p(pc, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 126 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vaapi))
  Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Too many packets buffered for output stream 0:1.
[libfdk_aac @ 0x560dc3006ac0] 2 frames left in the queue on closing
Conversion failed!

 

Share this post


Link to post
43 minutes ago, scubieman said:

having errors. Not sure what it is even pointing at.

 

 

hi scubieman,

I believe that error is caused when the audio codec is already aac in the original recording file.  The video you're getting from your TV provider is beyond the capabilities of this application.  I recommend using a more sophisticated and customizable application such as Handbrake, Unmanic or Tdarr (all available in CA Apps).  Those applications have the ability you need to check for different audio codecs, custom conversion options, etc. but at the same time still allowing for automatic file detection and automatic conversion like recordings-converter. 

Share this post


Link to post

Just found out about this fantastic plugins/docker images, great work.

 

I have a question about ComSkip and Recordings-autoconverter:

 

I keep my recordings in sub-folders (like recordings/Some Show/Season 01/Some Episode - S01E01.ts) because this is how nextpvr (which I am using) is set to handle and this gives me best compatibility with Plex and Emby, since they use the same structure for automatic matches.

 

Do the two plugins support working with subfolders also for the output? Putting all recordings in a single folder would require manual intervention which I would want to avoid.

 

Thanks for helping.

Share this post


Link to post
6 minutes ago, Coxeroni said:

Just found out about this fantastic plugins/docker images, great work.

 

I have a question about ComSkip and Recordings-autoconverter:

 

I keep my recordings in sub-folders (like recordings/Some Show/Season 01/Some Episode - S01E01.ts) because this is how nextpvr (which I am using) is set to handle and this gives me best compatibility with Plex and Emby, since they use the same structure for automatic matches.

 

Do the two plugins support working with subfolders also for the output? Putting all recordings in a single folder would require manual intervention which I would want to avoid.

 

Thanks for helping.

auto-comskip uses separate input and output directories and supports sub-directories.  The output directory or output sub-directory will match the input directory structure.  Ex. /input/tvshow/drama/mytvshowdramaS01E01.ts -> /output/tvshow/drama/tvshowdramaS01E01.ts

 

Recordings-autoconverter uses the same the directory for input and output (in place conversion).  Ex. /watch/tvshow/drama/mytvshowdramaS01E01.ts -> /watch/tvshow/drama/tvshowdramaS01E01.mp4.

 

Hope that helps explain it.

Share this post


Link to post
Posted (edited)
3 hours ago, dee31797 said:

auto-comskip uses separate input and output directories and supports sub-directories.  The output directory or output sub-directory will match the input directory structure.  Ex. /input/tvshow/drama/mytvshowdramaS01E01.ts -> /output/tvshow/drama/tvshowdramaS01E01.ts

 

Recordings-autoconverter uses the same the directory for input and output (in place conversion).  Ex. /watch/tvshow/drama/mytvshowdramaS01E01.ts -> /watch/tvshow/drama/tvshowdramaS01E01.mp4.

 

Hope that helps explain it.

This helps, thank you a lot. That indeed is killer and might work out well in my (somewhat) complex setup :)

 

I will give it a whirl then, thanks.

 

Edit: So I could first comskip from folder a to folder b and watch folder b to get it autoconverted, right?

 

Edit2: Auto-comskip does cut the ads out of the file if it finds them, right? I have tested some files and apart from stripping some redundant streams (e.g. teletext ) nothing happened and the files are as long as before. I use a comskip.ini for germany. Don't where to start to debug this. Any option I am missing?

 

Any help is appreciated.

Edited by Coxeroni

Share this post


Link to post

OK my latest issues were relates to me being not able to use Comskip properly ;)

Resolved by setting output to edl file :)

 

Another thing:

Recordings converter seems to fail on file names with German umlauts (ä, ö, ü).

Quote

[autovideoconverter] 0 title(s) to process.
/run/s6/services/autovideoconverter/autovideoconverter: line 250: basename: unbound variable

If I rename the files, everything is working fine.

 

@dee31797:

have you witnessed something like that before?

Share this post


Link to post
27 minutes ago, Coxeroni said:

OK my latest issues were relates to me being not able to use Comskip properly ;)

Resolved by setting output to edl file :)

 

Another thing:

Recordings converter seems to fail on file names with German umlauts (ä, ö, ü).

If I rename the files, everything is working fine.

 

@dee31797:

have you witnessed something like that before?

No sorry I don't have any recordings with German umlauts

Share this post


Link to post

Shouldn't this thread be moved to Docker Containers? Let me know and I will do it.

Share this post


Link to post
49 minutes ago, Coxeroni said:

Too bad. Anything I can do about it?

sorry again I'm really stumped on this, my first guess is it's something to do with locales but alpine is very difficult to deal with in that regard.

Share this post


Link to post

Ok thanks anyway. Maybe I can replace the umlauts with my mediocre Linux skills prior to conversion. Have to Google my way through this.

Share this post


Link to post

Hi,

 

I installed recordings-converter and tested it out on a couple of files. Conversion logs are attached.

 

A few questions:

1) I think I've set it up correctly to use my GPU. Is the attached configuration correct?
2) While the docker was doing its work, I did see load on the GPU and an ffmpeg process. But there was still 5-10% load on the CPU (and no other docker should have been using that much CPU resources at the time). Is this expected behavior?
3) Does the container work on multiple files sequentially or concurrently? So far I've tested the container only on folders with one recording in them. When I switch to my entire recordings folder, I just want to make sure my other containers/VMs etc aren't affected. 
4) I am using my GPU for hardware transcoding in Plex and Emby as well. It is my understanding that as long as my GPU isn't passed through to a VM, multiple Dockers can use the GPU concurrently. Is my understanding correct?

 

Thanks for your help and a bigger thanks for all your work on these containers. Definitely makes my life easier!

Recordings-Converter Configuration.jpg

postProcess.18-03-2020-0712.log postProcess.18-03-2020-0839.log

Share this post


Link to post
6 minutes ago, rragu said:

Hi,

 

I installed recordings-converter and tested it out on a couple of files. Conversion logs are attached.

 

A few questions:

1) I think I've set it up correctly to use my GPU. Is the attached configuration correct?
2) While the docker was doing its work, I did see load on the GPU and an ffmpeg process. But there was still 5-10% load on the CPU (and no other docker should have been using that much CPU resources at the time). Is this expected behavior?
3) Does the container work on multiple files sequentially or concurrently? So far I've tested the container only on folders with one recording in them. When I switch to my entire recordings folder, I just want to make sure my other containers/VMs etc aren't affected. 
4) I am using my GPU for hardware transcoding in Plex and Emby as well. It is my understanding that as long as my GPU isn't passed through to a VM, multiple Dockers can use the GPU concurrently. Is my understanding correct?

 

Thanks for your help and a bigger thanks for all your work on these containers. Definitely makes my life easier!

 

postProcess.18-03-2020-0712.log 64.67 kB · 0 downloads postProcess.18-03-2020-0839.log 45.24 kB · 0 downloads

1) yep looks good. post process logs confirm.

2) yep, the cpu will always need to be utilized somewhat, even for GPU decoding/encoding.  The advantage is only "5-10%" CPU for ~10 mins vs 100% CPU for an hour for example, based on video file.

3) only 1 file at a time.

4) multiple containers can use the gpu at the same time.

Share this post


Link to post
On 9/18/2019 at 3:21 AM, dee31797 said:

Hi @robsch, I will have to think about how to do that and test it out.  Right now it can only move the final video to "/output". Also, I kinda like everything for me to be in the root of /output so I would have to make it an optional setting at startup.  I'll let you know what I come up with. My first guess would be to end the script with "mv $1 /output$1".  Easiest way I can think of right off hand, it would move for example "/watch/movie/movie.mp4" to "/output/watch/movie/movie.mp4" would that work for you?

 

The comskip.ini is only for comskip settings like commercial detection schemes, cpu threads, etc.

Hi @dee31797, thanks for the effort! However running this docker doesnt remove all comersials for me. Is there any chance to have the option to edit the comskip.ini? Thanks!

Share this post


Link to post
29 minutes ago, dee31797 said:

Hi,

Yes certainly! Please find that info here: https://hub.docker.com/r/djaydev/auto-comskip

 

Thanks for the quick reply and sorry for being such a noob. So I edit in this line somewhere? "-v /docker/appdata/auto-comskip/comskip.ini:/opt/comskip.ini". Where? Also I place the comskip.ini in thea ppdata/auto-comskip/ folder? 🙂

Share this post


Link to post
45 minutes ago, Adam_ said:

Thanks for the quick reply and sorry for being such a noob. So I edit in this line somewhere? "-v /docker/appdata/auto-comskip/comskip.ini:/opt/comskip.ini". Where? Also I place the comskip.ini in thea ppdata/auto-comskip/ folder? 🙂

That line is referring to the volume that needs to be added to the container. 

Instructions on volumes in Unraid can be found here: https://wiki.unraid.net/UnRAID_6/Docker_Management#Volume_Mappings

and if you prefer video: https://www.youtube.com/watch?v=ISJczs06pD8

 

 

You can place the comskip.ini anywhere you want on Unraid's side, but on the container side it has to be at " /opt/comskip.ini ".

 

Share this post


Link to post
On 3/29/2020 at 7:27 PM, dee31797 said:

That line is referring to the volume that needs to be added to the container. 

Instructions on volumes in Unraid can be found here: https://wiki.unraid.net/UnRAID_6/Docker_Management#Volume_Mappings

and if you prefer video: https://www.youtube.com/watch?v=ISJczs06pD8

 

 

You can place the comskip.ini anywhere you want on Unraid's side, but on the container side it has to be at " /opt/comskip.ini ".

 

Thanks, I have read and watched the Spaceinvader video but Im still a bit lost because I dont recognize the format of the string "-v /docker/appdata/auto-comskip/comskip.ini:/opt/comskip.ini". I have installed other dockers with custom host path etc before but only when watching Spaceinvaders guides as Im totally new to unRaid and Linux 😅

 

Should I add a path named "Host Path 5" with container path "/opt/" and Host Path pointing to where my unraid stores my docker and place the comskip.ini file in there?

 

Thanks for your help!

Share this post


Link to post
18 hours ago, Adam_ said:

Thanks, I have read and watched the Spaceinvader video but Im still a bit lost because I dont recognize the format of the string "-v /docker/appdata/auto-comskip/comskip.ini:/opt/comskip.ini". I have installed other dockers with custom host path etc before but only when watching Spaceinvaders guides as Im totally new to unRaid and Linux 😅

 

Should I add a path named "Host Path 5" with container path "/opt/" and Host Path pointing to where my unraid stores my docker and place the comskip.ini file in there?

 

Thanks for your help!

Forget about that string and it's formatting, that's for running docker from the command line. 

From the webgui you have to add what unraid calls a "path" to the comskip docker container.  In the path, on the host side will be your comskip.ini file, and on the container side will be " /opt/comskip.ini ".  

example:

docker settings

add path

host path: /mnt/user/myshare/comskip/comskip.ini (just an example, please use your directory path)

container path: /opt/comskip.ini

Share this post


Link to post
2 hours ago, dee31797 said:

Forget about that string and it's formatting, that's for running docker from the command line. 

From the webgui you have to add what unraid calls a "path" to the comskip docker container.  In the path, on the host side will be your comskip.ini file, and on the container side will be " /opt/comskip.ini ".  

example:

docker settings

add path

host path: /mnt/user/myshare/comskip/comskip.ini (just an example, please use your directory path)

container path: /opt/comskip.ini

Thanks for your patience and help. I think I get it now.

In my case:

host path: /mnt/disks/Kingston_SHPM2280P2_240G_50026B727604BA4C/appdata/auto-comskip/

container path: /opt/

 

No comskip.ini after the last / right? Although I have put the comskip.ini file in the folder /mnt/disks/Kingston_SHPM2280P2_240G_50026B727604BA4C/appdata/auto-comskip/

 

With the settings above, auto-comskip only moves the file from watch folder to output folder without doing its thing 

Share this post


Link to post
Quote

Thanks for your patience and help. I think I get it now.

no prob

 

Quote

In my case:

host path: /mnt/disks/Kingston_SHPM2280P2_240G_50026B727604BA4C/appdata/auto-comskip/

container path: /opt/

Ok since you've provided the actual paths, please follow my instructions exactly.

docker settings

add path

host path: /mnt/disks/Kingston_SHPM2280P2_240G_50026B727604BA4C/appdata/auto-comskip/comskip.ini

container path: /opt/comskip.ini

 

Quote

No comskip.ini after the last / right? Although I have put the comskip.ini file in the folder /mnt/disks/Kingston_SHPM2280P2_240G_50026B727604BA4C/appdata/auto-comskip/

why not?  I've said that several times, definitely with comskip.ini after the last / please...

 

Quote

With the settings above, auto-comskip only moves the file from watch folder to output folder without doing its thing 

That's because you mapped the whole folder instead of the comskip.ini file, now the container can't find the scripts it needs to do its thing

Share this post


Link to post

Great software, Installed your handbreak docker earlier today. And just found your Recordings-autoconverter docker. It seems to be exactly what I want. Thank you.

 

I have one question: Where does it put the converted file? Does it put it back into the directory where it found the file?

 

I added an additional path /output, maybe that will work.

Share this post


Link to post
8 hours ago, James Struble said:

Great software, Installed your handbreak docker earlier today. And just found your Recordings-autoconverter docker. It seems to be exactly what I want. Thank you.

 

I have one question: Where does it put the converted file? Does it put it back into the directory where it found the file?

 

I added an additional path /output, maybe that will work.

recordings-converter puts it back into the directory where it found the file.

Share this post


Link to post
Posted (edited)
On 5/6/2019 at 6:48 PM, positronicP said:

I followed what @dee31797 posted - here are the itemized steps:

  1. turn on netcat-openbsd in Nerd Pack, apply, and restart computer
  2. open the virt-manager docker GUI - you'll still have the error
  3. in the Docker container, go File > Add Connection
  • Hypervisor: QEMU
  • Select connect to remote host over SSH
  • Hostname: the internal network IP of your server
  • Hit connect

It will then ask you to accept certificate, then for your unRaid password. Once cleared you're in

Hello,

I've installed netcat-openbsd and tried to connect.

Unfortunately when I try to connect I get:

image.png.50ead2a9487f2abe3451111edfa07154.png

If I write YES an error appears saying it can connect.

If I write my root password I get this screen:

image.png.b69c041899f8f6b5b70deceb96914e8f.png

If I type yes and then OK - I get that it can connect.

 

What I'm doing wrong?

 

EDIT: Probabily isn't possible with my setup ; I've setup that SSH is only possible with a SSHCert.

Edited by TDA

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.