[Support] ich777 - Nvidia/DVB/ZFS/iSCSI/MFT Kernel Helper/Builder Docker


Recommended Posts

Hello, a VM begginer question.

Is it still possible to passtrough the gpu to a VM with this build ?

I passed the gpu to a plex container and the transcode works like a charm but I m getting an error when I try to passtrough the gpu to a VM. 

I m wondering if it s the build or another issue that I need to look into (probably the latter ...)

 

 

Link to comment
32 minutes ago, suyac said:

Hello, a VM begginer question.

Is it still possible to passtrough the gpu to a VM with this build ?

I passed the gpu to a plex container and the transcode works like a charm but I m getting an error when I try to passtrough the gpu to a VM. 

I m wondering if it s the build or another issue that I need to look into (probably the latter ...)

 

 

Yes and no.

 

This was also adressed in the Linuxerserver.io thread (second post from the thread below see 2 and 4).

 

 

 

If you want to Passthrough a Graphics Card to a VM and also try to Passthrough the same Graphics Card to a Container that simply doesn't work (from my understanding what you can do is put two nVidia GPU's in your server and use one for VM and one for the Containers, but since I don't have another spare nVidia GPU lying around I can't test if that is working).

 

 

Think the other way around, you also can use the GPU only in one VM and not for more at the same time (but you can use the card for more than one Container - in my case I use it for transcoding in Emby and in a second Container (DebianBuster-Nvidia) for 3D Acceleration with Steam-Link to Stream my games to an old laptop).

Edited by ich777
  • Thanks 1
Link to comment
On 5/25/2020 at 9:13 PM, ich777 said:

Unraid Kernel Helper/Builder


With this container you can build your own customized Unraid Kernel.

 

Prebuilt images for direct download are on the bottum of this post.

 

By default it will create the Kernel/Firmware/Modules/Rootfilesystem with the nVidia drivers and also DVB drivers (currently DigitalDevices and LibreElec built in).

 

nVidia Driver installation: If you build the images with the nVidia drivers please make sure that no other process is using the graphics card otherwise the installation will fail and no nVidia drivers will be installed.

 

ZFS installation: Make sure that you uninstall every Plugin that enables ZFS for you otherwise it is possible that the built images are not working.

 

ATTENTION: Please read the discription of the variables carefully! If you started the container don't interrupt the build process, the container will automatically shut down if everything is finished.

I recommend to open a console window and type in 'docker attach Unraid-Kernel-Helper' (without quotes and replace 'Unraid-Kernel-Helper' with your Container name) to view the log output. (You can also open a log window from the Docker page but this can be verry laggy if you select much build options). The build itself can take very long depending on your hardware but should be done in ~30minutes (some tasks can take very long depending on your hardware, please be patient).

 

This is how the build of the Images is working (simplyfied):

  1. The build process begins as soon as the docker starts (you will see the docker image is stopped when the process is finished)
    Please be sure to set the build options that you need.
  2. Use the logs or better open up a Console window and type: 'docker attach Unraid-Kernel-Helper' (without quotes) to also see the log (can be verry laggy in the browser depending on how many components you choose).
    The whole process status is outlined by watching the logs (the button on the right of the docker).
  3. The image is built into /mnt/cache/appdata/kernel/output-VERSION by default. You need to copy the output files to /boot on your USB key manually and you also need to delete it or move it for any subsequent builds.
  4. There is a backup copied to /mnt/cache/appdata/kernel/backup-version. Copy that to another drive external to your Unraid Server, that way you can easily copy it straight onto the Unraid USB if something goes wrong.

 

THIS CONTAINER WILL NOT CHANGE ANYTHING TO YOUR EXISTING INSTALLATION OR ON YOUR USB KEY/DRIVE, YOU HAVE TO MANUALLY PUT THE CREATED FILES IN THE OUTPUT FOLDER TO YOUR USB KEY/DRIVE AND REBOOT YOUR SERVER.

 

PLEASE BACKUP YOUR EXISTING USB DRIVE FILES TO YOUR LOCAL COMPUTER IN CASE SOMETHING GOES WRONG!
I AM NOT RESPONSIBLE IF YOU BREAK YOUR SERVER OR SOMETHING OTHER WITH THIS CONTAINER, THIS CONTAINER IS THERE TO HELP YOU EASILY BUILD A NEW IMAGE AND UNDERSTAND HOW THIS IS WORKING.

 

UPDATE NOTICE: If a new Update of Unraid is released you have to change the repository in the template to the corresponding build number (I will create the appropriate container as soon as possible) eg: 'ich777/unraid-kernel-helper:6.8.3'.

 

Forum Notice: When something isn't working with or on your server and you make a forum post always include that you use a Kernel built by this container!

Note that LimeTech supports no custom Kernels and you should ask in this thread if you are using this specific Kernel when something is not working.

 

CUSTOM_MODE:
This is only for Advanced users!
In this mode the container will stop right at the beginning and will copy over the build script and the dependencies to build the kernel modules for DVB and joydev in the main directory (I highly recommend using this mode for changing things in the build script like adding patches or other modules to build, connect to the console of the container with: 'docker exec -ti NAMEOFYOURCONTAINER /bin/bash' and then go to the /usr/src directory, also the build script is executable).

 

Note: You can use the nVidia & DVB Plugin from linuxserver.io to check if your driver is installed correctly (keep in mind that some things will display wrong and or not showing up like the driver version in the nVidia Plugin - but you will see the installed grapics cards and also in the DVB plugin it will show that no kernel driver is installed but you will see your installed cards - this is simply becaus i don't know how their plugins work).

 

Thanks to @Leoyzen, klueska from nVidia and linuxserver.io for getting the motivation to look into this how this all works... ;)

 

For safety reasons I recommend you to shutdown all other containers and VM's during the build process especially when building with the nVidia drivers!

 

After you finished building the images i recommend you to delete the container! If you want to build it again please redownload it from the CA App so that the template is always the newest version!

 

Here you can download the prebuilt image:

  • Unraid Custom nVidia & DVB builtin v6.8.3: Download
    (nVidia driver: 440.82 | DD driver: 0.9.37 | LE driver: 1.4.0)
  • Unraid Custom nVidia & DVB & ZFS builtin v6.8.3: Download
    (nVidia driver: 440.82 | DD driver: 0.9.37 | LE driver: 1.4.0 | ZFS version: 0.8.4)
  • Unraid Custom nVidia builtin v6.8.3: Download
    (nVidia driver: 440.82)
  • Unraid Custom nVidia & ZFS builtin v6.8.3: Download
    (nVidia driver: 440.82)
  • Unraid Custom DVB builtin v6.8.3: Download
    (DD driver: 0.9.37 | LE driver: 1.4.0)
  • Unraid Custom DVB & ZFS builtin v6.8.3: Download
    (DD driver: 0.9.37 | LE driver: 1.4.0 | ZFS version: 0.8.4)
  • Unraid Custom ZFS builtin v6.8.3: Download
    (ZFS version: 0.8.4)

Hi this is so cool !! i have been looking for a solution for this. Could you make a prebuild image with nvidia and tbs drivers also?

Thank you very much for your work ;)

  • Thanks 1
Link to comment
6 hours ago, monstahnator said:

Hi this is so cool !! i have been looking for a solution for this. Could you make a prebuild image with nvidia and tbs drivers also?

Thank you very much for your work ;)

I have to look into this since I'm not using TBS cards.

Eventually I can build support for this into the Container.

 

Do you know where to get drivers from? Do they have a github repo with instructions?

 

What have you installed at the time?

 

Eventually you can help me test this... Write a short message to me and i will contact you after the weekend :)

Edited by ich777
Link to comment

Hey, @ich777 thanks so much for the ZFS support, now when new kernels come out, I don't need to wait for any other devs to compile nvidia / zfs etc which is fantastic!  And the ZFS works the same as the other ZFS plugin, so can switch back and forth if I so desired, but probably just going to use your version now, thanks!

Edited by Marshalleq
  • Like 1
Link to comment
16 hours ago, ich777 said:

I have to look into this since I'm not using TBS cards.

Eventually I can build support for this into the Container.

 

Do you know where to get drivers from? Do they have a github repo with instructions?

 

What have you installed at the time?

 

Eventually you can help me test this... Write a short message to me and i will contact you after the weekend :)

Hi thakns for your quick reply ;) That would be very cool. i have found the driver on the tbs webpage, and until now been trying to get the card passed to a ubuntu vm, but with no luck. Now i use the dvb plugin from linux server.io, but then i have to choose plex or dvb. I ready to test and give feedback when ever you need it ;).

 

https://www.tbsdtv.com/download/index.html?path=13&id=11

here is the info:

image.thumb.png.9514235408d2b80579f4d238aa22f2db.png

  • Like 1
Link to comment

I've also did a little research, it's recommended to use the open source drivers (also saw that you are using the open source drivers from your screenshot) eventually i got a few spare hours and can try to build the images.

 

I will report back.

Link to comment
On 5/25/2020 at 9:14 PM, ich777 said:

CHANGELOG:

 

06.06.2020:

  • Added TBS OpenSource drivers to the DVB build step

 

05.06.2020:

  • Corrected an error that zpools not loaded automatically on boot

 

04.06.2020:

  • Added ZFS to Kernel build options
  • Added option to include user specific Kernel patch files with automatic build
  • Words are hard (fixed a few typos)

 

31.05.2020:

  • Added possibility to insert custom Kernel version

 

26.05.2020:

  • Fixed build steps so that the latest 'nvidia-container-runtime' and 'nvidia-container-toolkit' can be built
  • Fixed CUSTOM_MODE sleep (if CUSTOM_MODE was set to true and the script 'buildscript.sh' was executed from the main directory it says again that CUSTOM_MODE is enabled)
  • Added end message with version numbers
  • Added a warning if build mode nVidia is selected (if a process uses the graphics card the installation of the nVidia drivers will fail)
  • Words are hard (fixed a few typos and sentences that were not comprehensible)

 

25.05.2020:

  • Initial release

Hi everyone I can confirm both TBS Opensource dvb tuner and nvidia card is working with this prebuild image. Transcoding is also working with plex thanks again for the great work ;) @ich777 

  • Like 2
Link to comment
5 minutes ago, scottc said:

Can we build nvidia with the latest beta 6.9.0-beta1? or is this only for stable unraid builds?

I have only developted this container for stable builds but I've also tried it for the latest development version 6.9.0-beta12 (there are a few modifications to do since it's using a newer kernel, but these are really small changes for DVB and nVidia).

 

If you really need this I can make a prebuilt container for official beta versions, but give me a few days (at least the weekend, weekend is family time) but keep in mind if i make something for that you have to type in few extra variables in the template then it will work.

 

screenshot.png.88a87f18321b5b15321a4abbe9a34a59.png

  • Like 1
Link to comment
2 minutes ago, ich777 said:

I have only developted this container for stable builds but I've also tried it for the latest development version 6.9.0-beta12 (there are a few modifications to do since it's using a newer kernel, but these are really small changes for DVB and nVidia).

 

If you really need this I can make a prebuilt container for official beta versions, but give me a few days (at least the weekend, weekend is family time) but keep in mind if i make something for that you have to type in few extra variables in the template then it will work.

 

screenshot.png.88a87f18321b5b15321a4abbe9a34a59.png

No hurry at all :)  Family time comes first in my book!

Link to comment
Just now, scottc said:

Do not worry about it for me,  It was more of a question than a request.  Go do family time :)

I build one with all enabled, give me 30 minutes. My little boy wants to play alone for now... :D

So I got a little bit of time, since this container does everything on its own. ;)

Link to comment
1 minute ago, ich777 said:

I build one with all enabled, give me 30 minutes. My little boy wants to play alone for now... :D

So I got a little bit of time, since this container does everything on its own. ;)

Awesome :)   If you don't mind,  could you tell me what custom variables are needed to use beta/rc's that would be great so I could just build out a new version when the next beta/rc is released

 

Thank you for your time and effort put into this

Link to comment
10 minutes ago, scottc said:

Awesome :)   If you don't mind,  could you tell me what custom variables are needed to use beta/rc's that would be great so I could just build out a new version when the next beta/rc is released

 

Thank you for your time and effort put into this

It's not that easy, feel free to contact me if a new build is out and I will build one if that's ok for you...

Beta builds are not that easy since my container is not really designed for this use case...

Link to comment
Just now, ich777 said:

It's not that easy, feel free to contact me if a new build is out and I will build one if that's ok for you...

Beta builds are not that easy since my container is not really designed for this use case...

No problem at all :) 

 

 

Thank you

Link to comment
3 hours ago, ich777 said:

It's not that easy, feel free to contact me if a new build is out and I will build one if that's ok for you...

Beta builds are not that easy since my container is not really designed for this use case...

That's kinda the main reason I'm interested in this, is for beta builds.  I was hoping this would autocompile against those newer kernels.  I like not having to rely on others to get an update.

Link to comment
17 minutes ago, Marshalleq said:

That's kinda the main reason I'm interested in this, is for beta builds.  I was hoping this would autocompile against those newer kernels.  I like not having to rely on others to get an update.

The solution to that is to build the system yourself then 😉

  • Like 1
Link to comment

 

6 hours ago, Marshalleq said:

Perhaps with this docker, I can begin to manage that. :)

You can do that, but you have to do the steps manually, all steps...

There is also a option to build lets say v6.8.3 with a newer kernel (not in the template because this is for advanced users) but as i said then ypu have to do all steps manually because you have to check the kernel modules manually (every new kernel adds more modules an functions).

I will quietly release the 6.9.0 contaier in the next few days and got also a idea in mind that you can automate the process the only downside is that you have to upgrade the server first to the preferred beta versio and then start building with a variable set that's not inthe default template.

Edited by ich777
  • Like 1
Link to comment

@Marshalleq & @scottc

The container is now finished.

If you want to build your own beta builds, please redownload the template from the CA App and change the following things:

  1. Change the repository from 'ich777/unraid-kernel-helper:6.8.3' to 'ich777/unraid-kernel-helper:6.9.0'
  2. Select the build options that you prefer
  3. Click on 'Show more settings...'
  4. Set Beta Build to 'true'
  5. Start the container and it will create the folders '/stock/beta' inside the main folder
  6. Place the files bzimage bzroot bzmodules bzfirmware in the folder from step 5 (I set a timeout of 2 minutes after the container starts and it will continue, if it doesn't find one or more files after this download the container will sleep forever and you simply have to restart the container)

(Please look at the description of the Beta Build variable in the template, there is an exemple where you can get it or if you allready installed it on your server you can simply copy it from /boot to the folder described in step 5)

 

 

Btw it was always possible to build another Kernel with for example 6.8.3 you have to simply create a new variable with the Key 'UNAME' and as Value you enter the prefered Kernel version eg: '4.19.107' or '5.5.8'.

 

 

!!! Please also note that if you build anything Beta keep an eye on the logs, especially when it comes to building the Kernel (everything befor it says this message '---Starting to build Kernel vYOURKERNELVERSION in 10 seconds, this can take some time, please wait!---' is very important) !!!

Edited by ich777
  • Like 1
Link to comment

Hi all,

 

I am looking for a way to pass through my nvidia GPU to the docker containers that I use (mainly Plex one from "plexinc")

 

I am currently running 6.9.1 beta version (was experimenting with new kernel to get temp readings)

 

I see we have compiled stable 6.8.3 version

 

Question:

Can I just download it, send it USB ,  reboot and all should work more or less

 

Or it's too risky?

 

I would rly like to avoid resetup all Dockers and other things that I use

 

Thx in advance

Link to comment
2 hours ago, ich777 said:

@Marshalleq & @scottc

The container is now finished.

If you want to build your own beta builds, please redownload the template from the CA App and change the following things:

  1. Change the repository from 'ich777/unraid-kernel-helper:6.8.3' to 'ich777/unraid-kernel-helper:6.9.0'
  2. Select the build options that you prefer
  3. Click on 'Show more settings...'
  4. Set Beta Build to 'true'
  5. Start the container and it will create the folders '/stock/beta' inside the main folder
  6. Place the files bzimage bzroot bzmodules bzfirmware in the folder from step 5 (I set a timeout of 2 minutes after the container starts and it will continue, if it doesn't find one or more files after this download the container will sleep forever and you simply have to restart the container)

(Please look at the description of the Beta Build variable in the template, there is an exemple where you can get it or if you allready installed it on your server you can simply copy it from /boot to the folder described in step 5)

 

 

Btw it was always possible to build another Kernel with for example 6.8.3 you have to simply create a new variable with the Key 'UNAME' and as Value you enter the prefered Kernel version eg: '4.19.107' or '5.5.8'.

 

 

!!! Please also note that if you build anything Beta keep an eye on the logs, especially when it comes to building the Kernel (everything befor it says this message '---Starting to build Kernel vYOURKERNELVERSION in 10 seconds, this can take some time, please wait!---' is very important) !!!

Just built out 6.9 beta 1 on my system and it worked perfectly. THANK YOU :)

  • Like 2
Link to comment
1 hour ago, Jus said:

Hi all,

 

I am looking for a way to pass through my nvidia GPU to the docker containers that I use (mainly Plex one from "plexinc")

 

I am currently running 6.9.1 beta version (was experimenting with new kernel to get temp readings)

 

I see we have compiled stable 6.8.3 version

 

Question:

Can I just download it, send it USB ,  reboot and all should work more or less

 

Or it's too risky?

 

I would rly like to avoid resetup all Dockers and other things that I use

 

Thx in advance

With a Beta you risk everytime loosing some data or messing something up.

But it should not mess up your complete server (I hope you are doing backups with the CA Backup App).

You can always revert the changes if you do Backups.

 

It should not mess up something if you are currently using the beta1, you can also download the prebuilt 6.9.0 beta1 from the first post (doesn't matter if there are things included like DVB or ZFS if you don't need it).

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.