Jump to content
We're Hiring! Full Stack Developer ×

How to create a custom kernel with modified /lib/modules, /lib/firmware


untan

Recommended Posts

I am using Unraid 6.9.2 with changes in /lib/modules, /lib/firmware because I want to use less common devices with Docker containers.
For customization, I used unraid-kernel-helper from ich999.

I tried the other day to use hyper-v on a windows 10 vm, but it doesn't seem to work.
I tried 6.10.0rc4.
It worked like magic.

I would like to incorporate the driver for the device into 6.10.0rc4.
It seems that unraid-kernel-helper has been closed to the public.

Why has it been closed to the public?

I have no choice, so I am trying to build it by myself by trial and error.
I understood the procedure regarding bzimage, bzmodules from the following page.
https://forums.unraid.net/topic/82625-kernelcustom-kernel-build-with-treaks-20200307-v683558419108naviveganfsv4r8125zen2/
I would like some tips regarding bzfirmware.


I would also like to understand the following pages
https://wiki.unraid.net/Manual/Release_Notes/Unraid_OS_6.9.0#Better_Module.2FThird_Party_Driver_Support
According to this description, the objective can be achieved without creating our own bz*.
I checked with the mount command and found that the overlay is set as follows.
###
/boot/bzfirmware on /lib/firmware type squashfs (ro,relatime,errors=continue)
overlay on /lib/firmware type overlay (rw,relatime,lowerdir=/lib/firmware,upperdir=/var/local/overlay/lib/firmware,workdir=/var/local/overlay-work/lib/firmware)
/boot/bzmodules on /lib/modules type squashfs (ro,relatime,errors=continue)
overlay on /lib/modules type overlay (rw,relatime,lowerdir=/lib/modules,upperdir=/var/local/overlay/lib/modules,workdir=/var/local/overlay-work/lib/modules)
###

is the upperdir, /var/local/overlay/*
exists in ram and seems to disappear after reboot.

How are nvidia drivers etc. working?

Thank you in advance.

Translated with www.DeepL.com/Translator (free version)

Edited by untan
Link to comment
2 hours ago, untan said:

Why has it been closed to the public?

Because nowadays you can integrate almost everything with plugins and precompiled Kernel modules.

Another reason is because the backup functionality from Unraid itself won't work with custom Kernels/modules.

To close it down this was purely my decision because maintaining giving support for it was also sometimes a little hard for less experienced people.

 

2 hours ago, untan said:

I would like to incorporate the driver for the device into 6.10.0rc4.

Which or what driver(s) do you need? Maybe I can help with a custom package and if someone else can use it I can create a plugin.

 

2 hours ago, untan said:

exists in ram and seems to disappear after reboot.

Exactly, this is how Unraid works, everything from the bz* files on your USB Boot device is loaded on boot into RAM and executed from there and will be wiped after a reboot and will be again loaded into RAM on a new boot.

 

2 hours ago, untan said:

How are nvidia drivers etc. working?

As mentioned above they use precompiled packages with everything (and Kernel modules) included and they are installed on boot with a routine that loads the modules and so on.

 

Keep in mind that is necessary to create new packages for every Unraid version (with the modules included) otherwise the driver plugins won't work.

I've automated this process on my server for every driver plugin that is available that when a new Unraid version is found the build starts and the packages are built and uploaded to their corresponding repositories.

Link to comment

ich777, I am honored to hear back from you.
I am helped by the packages you publish.
I have a lot of respect for you.


I am Japanese.
I have a commercial TV tuner board built in to watch Japanese TV broadcasts.
In order to get it to work, I need to build and incorporate the driver published in the following Git repository, which is maintained by others.


https://github.com/nns779/px4_drv

 

Unfortunately, there are few Japanese users of unraid and demand for it is low.

But, If this problem cannot be solved, it is a very big problem for us to pass it on, because we cannot upgrade Unraid.

Please help me.
Thank you in advance.

Link to comment
44 minutes ago, untan said:

I have a commercial TV tuner board built in to watch Japanese TV broadcasts.

Can you give me a bit more information on this? Is or are these common tuners that are used over there? Maybe I could integrate this in my DVB Driver plugin.

 

I also have to look into this if I can compile this since everything is in Japanese (at least I think)... :D

Do you know other people that maybe can test this or can you create a post on the Japanese language sub Forums here on the Forums (at the bottom at the main Unraid Forums page).

I don't know if there that many Japanese users from Unraid here: Click

 

I would really like to make sure that this is working properly.

 

51 minutes ago, untan said:

But, If this problem cannot be solved, it is a very big problem for us to pass it on, because we cannot upgrade Unraid.

On what Unraid version are you currently?

 

51 minutes ago, untan said:

ich777

Oh and if you want to mention me here so that I get a notification, make a @ and then write my forum name and click on the pop up:

image.png.8c505a755e0ec89a6ff405ac99cd5cbd.png

(I've stumbled over this by accident)

Link to comment

@ich777

 

The most major TV tuners in Japan are PT1, PT2, and PT3 boards from a manufacturer called EarthSoft.
These drivers seem to be built in as standard if you have LinuxKernel3.18 or later.

 

Unfortunately, these are no longer available and the market price is at a premium.

Therefore, I am using a board called "PX-MLT8PE" from a manufacturer called "PLEX", which is still available at mass retailers.
"PLEX", the manufacturer of this board, does not release the source code of drivers for Linux.
Therefore, if you want to use this board with Linux, you have to use unofficial drivers made by volunteers.

If you want to buy a new TV tuner board that works with Linux in Japan, you have to buy one from "PLEX".

Using ich777's unraid-kernel-helper, I built the px4_drv mentioned earlier, based on 6.9.2, with "CUSTOM_MODE" enabled.
I have been running unraid stably with bz* for more than a year.

It was over a year ago, so I may have left some things out, but the general procedure should have been as follows.


# clone git repository
$ git clone https://github.com/nns779/px4_drv.git
$ cd px4_drv

 

# firmware extraction & install
$ cd fwtool
$ make
$ wget http://plex-net.co.jp/plex/pxw3u4/pxw3u4_BDA_ver1x64.zip -O pxw3u4_BDA_ver1x64.zip
$ unzip -oj pxw3u4_BDA_ver1x64.zip pxw3u4_BDA_ver1x64/PXW3U4.sys
$ . /fwtool PXW3U4.sys it930x-firmware.bin
$ sudo mkdir -p /lib/firmware
$ sudo cp it930x-firmware.bin /lib/firmware/

 

# kernel module build & install
$ cd ... /driver
$ make
$ make -C /lib/modules/5.8.13-Unraid/build M=/usr/src/px4_drv/driver modules
$ make -C /lib/modules/5.8.13-Unraid/build M=/usr/src/px4_drv/driver INSTALL_MOD_DIR=kernel/drivers/misc/ modules_install

 

I have friends who use this px4_drv on ubuntu_server and CentOS, but unfortunately no Unraid users.
It is possible to create a post on the Japanese forum, but it does not look very promising due to the small number of users:(
 

Link to comment
21 minutes ago, untan said:

$ make -C /lib/modules/5.8.13-Unraid/build M=/usr/src/px4_drv/driver INSTALL_MOD_DIR=kernel/drivers/misc/ modules_install

You only install the Kernel module and the firmware file or am I wrong?

 

In the installer is also a UDEV file included, do you installed that too on your system?

 

Currently in the process of compiling the module and creating a basic package before I can integrate it somewhere.

Link to comment

@ich777

 

I apologize.
I don't have a good understanding about linux and what UDEV refers to.

In unraid_kernel_helper, the only additional processing I have done is what I wrote above.
I may have done an "apt install *" to add build dependencies.

I'm not sure if this will help, but I'm attaching my custom 6.9.2 bzmodules and bzfirmware that I have running on my computer right now.

Sorry for the trouble you are going through.
Thank you for your kind response.
Thank you in advance for your help.

 

bzfirmware bzmodules

Link to comment
3 minutes ago, untan said:

I'm not sure if this will help, but I'm attaching my custom 6.9.2 bzmodules and bzfirmware that I have running on my computer right now.

You've only replaced bzmodules and bzfirmware and not bzroot on your currently running machine am I right?

Do you have to do any modprobe or anything or is the tuner recognized on boot without any intervention by you?

 

For what Unraid version did you prefer a prebuild package for testing? I have currently compiled it for 6.9.2 but can also compile it for 6.10.0-rc4 if that's what you are after.

 

After everything is working fine I can implement this in the DVB Driver plugin, but some manual testing is needed first to make sure everything is working fine.

Link to comment

@untan Plex PX Series Tuners (Japan) are now available in the DVB Driver plugin (currently for Unraid 6.9.2 & 6.10.0-rc4 and will also be compiled for the coming Unraid versions):

image.thumb.png.2f6892d95c6e331d8e520f1bc199f93a.png

 

Please update or install the plugin and make sure that the plugin version is 2022.03.30, select the Plex PX Series Tuners (Japan), click Update and restart your server.

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...