[KERNEL]custom kernel build with treaks (2020.03.07 v6.8.3|5.5.8|4.19.108|NAVI|VEGA|NFSv4|R8125|Zen2)


Recommended Posts

42 minutes ago, scorcho99 said:

@segator , did you ever find decent instructions on a custom kernel for unraid? I'm in a similar kind of situation. I would like to add some modules to test something before making a better request.

https://wiki.unraid.net/Building_a_custom_kernel

 

As the warning notes this is very outdated, but the basic steps are the same. If you can't adapt this (substituting the proper packages, kernel source, headers, download the correct unraid package and applying the unraid .patches, etc) you are probably in over your head. 

 

https://gist.github.com/gfjardim/c18d782c3e9aa30837ff

 

This script is slightly newer, which you can also analyze to see the basic steps. If you need guidance beyond that I would say building a custom kernel is ill advised.

Edited by Skitals
Link to comment
On 12/18/2019 at 12:50 PM, Skitals said:

Can you explain your problem? Is it locking up at boot, or randomly during use? With my 5700 XT, I would randomly crash unraid during gaming or even just using chrome in the win 10 vm. I made a ton of changes, but I finally got it stable. I settled on 6.8.0-rc5 with the kernel from the first thread. I added a second GPU in slot 3 I have set in my bios as the initial video device. With this setup I no longer have to pass a vbios to the vm. I also updated to Adrenalin 2020 drivers. With those changes, instead of locking up the entire host I would "only" lose signal where I was previously crashing. I could hear game audio continue, but the only recourse was the force stop the vm. The final fix was to DISABLE Radeon Anti-Lag and Radeon Enhanced Sync in Adrenalin. With that final change I am 100% stable and have no problem restarting my vm.

 

That's a long way of saying check if you have Radeon Anti-Lag and Radeon Enhanced Sync on and turn them OFF. They default to on, at least in Adrenalin 2020. The second gpu might not be necessary, it might have just changed the behavior from crashing all of unraid to only having for force stop the vm. Also, use q35-4.0.1 (or newer) if you want gen4 pcie speed without xml changes.

I think I had unrelated hardware issues. I've done a reseat of ram, gpu, etc. and it seems a lot more stable now.

Link to comment
  • 2 weeks later...
On 12/18/2019 at 7:50 AM, Skitals said:

The final fix was to DISABLE Radeon Anti-Lag and Radeon Enhanced Sync in Adrenalin. With that final change I am 100% stable and have no problem restarting my vm.

Thank you for this. I was wondering what I did to start causing super low FPS and crashing, and reading this made it click in my head. Back to normal 200+ fps. :)

 

In for future updates of this kernel as well.

 

Just got my 3900X/X570 system up and running with almost everything working great (CPU pinning, GPU/USB pass though to gaming VM, dockers, plugins/apps, etc.). Although, I haven't even taken a stab at getting RGB control working. I found the docker for MSI boards, but I'm using a Gigabyte Aorus Master.

 

I also haven't figured out how to pass through the on board audio controller to my VM. Having a weird issue where it locks up the entire Unraid OS went I attempt to boot with it passed through. Found some threads with others having the same problem, and I gave up. My workaround is to just use USB pass through to plug in my Xbox Wireless adapter for Windows for sound through my Xbox headset. Aside from this, my gaming VM is incredibly stable with no other issues.

 

After all of this, I came across this thread after attempting to setup the temp settings, and realizing it wasn't seeing them all/correctly. Installed and tested it out to see if it worked, and saw the temps showing in Dynamix finally.

 

Unfortunately, I'm new to Unraid, and this is my first ever build that I started fresh with 6.8 stable. I haven't found a way to convert to a next release or downgrade a fresh install. So, I had to revert back to stable, and will just have to wait and see what happens with this and the next releases.

Link to comment
1 hour ago, carnivorebrah said:

Thank you for this. I was wondering what I did to start causing super low FPS and crashing, and reading this made it click in my head. Back to normal 200+ fps. :)

 

In for future updates of this kernel as well.

 

Just got my 3900X/X570 system up and running with almost everything working great (CPU pinning, GPU/USB pass though to gaming VM, dockers, plugins/apps, etc.). Although, I haven't even taken a stab at getting RGB control working. I found the docker for MSI boards, but I'm using a Gigabyte Aorus Master.

 

I also haven't figured out how to pass through the on board audio controller to my VM. Having a weird issue where it locks up the entire Unraid OS went I attempt to boot with it passed through. Found some threads with others having the same problem, and I gave up. My workaround is to just use USB pass through to plug in my Xbox Wireless adapter for Windows for sound through my Xbox headset. Aside from this, my gaming VM is incredibly stable with no other issues.

 

After all of this, I came across this thread after attempting to setup the temp settings, and realizing it wasn't seeing them all/correctly. Installed and tested it out to see if it worked, and saw the temps showing in Dynamix finally.

 

Unfortunately, I'm new to Unraid, and this is my first ever build that I started fresh with 6.8 stable. I haven't found a way to convert to a next release or downgrade a fresh install. So, I had to revert back to stable, and will just have to wait and see what happens with this and the next releases.

 

Passing through onboard audio is a lost cause, unfortunately. Luckily I've always used a USB dac (astro mixamp pro) it it's no real loss to me.

Link to comment

@carnivorebrah  I am new here too.  Came across this thread when searching for X570 as I have an MSI X570 Prestige Creation.  Can you tell me what you found for the MSI Board docker?   I have searched but nothing for MSI comes up.   I too would like the ability to turn off the god aweful rainbow RGB on my RAM.  Am wishing I hadn't bought it.  I have a neat Lian Li case on my desk, but the rainbow is killing me!

Link to comment
56 minutes ago, leftpawdog said:

@carnivorebrah  I am new here too.  Came across this thread when searching for X570 as I have an MSI X570 Prestige Creation.  Can you tell me what you found for the MSI Board docker?   I have searched but nothing for MSI comes up.   I too would like the ability to turn off the god aweful rainbow RGB on my RAM.  Am wishing I hadn't bought it.  I have a neat Lian Li case on my desk, but the rainbow is killing me!

 

  • Like 1
Link to comment
On 1/1/2020 at 6:34 AM, phbigred said:

Was there plans for a release updated for 6.8 stable?

 

1 hour ago, Arizuia said:

+1

No, 6.8-stable do not have suitable linux 5.x patches, it takes lots of risk to make it.If you wanna using this kernel, revert to latest version of unraid which support 5.x linux (like 6.8rc5 which I'm using).

  • Thanks 1
Link to comment
On 12/18/2019 at 1:50 PM, Skitals said:

Can you explain your problem? Is it locking up at boot, or randomly during use? With my 5700 XT, I would randomly crash unraid during gaming or even just using chrome in the win 10 vm. I made a ton of changes, but I finally got it stable. I settled on 6.8.0-rc5 with the kernel from the first thread. I added a second GPU in slot 3 I have set in my bios as the initial video device. With this setup I no longer have to pass a vbios to the vm. I also updated to Adrenalin 2020 drivers. With those changes, instead of locking up the entire host I would "only" lose signal where I was previously crashing. I could hear game audio continue, but the only recourse was the force stop the vm. The final fix was to DISABLE Radeon Anti-Lag and Radeon Enhanced Sync in Adrenalin. With that final change I am 100% stable and have no problem restarting my vm.

 

That's a long way of saying check if you have Radeon Anti-Lag and Radeon Enhanced Sync on and turn them OFF. They default to on, at least in Adrenalin 2020. The second gpu might not be necessary, it might have just changed the behavior from crashing all of unraid to only having for force stop the vm. Also, use q35-4.0.1 (or newer) if you want gen4 pcie speed without xml changes.

I’m planning to do something similar, but I’m wondering did this solution eliminate all the problems you had from crashing and the rest bug and the system lock and the random black screens while browsing through chrome? 

 

Link to comment

@Leoyzen Unfortunately this custom kernel still does not work for me, the system always crash after I run the reset application from gnif on level1tech forum

I have Vega frontier Edition that have different vendor id listed on the vega kernel patch here

Maybe that causing the problem

 

I think this line 14 and 15

+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATI, 0x687f, quirk_no_bus_reset);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATI, 0xaaf8, quirk_no_bus_reset);

must be replaced by this, to reflect my system

+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATI, 0x6863, quirk_no_bus_reset);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATI, 0xaaf8, quirk_no_bus_reset);

 

can you tell me how to apply patch unraid kernel?

Edited by hygoggx
typo
Link to comment
On 1/12/2020 at 7:05 PM, Idris Gsmalla said:

I’m planning to do something similar, but I’m wondering did this solution eliminate all the problems you had from crashing and the rest bug and the system lock and the random black screens while browsing through chrome? 

 

I have had zero--absolutely zero--issues with the latest drivers and the above "features" disabled.

Link to comment
  • 2 weeks later...
On 12/21/2019 at 12:13 AM, Skitals said:

https://wiki.unraid.net/Building_a_custom_kernel

 

As the warning notes this is very outdated, but the basic steps are the same. If you can't adapt this (substituting the proper packages, kernel source, headers, download the correct unraid package and applying the unraid .patches, etc) you are probably in over your head. 

 

https://gist.github.com/gfjardim/c18d782c3e9aa30837ff

 

This script is slightly newer, which you can also analyze to see the basic steps. If you need guidance beyond that I would say building a custom kernel is ill advised.

 

On 1/13/2020 at 4:53 AM, bubbl3 said:

I really appreciate this kind of work, would be wonderful if you could explain how you compiled it and/or how to compile kernel modules. I am trying to add support for RTL8156 USB3 2.5G chipset and finding many hurdles.

The compiling procedure updated in the main thread.

  • Like 1
Link to comment
  • 4 weeks later...

6.8.3 is out, here is the new kernel and some tweaks:

  1. Add Vega Reset Patch
  2. Add Navi Reset Patch
  3. Enable NFSv4 in kernel(God damned, we finally get nfsv4 to work)
  4. Add R8125 out tree driver.
  5. AMD onboard audio/usb controller flr patch, you can now passthrough them.
  6. Provide two version (linux-5.5.8 and linux-4.19.108) in case of bug. Notice that the linux-4.19.108 version still don't have AMD Zen 2 suppport.

For those who want to use NFSv4:

NFSv4 have some change compared to v2/v3, it must have root share and the nfs-utils/unRaid can't handle it well for now.You must add this script to UserScript Plugin triggered when array start:

#!/bin/bash
# Add NFSv4 root
echo '"/mnt/user/" -async,no_subtree_check,fsid=0 *(sec=sys,rw,insecure,anongid=100,anonuid=99,all_squash,crossmnt)' >> /etc/exports
# Load configuration
exportfs -ra
# Tunning
mkdir -p /var/lib/nfs/v4recovery
sed -i 's@/usr/sbin/rpc.nfsd 8@/usr/sbin/rpc.nfsd 32@g' /etc/rc.d/rc.nfsd
# Restart Services
/etc/rc.d/rc.nfsd restart

And if you have trouble mount nfsv4, you can specify mount vers=3 on client.

 

6.8.3-4.19.108.zip 6.8.3-5.5.8.zip

Edited by Leoyzen
Link to comment
9 hours ago, Leoyzen said:

6.8.3 is out, here is the new kernel and some tweaks:

  1. Add Vega Reset Patch
  2. Add Navi Reset Patch
  3. Enable NFSv4 in kernel(God damned, we finnaly get nfsv4 to work)
  4. Add R8125 out tree driver.
  5. AMD onboard audio/usb controller flr patch.
  6. Provide two version (linux-5.5.8 and linux-4.19.108) in case of bug. Notice that linux-4.19.108 still don't have AMD Zen 2 suppport.

For those who want to use NFSv4:

NFSv4 have some change compared to v2/v3, it must have root share and the nfs-utils version can't handle it well.You must add this script to UserScript Plugin triggered when array start:


#!/bin/bash
# Add NFSv4 root
echo '"/mnt/user/" -async,no_subtree_check,fsid=0 *(sec=sys,rw,insecure,anongid=100,anonuid=99,all_squash,crossmnt)' >> /etc/exports
# Load configuration
exportfs -ra
# Tunning
mkdir -p /var/lib/nfs/v4recovery
sed -i 's@/usr/sbin/rpc.nfsd 8@/usr/sbin/rpc.nfsd 32@g' /etc/rc.d/rc.nfsd
# Restart Services
/etc/rc.d/rc.nfsd restart

And if you have trouble mount nfsv4, you can specify mount vers=3 on client.

 

6.8.3-5.5.8.zip 13.31 MB · 0 downloads 6.8.3-4.19.108.zip 13.03 MB · 0 downloads 6.8.3-5.5.8.zip 13.31 MB · 0 downloads

Thank a lot, i go test it.

 

Why 2 file for 6.8.3 - 5.5.8 ?

 

EDIT : With your old bios and 6.7.2, my RX480 work with passtrough. With 6.8.3 and your new bios, stuck in D3 stat ... :(

EDIT : Fine with Bios upgrade ..

Edited by Balooforever
Link to comment

@Leoyzen, is there any chance you could compile a kernel for me with CONFIG_NO_HZ_FULL?

I am trying to use the nohz_full kernel argument to solve a latency problem I'm having with my NVIDIA card in passthrough to my Win10 VM.

I noticed that my kernel, Linux unRAID 4.19.107-Unraid, was not compiled with this option, as can be seen here:

root@unRAID:/usr/src/linux# dmesg | grep -Ei "rcu|hz"
[    0.000000] Command line: BOOT_IMAGE=/bzimage vfio-pci.ids=14e4:168e,1022:145c,144d:a804,10de:1c03,10de:10f1 pcie_aspm=off kvm_amd npt=1 nested=1 avic=1 rcu_nocb_poll nohz_full=1-11 rcu_nocbs=0-11 isolcpus=1-11 pcie_acs_override=multifunction initrd=/bzroot
[    0.000000] tsc: Detected 3200.000 MHz processor
[    0.106486] Kernel command line: BOOT_IMAGE=/bzimage vfio-pci.ids=14e4:168e,1022:145c,144d:a804,10de:1c03,10de:10f1 pcie_aspm=off kvm_amd npt=1 nested=1 avic=1 rcu_nocb_poll nohz_full=1-11 rcu_nocbs=0-11 isolcpus=1-11 pcie_acs_override=multifunction initrd=/bzroot
[    0.106567] Housekeeping: nohz unsupported. Build with CONFIG_NO_HZ_FULL
[    0.146514] rcu: Hierarchical RCU implementation.
[    0.146515] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=16.
[    0.146516] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=16
[    0.146693] rcu:     Offload RCU callbacks from CPUs: 0-11.
[    0.146694] rcu:     Poll for callbacks from no-CBs CPUs.
[    0.392301] rcu: Hierarchical SRCU implementation.
[   12.841669] tsc: Refined TSC clocksource calibration: 3200.075 MH

So far I have not had much luck figuring out building a kernel. I've been using the outdated documentation on the unRAID wiki (https://wiki.unraid.net/Building_a_custom_kernel) and suspect I'm not downloading/installing the proper unRAID packages and/or kernel headers. These are the commands I've used so far, after downloading the kernel source into /boot/packages/: 

 

tar -C /usr/src/ -zxvf /boot/packages/linux-4.19.107.tar.gz
ln -sf /usr/src/linux-4.19.107 /usr/src/linux

mkdir /mnt/dev
unzip /boot/packages/unRAIDServer-6.8.3-x86_64.zip -d /mnt/dev/
cp -rf /usr/src/linux-4.19.107-Unraid/* /usr/src/linux/
cp -f /usr/src/linux-4.19.107-Unraid/.config /usr/src/linux/

 

#Install kernel headers (I can't figure out the correct headers to use here. I found these on a manjaro repo http://repo.manjaro.org.uk/pool/overlay/linux419-headers-4.19.107-1-x86_64.pkg.tar.xz and renamed the extension then installed)
installpkg /boot/packages/linux419-headers-4.19.107-1-x86_64.pkg.txz

 

#Run make menuconfig after installing dependent packages (I have chosen/downloaded these package versions based on what I've found on unRAID release notes and Googling)
installpkg /boot/packages/make-3.82-x86_64-4.txz
installpkg /boot/packages/glibc-2.30-x86_64-1.txz
installpkg /boot/packages/binutils-2.33.1-x86_64-1.txz
installpkg /boot/packages/ncurses-6.2-x86_64-1.txz
installpkg /boot/packages/gcc-4.8.2-x86_64-1.txz
installpkg /boot/packages/gcc-g++-4.8.2-x86_64-1.txz

 

cd /usr/src/linux
find . -type f -iname '*.patch' -print0|xargs -n1 -0 patch -p 1 -i

 

Up to this point ^ everything seems to look good, patches succeeded. Then I run the following:

root@unRAID:/usr/src/linux# make oldconfig
  HOSTCC  scripts/basic/fixdep
/usr/libexec/gcc/x86_64-slackware-linux/4.8.2/cc1: error while loading shared libraries: libmpc.so.2: cannot open shared object file: No such file or directory
make[1]: *** [scripts/basic/fixdep] Error 1
make: *** [scripts_basic] Error 2


If I do this all again, using  gcc-9.2.0-x86_64-4.txz and gcc-g++-9.2.0-x86_64-4.txz: instead of 4.8.2, i get this output:

root@unRAID:/usr/src/linux# make oldconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
In file included from /usr/include/bits/posix1_lim.h:161,
                 from /usr/include/limits.h:183,
                 from /usr/lib64/gcc/x86_64-slackware-linux/9.2.0/include-fixed/limits.h:194,
                 from /usr/lib64/gcc/x86_64-slackware-linux/9.2.0/include-fixed/syslimits.h:7,
                 from /usr/lib64/gcc/x86_64-slackware-linux/9.2.0/include-fixed/limits.h:34,
                 from scripts/kconfig/conf.c:7:
/usr/include/bits/local_lim.h:38:10: fatal error: linux/limits.h: No such file or directory
   38 | #include <linux/limits.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [scripts/kconfig/conf.o] Error 1
make: *** [oldconfig] Error 2
root@unRAID:/usr/src/linux#

If I can figure out what I'm doing wrong here, and manage to run these commands without error, then I'll look into setting the CONFIG_NO_HZ_FULL option.

I am not that good with this stuff, but I really would like to figure out how to build it. If nobody is willing to guide me, then I am hoping someone will take the time to build me a kernel with the CONFIG_NO_HZ_FULL option. 

 

I have looked at the script here (https://gist.github.com/gfjardim/c18d782c3e9aa30837ff), which has many more packages. I am not sure if I'm supposed to install all of these packages (or newer versions?) before trying to use the make command, but it does seem like the current error is related to the version of gcc I've been using.

 

Any help with this from anyone would be very much appreciated.

Edited by beemac
link fix
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.