I'm having issues with multiple crashes of unraid a week and I believe it's related to the Nvidia plugin. Every crash that I have a syslog for starts with this:

Oct 12 21:35:22 Mercury kernel: BUG: kernel NULL pointer dereference, address: 0000000000000402
Oct 12 21:35:22 Mercury kernel: #PF: supervisor read access in kernel mode
Oct 12 21:35:22 Mercury kernel: #PF: error_code(0x0000) - not-present page
Oct 12 21:35:22 Mercury kernel: PGD 0 P4D 0 
Oct 12 21:35:22 Mercury kernel: Oops: 0000 [#1] SMP NOPTI
Oct 12 21:35:22 Mercury kernel: CPU: 9 PID: 97523 Comm: php-fpm7 Tainted: P           O      5.8.12-Unraid #1

I can still login for a variable amount of time when this happens (max 10 minutes ish) and some things still work (changing settings, docker and VMs have always gone down). Inevitably it completely crashes and I cannot even ping or ssh into the machine requiring a hard reset to fix.

 

I believe it's related to the nvidia plugin due to a repeated line that mentions Nvidia:

Oct 12 21:35:22 Mercury kernel: Modules linked in: veth nvidia_uvm(O) xt_CHECKSUM ipt_REJECT ip6table_mangle ip6table_nat iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost vhost_iotlb tap xt_nat iptable_filter xfs md_mod it87 hwmon_vid iptable_nat xt_MASQUERADE nf_nat ip_tables wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 libchacha poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic bonding nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) edac_mce_amd crc32_pclmul rapl aesni_intel drm_kms_helper glue_helper btusb btrtl btbcm crypto_simd btintel ghash_clmulni_intel cryptd drm bluetooth kvm backlight syscopyarea sysfillrect sysimgblt fb_sys_fops wmi_bmof mxm_wmi ecdh_generic agpgart ahci alx crct10dif_pclmul mpt3sas nvme i2c_piix4 ecc crc32c_intel k10temp ccp i2c_core libahci mdio raid_class nvme_core button scsi_transport_sas wmi acpi_cpufreq

I've also seen a couple Reddit threads that have not exact but similar looking logs but none of the suggested fixes worked (I can't seem to find the threads again, otherwise I would have linked them).

 

I've attached the full syslog, any help would be greatly appreciated. Thanks!

 

edit: a little more info, not using the plex script but I do run nvidia-smi -pm 1 on first boot to fix the power state issues

 

edit 2: just had an identical crash on stock build disregard this 

syslog.txt

Edited October 13, 20206 yr by emnclarke
Added more info

2 minutes ago, emnclarke said:

believe it's related to the nvidia plugin

Do you have the problem with stock Unraid build? 

1 hour ago, trurl said:

Do you have the problem with stock Unraid build? 

Just had my fastest crash after a reboot and it was on a stock build. Thanks for reminding me to never make assumptions. Guess I have to keep looking for a possible cause...

On 10/7/2020 at 11:51 AM, CHBMB said:

v6.9.0 beta30 uploaded

Great version, work very well with the latest Plex server available (v 1.20.2.3402-0fec14d92), good job !

 

Works perfectly with the Quadro P2200 : gpu is used for both transcoding and decoding...

 

My Ryzen 2700X also benefits with this upgrade, cooler than with Unraid 6.8.3, and now the idle temp is 10°C less...

 

Here are the stats from my tiny itx low consumption Ryzen : i've downgraded the bios of the Asus B450 Gaming-ITX/ac to version 1.70 to enable Gpu-passthrough effortlessly and set the TDP of the CPU to 35 in bios.

 

(note for other users : this bios is just buggy, to modify values you must press F10 to enter boot selection menu then click on enter Setup, directly enter in bios won't work).

 

Now the power consumption while plex is transcoding is always below 110 Watts (with less than 10 viewers simultaneously), and is about 60 Watts while idle.

 

Edited October 14, 20205 yr by doobyns

Trying to download the latest drivers after 6.9.0 beta 30 update and the download is frozen at 33%. Any suggestions on how to proceed?

 

EDIT: Eventually it downloaded and worked.

 

Edited October 14, 20205 yr by Madman2012

I'm hoping someone could help me get my GTX 1060 to both Encode and Decode with hardware acceleration. 

I'm using some left over pc parts just to get started, but they've been working fine. 

I have an intel 4790K cpu with a Gigabyte Z97X-UD5H-BK motherboard. 

My appdata folder is on an nvme drive under Unassigned Devices. 

My transcoding folder is under appdata/PMSlinux folder on that nvme.

 

I installed the Unraid Nvidia plugin.

Installed the 6.8.3 image. 

Switched to the linux server plex docker. I was using the official plexinc docker. 

Added the --runtime=nvidia to extra parameters in the Advanced section.

Added my guid to the Nvidia Visible Devices parameter.

 

I started a movie and transcoded down in resolution, but gpustats and Plex shows only that Decoding is taking place. My cpu is still doing the Encode. 

This is much better than no hardware acceleration only, but since Plex finally got around to allowing both Decode and Encode to take place, I would like to take advantage of it. 

The GTX1060 does have both decode and encode chips per the matrix. 

 

Hopefully I just missed something silly and it's an easy fix that someone here could spot. 

Any help is much appreciated. 

19 minutes ago, FQs19 said:

I'm hoping someone could help me get my GTX 1060 to both Encode and Decode with hardware acceleration. 

I'm using some left over pc parts just to get started, but they've been working fine. 

I have an intel 4790K cpu with a Gigabyte Z97X-UD5H-BK motherboard. 

My appdata folder is on an nvme drive under Unassigned Devices. 

My transcoding folder is under appdata/PMSlinux folder on that nvme.

 

I installed the Unraid Nvidia plugin.

Installed the 6.8.3 image. 

Switched to the linux server plex docker. I was using the official plexinc docker. 

Added the --runtime=nvidia to extra parameters in the Advanced section.

Added my guid to the Nvidia Visible Devices parameter.

 

I started a movie and transcoded down in resolution, but gpustats and Plex shows only that Decoding is taking place. My cpu is still doing the Encode. 

This is much better than no hardware acceleration only, but since Plex finally got around to allowing both Decode and Encode to take place, I would like to take advantage of it. 

The GTX1060 does have both decode and encode chips per the matrix. 

 

Hopefully I just missed something silly and it's an easy fix that someone here could spot. 

Any help is much appreciated.

 

I think you should ask this kind of question in the topic for the support for the Linuxserver.io plex server : https://forums.unraid.net/topic/40463-support-linuxserverio-plex-media-server/

 

I'm using the legacy Plex Media Server docker image from plexinc, i can't help for your version...

 

try to use plex server from plexinc (latest version) and Nvidia unraid v6.9.0 beta30, i think it will solve all your problems... (i tried differents plex docker images and the less problematic was the plexinc image).

 

last advice, if you have at least than 16Gb ram and you are mainly using your server for plex, you should set your /transcode path to /tmp.

 

I will not talk anymore about that...

2 minutes ago, doobyns said:

 

I think you should ask this kind of question in the topic for the support for the Linuxserver.io plex server : https://forums.unraid.net/topic/40463-support-linuxserverio-plex-media-server/

 

I'm using the legacy Plex Media Server docker image from plexinc, i can't help for your version...

 

try to use plex server from plexinc (latest version) and Nvidia unraid v6.9.0 beta30, i think it will solve all your problems... (i tried differents plex docker images and the less problematic was the plexinc image).

 

last advice, if you have at least than 16Gb ram and you are mainly using your server for plex, you should set your /transcode path to /tmp.

 

I will not talk anymore about that...

I'll try that forum. 

I first tried using the official PlexInc docker. It did the same thing which is why I switched to the linux plex version. 

Thanks for the help. 

Any chance of getting the driver for unRAID stable (6.8.3) updated from 440 to 450?

 

[FFMPEG] - Driver does not support the required nvenc API version. Required: 10.0 Found: 9.1
 

Seems like the latest version of nvenc API requires >450 version of driver:

https://forum.manjaro.org/t/nvidia-nvenc-not-working-with-applications/2202/39

 

edit: VVV thanks, I was hoping to avoid using a beta, but apart from one plugin being incompatible, it went remarkably smoothly.

Edited October 17, 20205 yr by ConnectivIT

8 hours ago, ConnectivIT said:

Any chance of getting the driver for unRAID stable (6.8.3) updated from 440 to 450?

 

[FFMPEG] - Driver does not support the required nvenc API version. Required: 10.0 Found: 9.1
 

Seems like the latest version of nvenc API requires >450 version of driver:

https://forum.manjaro.org/t/nvidia-nvenc-not-working-with-applications/2202/39

My question and yours is answered here :

you can grab the drivers yourself and copy them over to the flash drive or upgrade to the beta. 

16 hours ago, doobyns said:

 

I think you should ask this kind of question in the topic for the support for the Linuxserver.io plex server : https://forums.unraid.net/topic/40463-support-linuxserverio-plex-media-server/

 

I'm using the legacy Plex Media Server docker image from plexinc, i can't help for your version...

 

try to use plex server from plexinc (latest version) and Nvidia unraid v6.9.0 beta30, i think it will solve all your problems... (i tried differents plex docker images and the less problematic was the plexinc image).

 

last advice, if you have at least than 16Gb ram and you are mainly using your server for plex, you should set your /transcode path to /tmp.

 

I will not talk anymore about that...

Thank you for directing me to that forum. They answered my question. 

Incredible plugin, thanks so much for developing this - it's just what I need!

I'm looking for support though, I believe this is the correct place. If not, please point me in the right direction.

 

I have a single discrete graphics card in my tower, a GeForce GTX 1650. It's brand new, purchased specifically for this purpose. In my initial trials, I got that working with the Unraid Nvidia plugin almost immediately - simply installed the version which matched with the version of Unraid I was already running, shut down the server and installed the card, rebooted and it was ready to go (with tweaks to my Plex container of course).

 

It worked for about a day or two and then I ruined it all by forgetting the advice I'd heard from Space Invader One - that booting the server into GUI mode would obviously use the discrete graphics and make it unavailable to dockers and VMs. Okay, no problem I thought - I'll just reboot again and leave it to boot into the CLI as is the default.

 

Unfortunately though, ever since I did this I've had absolutely no luck in getting the card to be detected in the settings page for the plugin. It tells me I have 440.59 installed but that "No devices were detected".

 

If I look at the System Devices option in the Tools menu, I can clearly see that the card is found by the system:

IOMMU group 15:    [10de:1f82] 04:00.0 VGA compatible controller: NVIDIA Corporation TU117 [GeForce GTX 1650] (rev a1)
[10de:10fa] 04:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)

 

Here is my syslinux configuration as I believe it may be relevant:

kernel /bzimage
append pcie_acs_override=downstream initrd=/bzroot

modprobe cpufreq_powersave

echo powersave > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

I have tried uninstalling and reinstalling the plugin - both with and without the graphics card plugged in at the time of install, and I have also tried booting into the GUI mode whilst the graphics card is not present to ensure nothing is still claiming access to that card. I have also tried with and without the pcie_acs_override set in my syslinux config and I have also tried with and without vfio-pci.ids defined too. It was defined when it was originally working but it was pointing to hardware which was no longer present (so I suspect it wasn't really doing anything).

 

This card is listed as supported by that version of the Nvidia drivers, and as mentioned it was working initially, so I have no reason to believe the card is faulty or that it's incompatible.

 

Can anyone offer any insight into this?

Any help greatly appreciated,

 

Thanks!

 

EDIT:

Worth noting as well, I have not tried other versions of Unraid Nvidia - prior to version 6.8.3 of stock Unraid I was facing regular SQLite db corruption which is no good for me and I also am trying to keep well away from beta builds as it's not a test system, so I have only ever tried the Unraid Nvidia version 6.8.3.

Edited October 16, 20205 yr by Ninjadude101
Additional note about only having tried 6.8.3

9 hours ago, Ninjadude101 said:

Incredible plugin, thanks so much for developing this - it's just what I need!

I'm looking for support though, I believe this is the correct place. If not, please point me in the right direction.

 

I have a single discrete graphics card in my tower, a GeForce GTX 1650. It's brand new, purchased specifically for this purpose. In my initial trials, I got that working with the Unraid Nvidia plugin almost immediately - simply installed the version which matched with the version of Unraid I was already running, shut down the server and installed the card, rebooted and it was ready to go (with tweaks to my Plex container of course).

 

It worked for about a day or two and then I ruined it all by forgetting the advice I'd heard from Space Invader One - that booting the server into GUI mode would obviously use the discrete graphics and make it unavailable to dockers and VMs. Okay, no problem I thought - I'll just reboot again and leave it to boot into the CLI as is the default.

 

Unfortunately though, ever since I did this I've had absolutely no luck in getting the card to be detected in the settings page for the plugin. It tells me I have 440.59 installed but that "No devices were detected".

 

If I look at the System Devices option in the Tools menu, I can clearly see that the card is found by the system:

IOMMU group 15:    [10de:1f82] 04:00.0 VGA compatible controller: NVIDIA Corporation TU117 [GeForce GTX 1650] (rev a1)
[10de:10fa] 04:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)

 

Here is my syslinux configuration as I believe it may be relevant:

kernel /bzimage
append pcie_acs_override=downstream initrd=/bzroot

modprobe cpufreq_powersave

echo powersave > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

I have tried uninstalling and reinstalling the plugin - both with and without the graphics card plugged in at the time of install, and I have also tried booting into the GUI mode whilst the graphics card is not present to ensure nothing is still claiming access to that card. I have also tried with and without the pcie_acs_override set in my syslinux config and I have also tried with and without vfio-pci.ids defined too. It was defined when it was originally working but it was pointing to hardware which was no longer present (so I suspect it wasn't really doing anything).

 

This card is listed as supported by that version of the Nvidia drivers, and as mentioned it was working initially, so I have no reason to believe the card is faulty or that it's incompatible.

 

Can anyone offer any insight into this?

Any help greatly appreciated,

 

Thanks!

 

EDIT:

Worth noting as well, I have not tried other versions of Unraid Nvidia - prior to version 6.8.3 of stock Unraid I was facing regular SQLite db corruption which is no good for me and I also am trying to keep well away from beta builds as it's not a test system, so I have only ever tried the Unraid Nvidia version 6.8.3.

Post the output of lspci -k for the card.

5 minutes ago, saarg said:

Post the output of lspci -k for the card.

04:00.0 VGA compatible controller: NVIDIA Corporation TU117 [GeForce GTX 1650] (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd Device 3fcb
        Kernel driver in use: nvidia
        Kernel modules: nvidia_drm, nvidia
04:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd Device 3fcb

 

1 hour ago, Ninjadude101 said:

04:00.0 VGA compatible controller: NVIDIA Corporation TU117 [GeForce GTX 1650] (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd Device 3fcb
        Kernel driver in use: nvidia
        Kernel modules: nvidia_drm, nvidia
04:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd Device 3fcb

 

Looks good. I think there have been a few other 1650 owners having the same issue as you. I don't remember if they managed to solve it somehow or not. Check the previous post and you might find out.

Is the Quadro K5200 supported?

1 hour ago, SLNetworks said:

Is the Quadro K5200 supported?

I don't know. Check the driver page at Nvidia if it's supported.

Started running into this error after (i believe) the last driver update was pushed out: 

 

Oct 21 12:17:32 Unraid1 kernel: NVRM: GPU 0000:0d:00.0: Failed to copy vbios to system memory.

Oct 21 12:17:32 Unraid1 kernel: NVRM: GPU 0000:0d:00.0: RmInitAdapter failed! (0x30:0xffff:794)

Oct 21 12:17:32 Unraid1 kernel: NVRM: GPU 0000:0d:00.0: rm_init_adapter failed, device minor number 1

 

Running 6.9.0-Beta30 with nvidia driver 450.80.02

unraid1-diagnostics-20201021-1232.zip

Additionally I'm running into this error when attempting to revert to the stock build:

 

Downloading: /tmp/mediabuild/bzroot ... failed (Invalid URL / Server error response)

Hello everyone:
Please bare with me as Im going to be very detailed in regards to my issue. I did searched and found nothing related to what Im experiencing. Im fairly new to Unraid but Ive had my build working fine since it was completed this past April. All components have been the same since and nothing has changed software wise. I have to mention that I am using the Nvidia Unraid plugin, followed Spaceinvader's vid for configuration and it worked perfectly to pass the GPU to the Plex docker yet still having video coming from the card while in Unraid GUI. Ive also had transcodes going on while keeping video in GUI. 
This past saturday, I performed a shutdown on my server to move it out of the way for a work I was going to do. As mentioned, nothing was done or changed either hardware or software wise. The PC turned of and was moved. When I booted it, it started normally, MB brand logo came on, then came the UnRaid option selection, which I pick OS with GUI as always; it goes thru commands and such but right at the moment that the "Login" page is supposed to come up, the video dies. The monitor/TV says no input from device.
I did some troubleshooting, turned off the Plex docker since is whats using the GPU, even restared it with the docker not auto-staring but still the same, no video when GUI is supposed to be visible.

Last night I uninstalled the Nvidia Unraid plugin and did a reboot and viola!!!! Video worked as it did before. I got the Login page and everything. So I went ahead and reinstalled the plugin to see if it normalized and sure enough, lost video again. Im not a guru at this or even at reading the logs but I dont know what could be causing this.
Im asking if anyone has any ideas or possible steps to troubleshoot this some more before I do a full wipe and start from scratch. TBH, I wont loose much as i have backups of everything already done.
Can anyone help?

Much appreciated

Specs:
Asus Prime X570-Pro AMD Mb with latest Bios version
Rysen 9
Nvidia Quadro P2000
64 RAM

do-minion-syslog-20201021-2117.zip

Hey everyone,

 

All of a sudden I'm having issues with unRAID Nvidia not recognizing my GPU. I've got a Quadro P400 running in a Dell R510. I had the whole system running perfectly, and then all of a sudden, my plex installation crashed. After having to hard boot my system and a subsequent failure of plex to start, I narrowed the culprit down to the GPU. I disabled the additional arguments in the docker shell that allowed it to use the GPU for hardware accelerated transcoding and then began my troubleshooting.

I moved the GPU into my desktop computer and it seemed to run just fine. I then plugged the GPU into the riser cable I had, and plugged that into my desktop - no go. Assuming I had found the issue, I ordered a new riser cable. It arrived today and so I eagerly shut down the server, plugged the riser cable and GPU in and booted the server. At this time, unRAID Nvidia does not recognize it however when I run the command "lspci | grep -i nvidia" my output is as follows: 

03:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1)
03:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)

 

In the system log, I see this:

Oct 21 18:58:01 PowerEdgeR510 kernel: NVRM: GPU 0000:03:00.0: RmInitAdapter failed! (0x31:0xffff:973)
Oct 21 18:58:01 PowerEdgeR510 kernel: NVRM: GPU 0000:03:00.0: rm_init_adapter failed, device minor number 0

 

Any insight would be fantastic as I'm unsure of what to do next.

 

OK, I know this has been posted a ton. I've read the past 60+ pages in here.

 

I've deleted my network.cfg & network-rules.cfg, totally fresh settings. No Pi-Hole or dns filter. I even put my server on the DMZ for a moment to be sure.

 

Has anyone run into these conditions before?

It's having problems downloading the sha256 files for unraid 6.8.3

 

Now installing Nvidia version 6.8.3

Base URL: https://lsio.ams3.digitaloceanspaces.com/unraid-nvidia/6-8-3/nvidia

TO AVOID CORRUPTION
DO NOT CLOSE THIS WINDOW UNTIL YOU SEE THE DONE PROMPT

Downloading: /tmp/mediabuild/bzimage ... done
Downloading: /tmp/mediabuild/bzroot ... done
Downloading: /tmp/mediabuild/bzroot-gui ... done
Downloading: /tmp/mediabuild/bzmodules ... done
Downloading: /tmp/mediabuild/bzfirmware ... done
Downloading: /tmp/mediabuild/bzimage.sha256 ...

I left it there for tens of minutes and it doesn't pass this step.

On 10/16/2020 at 10:18 PM, saarg said:

Looks good. I think there have been a few other 1650 owners having the same issue as you. I don't remember if they managed to solve it somehow or not. Check the previous post and you might find out.

Okay, thanks for the response but unfortunately that doesn't help.

Searching this thread, I did find the previous posts about issues with the 1650, I have tried the suggestions of moving the card to an alternate PCI slot, and connecting a monitor. Neither of those have helped the plugin to detect the card

 

What irks me is that it worked before, transcoding in Plex and all, and all I did to break it was to boot into the GUI. Once that was done, even after rebooting back into CLI mode, no graphics cards can be found by the plugin despite it showing up in IOMMU groups...

 

Is it possible that booting into the GUI has set up some boot-surviving configuration for the Xorg whereby it claims the card, even if it's not actually booting into the GUI? If so, I am guessing I would need to find the configuration files for that to remove such an entry?

Sorry I do not have a good understanding of the UNRAID booting process, or Xorg in general. I may be way off here, but grasping at straws seems to be my only option here.

 

Does anyone have any new suggestions?

 

Many thanks

 

EDIT:

Poking around on the filesystem, I found this file:

Am I barking up a tree or does this information help at all? Does it confirm my suspicion that Xorg is getting in my way, or is this a relatively normal, expected configuration?

Edited October 23, 20205 yr by Ninjadude101

21 minutes ago, Ninjadude101 said:

Okay, thanks for the response but unfortunately that doesn't help.

Searching this thread, I did find the previous posts about issues with the 1650, I have tried the suggestions of moving the card to an alternate PCI slot, and connecting a monitor. Neither of those have helped the plugin to detect the card

 

What irks me is that it worked before, transcoding in Plex and all, and all I did to break it was to boot into the GUI. Once that was done, even after rebooting back into CLI mode, no graphics cards can be found by the plugin despite it showing up in IOMMU groups...

 

Is it possible that booting into the GUI has set up some boot-surviving configuration for the Xorg whereby it claims the card, even if it's not actually booting into the GUI? If so, I am guessing I would need to find the configuration files for that to remove such an entry?

Sorry I do not have a good understanding of the UNRAID booting process, or Xorg in general. I may be way off here, but grasping at straws seems to be my only option here.

 

Does anyone have any new suggestions?

 

Many thanks

 

EDIT:

Poking around on the filesystem, I found this file:

Am I barking up a tree or does this information help at all? Does it confirm my suspicion that Xorg is getting in my way, or is this a relatively normal, expected configuration?

I never use the gui of unraid with an Nvidia GPU and passing it through to a container, but I don't think that should come in the way. You can easily check that by booting the non-gui mode.

 

Did you also run the command to manually find the GPU UUID that is somewhere in this thread, posted by chbmb?

 

Have you updated your bios or changed any settings lately?