[Plugin] Nvidia-Driver

[pinion]

Has anyone ran into an issue where if you reboot the GPU isn't visible to the plugin and if you reboot again it suddenly is?

 

Edit: It appears that it won't let me use an older driver. And when I try after the reboot no GPU. Reboot again and I'm the latest driver again. Diag attached

 

 

unraid-diagnostics-20220729-1634.zip

Edited July 29, 2022 by pinion

[mrwookie]

4 hours ago, ich777 said:

Do you have enabled Above 4G Decoding and Resizabel BAR Support in your BIOS too?

 

I checked and I'm not sure I have any of those options in my BIOS. I searched online and I cannot find any alternate naming for either of those options..... I'm not even sure my GPU supports Resizable Bar.

[ich777]

10 hours ago, pinion said:

Edit: It appears that it won't let me use an older driver. And when I try after the reboot no GPU. Reboot again and I'm the latest driver again. Diag attached

How are you installing a older driver, what did you do exactly?

 

Can you please tell me what the gow uinput plugin is and where do you have it from and for what it used?

Why does it download the Nvidia driver?

Jul 29 16:28:05 unRaid root: Downloading https://us.download.nvidia.com/XFree86/Linux-x86_64/510.73.05/NVIDIA-Linux-x86_64-510.73.05.run

 

 

EDIT: I've now tested it on my server and everything is working as expected:

 

Here with the installed driver version 515.57:

 

 

The Downloading Package box opens after selecting a static version and clicking Update & Download (I've waited for the download to finish and clicked on DONE):

 

 

After the reboot the plugin now shows the correct driver version:

[ich777]

7 hours ago, mrwookie said:

I'm not even sure my GPU supports Resizable Bar.

No it doesn‘t but I would recommend that you‘ve enable it anyways.

 

Do you run the card in a PCIe x16 slot?

 

I‘m really not too sure what‘s going on on your system…

The driver is loaded from what I see in the system devices list but it seems it isn‘t fully loaded from what I see in the syslog on the other hand…

 

Have you turned on Vt-d? If yes try to turn it off.

Also make sure that you are on the latest BIOS version but from what I see it‘s a pretty new version.

[tDames]

16 hours ago, ich777 said:

Is this Diagnostics after it dropped and isn't recognized anymore by your system or is this a Diagnostics file when it is still recognized?

This seems like after a restart because the card is recognized, it would be really helpful if you can post the Diagnostics after it isn't recognized anymore by your system.

 

Please also try to boot with Legacy Boot (CSM) instead of UEFI.

That's odd. That was after the issue but here's another diag of the same issue of the plugin not finding the GPU also I did switch over to CSM now as of this diag. 

ryzennas-diagnostics-20220730-0101 - no card found.zip

Edited July 30, 2022 by tDames

[ich777]

1 hour ago, tDames said:

That's odd.

Sorry, I've completely overlooked this:

Jul 30 00:54:18 RyzenNAS kernel: NVRM: Xid (PCI:0000:08:00): 79, pid='<unknown>', name=<unknown>, GPU has fallen off the bus.
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: GPU has fallen off the bus.
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: RmInitAdapter failed! (0x24:0xffff:1394)
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: rm_init_adapter failed, device minor number 0
Jul 30 00:54:19 RyzenNAS kernel: NVRM: Xid (PCI:0000:08:00): 79, pid='<unknown>', name=<unknown>, GPU has fallen off the bus.
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: GPU has fallen off the bus.
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: RmInitAdapter failed! (0x24:0xffff:1394)
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: rm_init_adapter failed, device minor number 0

 

Have you enabled C-States in the BIOS? If yes, please try to disable them.

Also maybe try to disable PCIe power saving in the BIOS if you have that option (PCIe Auto Supsend or something like that).

Do you have any other power saving features enabled installed on your server?

[pinion]

12 hours ago, ich777 said:

How are you installing a older driver, what did you do exactly?

 

Can you please tell me what the gow uinput plugin is and where do you have it from and for what it used?

Why does it download the Nvidia driver?

Jul 29 16:28:05 unRaid root: Downloading https://us.download.nvidia.com/XFree86/Linux-x86_64/510.73.05/NVIDIA-Linux-x86_64-510.73.05.run

 

 

EDIT: I've now tested it on my server and everything is working as expected:

 

Here with the installed driver version 515.57:

 

 

The Downloading Package box opens after selecting a static version and clicking Update & Download (I've waited for the download to finish and clicked on DONE):

 

 

After the reboot the plugin now shows the correct driver version:

 

Thank you for the response. To install an older version I selected v510.73.05 which is in my available version list similar to your screenshots. Once it completed I closed the window and had the notification I needed to reboot. Upon reboot the GPU did not show up in the settings for the plugin. I rebooted again and this time the latest driver was installed. This entire thing happened a second time. Additionally, I had been running v510.73.05 for awhile so I'm not even sure how the latest got installed.

 

I am now running the version I want after uninstalling the plugin, rebooting, installing the plugin, installing the driver I want, then rebooting again.

 

Games on Whales (GOW) https://games-on-whales.github.io/gow/index.html

It's a project that lets you steam games from retroarch, steam, etc that's containerized using a docker-compose file. Why it needs to download the nvidia driver. I think uniput is needed for controllers? I'm not totally sure.

 

For xorg I have it point to the nvidia stuff that I think you have installed so again I'm not sure why the plugin downloads anything.

      - /usr/lib64/xorg/modules/drivers/nvidia_drv.so:/nvidia/xorg/nvidia_drv.so:ro
      - /usr/lib64/xorg/modules/extensions/libglxserver_nvidia.so:/nvidia/xorg/libglxserver_nvidia.so:ro

I wish I knew if this was an issue before or after I started messing with GOW. Regardless after uninstalling and reinstalling the plugin I'm back on the driver version I wanted. Thanks for your help!

[tDames]

On 7/30/2022 at 2:26 AM, ich777 said:

Sorry, I've completely overlooked this:

Jul 30 00:54:18 RyzenNAS kernel: NVRM: Xid (PCI:0000:08:00): 79, pid='<unknown>', name=<unknown>, GPU has fallen off the bus.
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: GPU has fallen off the bus.
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: RmInitAdapter failed! (0x24:0xffff:1394)
Jul 30 00:54:18 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: rm_init_adapter failed, device minor number 0
Jul 30 00:54:19 RyzenNAS kernel: NVRM: Xid (PCI:0000:08:00): 79, pid='<unknown>', name=<unknown>, GPU has fallen off the bus.
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: GPU has fallen off the bus.
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: RmInitAdapter failed! (0x24:0xffff:1394)
Jul 30 00:54:19 RyzenNAS kernel: NVRM: GPU 0000:08:00.0: rm_init_adapter failed, device minor number 0

 

Have you enabled C-States in the BIOS? If yes, please try to disable them.

Also maybe try to disable PCIe power saving in the BIOS if you have that option (PCIe Auto Supsend or something like that).

Do you have any other power saving features enabled installed on your server?

It doesn't have any power saving properties as far as I could see, but I did run through the bios and disable things that could possibly cause a conflict. Its a change my GPU could be bad also because I noticed it would blank out from bios as I was using it. I'm going to try it in its current configuration right now because Unraid sees the card for now. I'll give it a 24 hours and if it doesn't work, I'll go with one of my older cards. 

[danimal86]

Ruh Row

 

 

Anyone know whats up?

[ich777]

1 hour ago, danimal86 said:

Anyone know whats up?

Diagnostics?

 

On what plugin version are you? Please make sure your plugin is up to date.

 

Tried it now and I have no issue at all:

[danimal86]

14 minutes ago, ich777 said:

Diagnostics?

 

On what plugin version are you? Please make sure your plugin is up to date.

I'm on 2022.07.22 plugin version, it doesn't show any plugin updates.  

Attached are diagnostics

 

 

tower-diagnostics-20220802-1239.zip

[Bearco]

diagnostics-20220802-1240.zip

 

I have the same issue. version 2022.07.22

Edited August 2, 2022 by Bearco

[ich777]

Just now, Bearco said:

diagnostics-20220802-1240.zip 122.44 kB · 0 downloads

?

[ich777]

2 minutes ago, danimal86 said:

I'm on 2022.07.22 plugin version, it doesn't show any plugin updates.  

Please post a screenshot from your plugin page.

Which driver did you choose?

[Bearco]

 

Sorry, I'm not sure why the forum isn't saving my text, but I get the same errors Danimal gets.

Edited August 2, 2022 by Bearco
Text didn't save.

[ich777]

9 minutes ago, Bearco said:

Sorry, I'm not sure why the forum isn't saving my text, but I get the same errors Danimal gets.

I will take a quick look at this, give me a bit please.

Have to downgrade to 6.10.3

[ich777]

@danimal86 & @Bearco have found an bug please execute this from a Unraid terminal and click the Update & Download butten after executing the command:

rm -rf /boot/config/plugins/nvidia-driver/packages/$(uname -r | cut -d '-' -f1)

 

I will fix this ASAP so that this won't happen the next time you try this.

[danimal86]

25 minutes ago, ich777 said:

Please post a screenshot from your plugin page.

Which driver did you choose?

 

[danimal86]

2 minutes ago, ich777 said:

@danimal86 & @Bearco have found an bug please execute this from a Unraid terminal and click the Update & Download butten after executing the command:

rm -rf /boot/config/plugins/nvidia-driver/packages/$(uname -r | cut -d '-' -f1)

 

I will fix this ASAP so that this won't happen the next time you try this.

Will do.  
Ran the command and updating/downloading now.....patiently waiting 🙂

[Bearco]

14 minutes ago, ich777 said:

@danimal86 & @Bearco have found an bug please execute this from a Unraid terminal and click the Update & Download butten after executing the command:

rm -rf /boot/config/plugins/nvidia-driver/packages/$(uname -r | cut -d '-' -f1)

 

I will fix this ASAP so that this won't happen the next time you try this.

Thank you! it downloaded and restarting now.

[danimal86]

2 hours ago, ich777 said:

@danimal86 & @Bearco have found an bug please execute this from a Unraid terminal and click the Update & Download butten after executing the command:

rm -rf /boot/config/plugins/nvidia-driver/packages/$(uname -r | cut -d '-' -f1)

 

I will fix this ASAP so that this won't happen the next time you try this.

Just coming back to say that it seems to be fixed!

Thanks for the ridiculously quick resolution to the problem!

 

Any insight on what happened?

[Schwiing]

Updated the plugin, ran the command but still no joy on the driver download.

4 hours ago, danimal86 said:

Just coming back to say that it seems to be fixed!

Thanks for the ridiculously quick resolution to the problem!

 

Any insight on what happened?

 

Updated the plugin, ran the command and still failed to update the driver.

[ich777]

1 hour ago, Schwiing said:

Updated the plugin, ran the command and still failed to update the driver.

Can you provide a bit more information please (Screenshot from your plugin page, Screenshot from the Error message, Diagnostics).

 

Does the version numbers display? I can imagine a few things that happened on your system but it should work just fine.

[ich777]

7 hours ago, danimal86 said:

Any insight on what happened?

Recently there where made a few necessary changes for 6.11+ and on how the plugins works (download, detection on available versions, paths to the packages) and I‘ve simply made a few trivial mistakes so that this was happening.

 

Sorry for the inconvenience...

[Scootter]

First, thanks for all the work you do to provide this plugin! I suddenly have started having issues with any docker that uses --runtime=nvidia. I first noticed after a system reboot and saw that Plex had not started up. When I tried to start it I immediately got "Execution Error, Bad parameter".

 

Trying to update and run the container and I get this.

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='plex' --net='host' -e TZ="America/Chicago" -e HOST_OS="Unraid" -e HOST_HOSTNAME="NetFlex" -e HOST_CONTAINERNAME="plex" -e 'PLEX_CLAIM'='' -e 'PLEX_UID'='99' -e 'PLEX_GID'='100' -e 'VERSION'='plexpass' -e 'DOCKER_MODS'='ghcr.io/gilbn/theme.park:plex' -e 'TP_THEME'='plex' -e 'NVIDIA_VISIBLE_DEVICES'='GPU-bb6e673d-6535-f173-986d-4702abf9b20e' -e 'TP_SCHEME'='http' -e 'TP_DOMAIN'='192.168.1.223:8981' -e 'TP_COMMUNITY_THEME'='' -e 'NVIDIA_DRIVER_CAPABILITIES'='all' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:32400]/web' -l net.unraid.docker.icon='https://raw.githubusercontent.com/plexinc/pms-docker/master/img/plex-server.png' -v '/dev/shm/':'/transcode':'rw' -v '/mnt/':'/UNRAID':'rw' -v '/mnt/user/appdata/PlexMediaServer':'/config':'rw' --runtime=nvidia 'plexinc/pms-docker:plexpass'

docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: open failed: /proc/sys/kernel/overflowuid: permission denied: unknown.

The command failed.

 

I get the same error on Unmanic and Fileflows and if I remove --runtime=nvidia they work fine but of course no gpu transcoding.

 

I have tried different drivers along with removing the plugin and reinstalling it. Here is what my Nvidia driver page looks like.

 

I did have the nvidia power script that someone else here had an issue with but I have disabled, removed and rebooted before trying everything else. I am not sure what to try next.  Any ideas?

diagnostics-20220803-0156.zip

Edited August 3, 2022 by Scootter