9 minutes ago, Cessquill said:

are you saying my trusty 1050 won't be supported soon? Roughly how long have I got until I need to find a quiet and efficient replacement?

as long there are legacy drivers (<580) compiled with new Unraid Versions.

how long ... in the tech branch ... there is no decent answer to those questions ;) timelines ... ;)

personal guess, i assume it ll take a while ... just wanted to point out that there is already a eol driver scheduled by Nvidia ...

but to be honest, the card swhere released in 2016 ... ;)

24 minutes ago, alturismo said:

personal guess, i assume it ll take a while ... just wanted to point out that there is already a eol driver scheduled by Nvidia ...

but to be honest, the card swhere released in 2016 ... ;)

Thanks for the head's up - I appreciate the card's getting on, but it's perfectly good enough for my transcoding at the moment. This might tip things into a full system upgrade as everything else is starting to age.

Hello - Is there a CLI command to install older drivers if needed? Like legacy GPU ones? Thanks

Just now, Shawn Dlima said:

Hello - Is there a CLI command to install older drivers if needed? Like legacy GPU ones? Thanks

Please give more information which card do you have and what you want to do with it.

Please also see this recommended post:
https://forums.unraid.net/topic/98978-plugin-nvidia-driver/page/87/#findComment-1164123

Just now, ich777 said:

Please give more information which card do you have and what you want to do with it.

I saw that support for older cards will stop soon such as a 1070 FTW, k2200. Just wanted to make sure I had a way to revert to an older driver in the future.

54 minutes ago, Shawn Dlima said:

I saw that support for older cards will stop soon such as a 1070 FTW, k2200. Just wanted to make sure I had a way to revert to an older driver in the future.

sorry, may my Info was not clear, driver 580 + will drop support, there will be legacy drivers here for some time ...

and no, driver is build vs. kernel, so either some day you stay on an "older" Unraid Version or ...

but that will pretty sure take some time from now on ;)

1 hour ago, Shawn Dlima said:

I saw that support for older cards will stop soon such as a 1070 FTW, k2200. Just wanted to make sure I had a way to revert to an older driver in the future.

Please don‘t overthink that, you know that I have the driver version 4xx in the plugin available?

This is basically the old legacy driver which has support for pretty old cards (GTX 7xx series cards, these cards are now about 10 years old) and I will continue to compile that driver version as long as it compiles against newer Kernel versions and I will continue that for the new legacy cards.

Please don‘t panic that hard.

…and no there are no cli tools for that. The plugin just downloads precompiled drivers for your Unraid version that I have to compile in the first place and upload.

You can compile drivers yourself yes bit you have to set up your own development environment for that which is not necessary.

23 hours ago, ich777 said:

Please check that you are on the latest BIOS, switch to PCIe Gen 4 or even Gen 3 in the BIOS if possible, make sure that Resizable BAR Support and Above 4G Decoding is enabled.

Usually that is related to some kind of bus error what you are seeing.

23 hours ago, ich777 said:

Please check that you are on the latest BIOS, switch to PCIe Gen 4 or even Gen 3 in the BIOS if possible, make sure that Resizable BAR Support and Above 4G Decoding is enabled.

Usually that is related to some kind of bus error what you are seeing.

It doesn't look like I have the option to switch the gen in bios. I confirmed those setting are enabled. The card appears to run as expected for a couple hours then Assertion failed: Out of memory [NV_ERR_NO_MEMORY] error.

Jul 9 20:47:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from memdescAllocInternal(pMemDesc) @ memdesc.c:1353

Jul 9 20:47:00 Jarvis kernel: NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from status @ kernel_gsp.c:4615

Jul 9 20:47:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from kgspCreateRadix3(pGpu, pKernelGsp, &pKernelGsp->pGspUCodeRadix3Descriptor, NULL, pGspFw->pImageData, pGspFw->imageSize) @ kernel_gsp.c:4502

Jul 9 20:47:00 Jarvis kernel: NVRM: kgspInitRm_IMPL: Error preparing GSP-RM image

Jul 9 20:47:00 Jarvis kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM

Jul 9 20:47:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x62:0x51:1941)

Jul 9 20:47:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0

Jul 9 20:48:01 Jarvis kernel: NVRM: kbifInitLtr_GB202: LTR is disabled in the hierarchy

Jul 9 20:49:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from memdescAllocInternal(pMemDesc) @ memdesc.c:1353

Jul 9 20:49:00 Jarvis kernel: NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from status @ kernel_gsp.c:4615

Jul 9 20:49:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from kgspCreateRadix3(pGpu, pKernelGsp, &pKernelGsp->pGspUCodeRadix3Descriptor, NULL, pGspFw->pImageData, pGspFw->imageSize) @ kernel_gsp.c:4502

Jul 9 20:49:00 Jarvis kernel: NVRM: kgspInitRm_IMPL: Error preparing GSP-RM image

Jul 9 20:49:00 Jarvis kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM

Jul 9 20:49:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x62:0x51:1941)

Jul 9 20:49:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0

7 hours ago, Jeffrolc said:

It doesn't look like I have the option to switch the gen in bios. I confirmed those setting are enabled. The card appears to run as expected for a couple hours then Assertion failed: Out of memory [NV_ERR_NO_MEMORY] error.

Are you also sure that you are on the latest BIOS version?

Such issues are usually related to Hardware/Firmware incompatibility. I'm not the biggest fan from AMD since they have done some funky stuff in the past with the PCIe bus and that issue could be a outcome of that, don't know if they fixed that already.

23 hours ago, ich777 said:

Please check that you are on the latest BIOS, switch to PCIe Gen 4 or even Gen 3 in the BIOS if possible, make sure that Resizable BAR Support and Above 4G Decoding is enabled.

Usually that is related to some kind of bus error what you are seeing.

23 hours ago, ich777 said:

Please check that you are on the latest BIOS, switch to PCIe Gen 4 or even Gen 3 in the BIOS if possible, make sure that Resizable BAR Support and Above 4G Decoding is enabled.

Usually that is related to some kind of bus error what you are seeing.

It doesn't look like I have the option to switch the gen in bios. I confirmed those setting are enabled. The card appears to run as expected for a couple hours then Assertion failed: Out of memory [NV_ERR_NO_MEMORY] error.

Jul 9 20:47:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from memdescAllocInternal(pMemDesc) @ memdesc.c:1353

Jul 9 20:47:00 Jarvis kernel: NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from status @ kernel_gsp.c:4615

Jul 9 20:47:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from kgspCreateRadix3(pGpu, pKernelGsp, &pKernelGsp->pGspUCodeRadix3Descriptor, NULL, pGspFw->pImageData, pGspFw->imageSize) @ kernel_gsp.c:4502

Jul 9 20:47:00 Jarvis kernel: NVRM: kgspInitRm_IMPL: Error preparing GSP-RM image

Jul 9 20:47:00 Jarvis kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM

Jul 9 20:47:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x62:0x51:1941)

Jul 9 20:47:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0

Jul 9 20:48:01 Jarvis kernel: NVRM: kbifInitLtr_GB202: LTR is disabled in the hierarchy

Jul 9 20:49:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from memdescAllocInternal(pMemDesc) @ memdesc.c:1353

Jul 9 20:49:00 Jarvis kernel: NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from status @ kernel_gsp.c:4615

Jul 9 20:49:00 Jarvis kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from kgspCreateRadix3(pGpu, pKernelGsp, &pKernelGsp->pGspUCodeRadix3Descriptor, NULL, pGspFw->pImageData, pGspFw->imageSize) @ kernel_gsp.c:4502

Jul 9 20:49:00 Jarvis kernel: NVRM: kgspInitRm_IMPL: Error preparing GSP-RM image

Jul 9 20:49:00 Jarvis kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM

Jul 9 20:49:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x62:0x51:1941)

Jul 9 20:49:00 Jarvis kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0

On 7/10/2025 at 4:12 AM, ich777 said:

Are you also sure that you are on the latest BIOS version?

Such issues are usually related to Hardware/Firmware incompatibility. I'm not the biggest fan from AMD since they have done some funky stuff in the past with the PCIe bus and that issue could be a outcome of that, don't know if they fixed that already.

I updated the BIOS a couple of months ago and assumed it was on the latest. I checked again and there was another update. I applied it and so far so good, thanks for your help! I let it run for a couple more days and report back.

Hello @ich777 ,

First of all thank you for your contributions to the community (i have been using a couple of your plugins/docker templates) ^^
Now regarding the issue of @Jeffrolc , i believe i am encountering it as well.
I have an RTX 5060 as well and as i understand it, RTX 5000 are only compatible with open source kernel drivers only ( such as 575.57.08 ) and i see issues regarding the GSP ( which can't be disabled anymore in the open source drivers ) popping in the logs.
My BIOS is up to date , IOMMU and Re-Size BAR enabled and PCI force to GEN 4.
I've tried other drivers and my GPU won't even be detected.
Could it be just a thing related to that version of the drivers and would require for us to wait ?

2 hours ago, micka said:

Could it be just a thing related to that version of the drivers and would require for us to wait ?

Do you have Diagnostics please?

tower-diagnostics-20250711-2206.zip
Sure thing , here you go !
I wanted to tell you i'm on AMD as well, might be the problem as you suggested as well

Edited July 11, 2025Jul 11 by micka

12 hours ago, micka said:

Sure thing , here you go !

may consider removing this from your modprobe

options nvidia NVreg_EnableGpuFirmware=0

when debugging, base setup would always be prefered ...

13 hours ago, micka said:

I wanted to tell you i'm on AMD as well, might be the problem as you suggested as well

Possibly AMD related but I really can't tell since I don't have a 50 series card on hand however it seems that some cards run out of Kernel memory and that's why you are seeing this issue, to all of that why that happens I really can't tell whats happening there.

17 minutes ago, alturismo said:

may consider removing this from your modprobe

options nvidia NVreg_EnableGpuFirmware=0

when debugging, base setup would always be prefered ...

Ah yes, forgot i added this to test without GSP earlier.( same issue without it though )

5 minutes ago, ich777 said:

Possibly AMD related but I really can't tell since I don't have a 50 series card on hand however it seems that some cards run out of Kernel memory and that's why you are seeing this issue, to all of that why that happens I really can't tell whats happening there.

Yeah well, after looking on github's issues it seems like it is related to power state, some users are reporting kinda similar issues.

Looks like there is a patch you can build (https://github.com/NVIDIA/open-gpu-kernel-modules/commit/c7e72135da83ff027755b4a61a3ff09a32fe00c3) but the drawback is you don't have eco mode anymore and you draw more power at idle.
I'll keep looking at github, thanks for the help guys ^^

Edited July 12, 2025Jul 12 by micka

I ended dropping back to the production driver and the GPU is no longer being grabbed by the VM. Seems to be something introduced with the latest driver(s)

On 6/28/2025 at 12:54 PM, ich777 said:

Can you possibly try to not stop the containers that use the Nvidia GPU when running appdata backup, I would also report that in the CA Backup/Restore thread since this not seems like a driver issue to me, a running VM usually don't takes over a GPU.

BTW, did you know that I and @Joly0 have a LXC container for AMP ready?

You can also try to uninstall GPU Statistics since it seems to cause issues for a few users lately and see if that fixes the issue.

On 6/25/2025 at 8:48 PM, jm028 said:

I just got a RTX 5060ti 16gb to use some AI docker containers, I installed the Nvidia drivers, but the GPU isn't being detected in the Nvidia drivers settings page. The device is outputting a display, and I see it as an option to to passthrough to VMs (I haven't actually done this, I just see it's available). Running nvidia-smi results in no devices were found.

I have the following driver selected: Open Source Driver: v575.57.08 (rebooted to apply)

I'm seeing these errors repeatedly in the logs:

Jun 25 20:04:46 WHALE kernel: NVRM: _kgspBootGspRm: unexpected WPR2 already up, cannot proceed with booting GSP
Jun 25 20:04:46 WHALE kernel: NVRM: _kgspBootGspRm: (the GPU is likely in a bad state and may need to be reset)
Jun 25 20:04:46 WHALE kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
Jun 25 20:04:46 WHALE kernel: NVRM: GPU 0000:0a:00.0: RmInitAdapter failed! (0x62:0x40:1941)
Jun 25 20:04:46 WHALE kernel: NVRM: GPU 0000:0a:00.0: rm_init_adapter failed, device minor number 0

Things I've tried:

• Update mobo bios

• Update gpu vbios using the nvidia update tool

• Enable resizable bar

• updated Unraid to 7.1.4

• Verified I don't have the gpu stats plugin installed

whale-diagnostics-20250625-2042.zip

I wanted to share how I was able to resolve this issue:

• From Flash settings in Unraid GUI, Enable Permit UEFI boot mode

• In BIOS, Disable Boot > CSM (Compatibility Support Module)

It is 2025. Nobody should be booting a computer with legacy BIOS. Developers pretty much expect the features of UEFI to be available for them to use in their code.

If someone is using 20 year old hardware, they need to realize that it needs to be kept sealed in a frozen in time state, and expect issues whenever something modern (hardware or software) is introduced to the equation.

Yeah i have this error as well and the option of UEFI boot enabled and CSM disabled are my default options so i don't think it's related.

How many time has it been since you applied this fix without getting any errors ?

So... I really hope someone can help me or atleast give me an answer.
Today I tried to update to Unraid 7.1.4 but that did not go to plan... at all.

I have updated all plugins, apps, and done everything suggested, so I went ahead and updated the system.
Once rebooted, everything that was relying on the Nvidia Plugin (like plex for transcoding) did not start.
Even the Nvidia Plugin was gone.
So I downloaded the plugin again but now the plugin can't find my GPU... "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver"
I have a Dell R720XD with a Quadro P2000 card.
The system have been running flawlessly on Unraid 7.0
Can this have something to do with the system booting on Legacy mode? There is no way to get a Dell Server to boot using UEFI

This is really frustrating.

15 minutes ago, Christian_P said:

So... I really hope someone can help me or atleast give me an answer.
Today I tried to update to Unraid 7.1.4 but that did not go to plan... at all.

I have updated all plugins, apps, and done everything suggested, so I went ahead and updated the system.
Once rebooted, everything that was relying on the Nvidia Plugin (like plex for transcoding) did not start.
Even the Nvidia Plugin was gone.
So I downloaded the plugin again but now the plugin can't find my GPU... "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver"
I have a Dell R720XD with a Quadro P2000 card.
The system have been running flawlessly on Unraid 7.0
Can this have something to do with the system booting on Legacy mode? There is no way to get a Dell Server to boot using UEFI

This is really frustrating.

Hello , i do not see any breaking changes that would explain this , what driver version did you install ?

Edited July 14, 2025Jul 14 by micka

2 minutes ago, micka said:

Hello , i do not see any breaking changes that would explain this , what driver version did you install ?

I tried a few.
v570.86.16
v565.57.01
v470.256.02
and now when rolled back to Unraid 7.0 I have it working on v565.77

When you are on 7.1.4 can you see the gpu within your pci devices ?
You can test it with that command : "lspci | grep -i nvidia"

5 minutes ago, micka said:

When you are on 7.1.4 can you see the gpu within your pci devices ?
You can test it with that command : "lspci | grep -i nvidia"

I went into "Tools" > "System devices" and I could see the P2000 as a PCI device