1 minute ago, ich777 said:

May I ask to what interval you've set the refresh from GPU Statistics?

However this seems like a firmware issue to me...

2000ms. Don't think I have change it from the plugin being installed.

On 6/6/2025 at 5:14 PM, ich777 said:

Nope, not yet since this requires almost a complete rewrite from the plugin.

On 6/14/2025 at 7:14 PM, anticholium said:

I just concluded because of this comment (see below), but was asking in my last post if I got right. Obviously no...

I'm on Unraid 7.1.3, using the latest Opensource Nvidia Driver (MSI Geforce RTX 5060Ti), latest Steam Headless Container and I'm trying to fire up some more up to date gameing titles and was hoping to streamline them with Sunshine/Moonlight to the more weak laptops of my son. Sadly, despite the quite powerful GPU, I simply can achieve just around 8-10fps even for older games like Counter Strike 2; hardware acceleration is enabled, the container is using the GPU (nvtop, nvidia-smi)... In other use cases like Handbrake video transcoding (NVENC AV1, 10bit, 30fps, CQ30) it works very well, for LLM-based text-toaudiobook-"conversion" it works pretty well too. So my thoghts were, that the current open source driver might only be partyl supporting the quite actual GPU gen? Do you know, when there might be a change in this? I will perform a diagnostics when using the scenario as described and will attach the zip, if that might be helpful. Thank you so much!

Strange observations: when starting/playing a game via Steam Headless container, GPU Statistics shows the GPU and VRAM load etc, which is quite substantial even for very old titles, but nvidia-smi shows 0%! Which one is true? Official Nvidia driver download section states 5060 Ti support only for 570.153.02, not for 570.144, maybe the reason for generic „NVIDIA graphics device“ (and not the specific model) in Unraid?

12 minutes ago, anticholium said:

GPU Statistics shows the GPU and VRAM load etc, which is quite substantial even for very old titles, but nvidia-smi shows 0%! Which one is true?

Sorry but this is the Nvidia Driver support thread and not for GPU Statistics, please report that in the GPU Statistics plugin support thread since this might be an issue with GPU Statistics which pulls the data actually from nvidia-smi

You still haven't posted your Diagnostics or did you?

It would be very interesting to see your syslog alongside with other information about your system and of course which Unraid version that you are running, I really can't help without you providing the requested information.

14 hours ago, ich777 said:

Nope, there is no partly support for GPUs it‘s either working or not.

Possibly Sunshine/Moonlight is the coolprit?

I don't think they are culprits because Steam Headless is recognizing the correct driver version (has to be same as host system), but both, Unraid and of course Steam Headless also, are recognizing the graphics card as "generic" N

10 hours ago, ich777 said:

Sorry but this is the Nvidia Driver support thread and not for GPU Statistics, please report that in the GPU Statistics plugin support thread since this might be an issue with GPU Statistics which pulls the data actually from nvidia-smi

You still haven't posted your Diagnostics or did you?

It would be very interesting to see your syslog alongside with other information about your system and of course which Unraid version that you are running, I really can't help without you providing the requested information.

Yeah, I'm aware, what this support forum is for, sorry for the inconvenience, I just wanted to demonstrate, that appeared very strange to me, that GPU-statistics is showing some GPU load etc produced by X Window System/Sunshine, and at the same time, I thought nvidia-smi is showing 0% load/activty despite some specific processes which are linked to the steam headless container... but I was mistaken, sorry. - The screenshots are 1-2s apart. - Please see my diagnostics attached, first while running oldy FPS Quake I, second while (trying) to run Counter Strike 2 (which should theoretically be around 200-300fps, but is 7-10fps). I'm thankful for any hint/advise. Thx!

diagnostics-20250616-2040.zip

diagnostics-20250616-2059.zip

Edited June 16, 20251 yr by anticholium

1 hour ago, anticholium said:

that GPU-statistics is showing some GPU load etc produced by X Window System/Sunshine, and at the same time, I thought nvidia-smi is showing 0% load/activty despite some specific processes which are linked to the steam headless container...

In both of your screenshots I see utilisation which doesn't match, probably because of the difference where the data was queried but you'll at least see it:

I just triggered a build for the latest New Feature Branch for Unraid 7.1.3 which is Nvidia Driver version 575.57.08, the build should be done in about 5 minutes, however to get the build you must visit the driver page and it should show a new driver version at the Open Source Branch and then you have to click Download & Install again to get the new driver, after the download finished you have to reboot.

34 minutes ago, ich777 said:

In both of your screenshots I see utilisation which doesn't match, probably because of the difference where the data was queried but you'll at least see it:

I just triggered a build for the latest New Feature Branch for Unraid 7.1.3 which is Nvidia Driver version 575.57.08, the build should be done in about 5 minutes, however to get the build you must visit the driver page and it should show a new driver version at the Open Source Branch and then you have to click Download & Install again to get the new driver, after the download finished you have to reboot.

34 minutes ago, ich777 said:

In both of your screenshots I see utilisation which doesn't match, probably because of the difference where the data was queried but you'll at least see it:

I just triggered a build for the latest New Feature Branch for Unraid 7.1.3 which is Nvidia Driver version 575.57.08, the build should be done in about 5 minutes, however to get the build you must visit the driver page and it should show a new driver version at the Open Source Branch and then you have to click Download & Install again to get the new driver, after the download finished you have to reboot.

Awesome!! Thank you so much! Right now, no more "generic" Nvidia Graphics Device but: NVIDIA GeForce RTX 5060 Ti. And: Counter Strike 2 before (as already mentioned, sorry) 7-10fps, right now: 60fps (capped by configs), low GPU load, everything is fine now. Let's see which other changes also come together with this

attempting to update to 7.1.3 and the driver helper had a issue getting this driver

Nvidia Driver v570.169 download failed

6 hours ago, toonamo said:

Nvidia Driver v570.169 download failed

What network equipment are you using? Are you using Unifi gear?

Do you have enough free space on your usb boot device?

It should be safe to reboot since the Driver plugin tries to download the new version on boot if no compatible version is found, the boot time will be a bit longer since the drivers are abour 300MB in size.

On 6/15/2025 at 2:54 PM, tykateetee said:

2000ms. Don't think I have change it from the plugin being installed.

Did you figure out this drop out issue?

I've been running unraid for about 5 years on a Core i7 3770K. Just swapped the hardware yesterday to a AMD 5950X and RTX 3090. I haven't had a GPU in there before. My log is full of "RmInitAdapter" and the GPU drops in and out of GPU statistics and the Nvidia driver plugin page just like you describe.

I'm running the latest nvidia driver and still on Unraid 6.12.15. I haven't tried updating to 7 yet. Seems like several of us are having this problem now.

Edited June 22, 20251 yr by Groto

I had to drop down to using the previous version in order for it to be able to see my Tesla P4. The official previous version. On the latest version of unraid. Or did they fix it after the second recent update?

2 hours ago, Groto said:

Did you figure out this drop out issue?

I've been running unraid for about 5 years on a Core i7 3770K. Just swapped the hardware yesterday to a AMD 5950X and RTX 3090. I haven't had a GPU in there before. My log is full of "RmInitAdapter" and the GPU drops in and out of GPU statistics and the Nvidia driver plugin page just like you describe.

I'm running the latest nvidia driver and still on Unraid 6.12.15. I haven't tried updating to 7 yet.

Did you figure out this drop out issue?

I've been running unraid for about 5 years on a Core i7 3770K. Just swapped the hardware yesterday to a AMD 5950X and RTX 3090. I haven't had a GPU in there before. My log is full of "RmInitAdapter" and the GPU drops in and out of GPU statistics and the Nvidia driver plugin page just like you describe.

I'm running the latest nvidia driver and still on Unraid 6.12.15. I haven't tried updating to 7 yet. Seems like several of us are having this problem now.

You may or may not be in luck. I was just coming back to do my one week later update.

Yes - I have fixed my issue. It has been a week and the error has not appeared once. You're probably not going to like how.

I recreated the Unraid flash device. I went out, bought a new USB stick, downloaded the creation tool, chose 7.1.3 as the version (latest when I did it), and restored my systems hardware to what it was before (read my previous posts). Once completed, I recreated the array and cache in the exact order it was before and chose the option that parity was valid, redownloaded the Nvidia-Driver plugin chosing the latest driver at the time (575.57.08), and rebooted. Once complete, re-setup my docker containers knowing that the bulk of the configuration was in the appdata share and I just had to re-setup the docker aspect. I did not have any VM's on the device.

I am now back to a BIOS that just has DOCP enabled and an almost identical OS install as to what I had before. No tricks anywhere else - just a plug and play configuration. I am unaware as to how the flash got to be the issue, as I secretly was agreeing with @ich777 in thinking it was something firmware related. For my case, it appears that it was not.

Hopefully this helps. I know re-installing the OS is not feasible for most but I knew that I could be back to where the system was within the hour. It totally beats seeing those yellow lines in the syslog ever again.

4 hours ago, tykateetee said:

You may or may not be in luck. I was just coming back to do my one week later update.

Yes - I have fixed my issue. It has been a week and the error has not appeared once. You're probably not going to like how.

I recreated the Unraid flash device. I went out, bought a new USB stick, downloaded the creation tool, chose 7.1.3 as the version (latest when I did it), and restored my systems hardware to what it was before (read my previous posts). Once completed, I recreated the array and cache in the exact order it was before and chose the option that parity was valid, redownloaded the Nvidia-Driver plugin chosing the latest driver at the time (575.57.08), and rebooted. Once complete, re-setup my docker containers knowing that the bulk of the configuration was in the appdata share and I just had to re-setup the docker aspect. I did not have any VM's on the device.

I am now back to a BIOS that just has DOCP enabled and an almost identical OS install as to what I had before. No tricks anywhere else - just a plug and play configuration. I am unaware as to how the flash got to be the issue, as I secretly was agreeing with @ich777 in thinking it was something firmware related. For my case, it appears that it was not.

Hopefully this helps. I know re-installing the OS is not feasible for most but I knew that I could be back to where the system was within the hour. It totally beats seeing those yellow lines in the syslog ever again.

4 hours ago, tykateetee said:

You may or may not be in luck. I was just coming back to do my one week later update.

Yes - I have fixed my issue. It has been a week and the error has not appeared once. You're probably not going to like how.

I recreated the Unraid flash device. I went out, bought a new USB stick, downloaded the creation tool, chose 7.1.3 as the version (latest when I did it), and restored my systems hardware to what it was before (read my previous posts). Once completed, I recreated the array and cache in the exact order it was before and chose the option that parity was valid, redownloaded the Nvidia-Driver plugin chosing the latest driver at the time (575.57.08), and rebooted. Once complete, re-setup my docker containers knowing that the bulk of the configuration was in the appdata share and I just had to re-setup the docker aspect. I did not have any VM's on the device.

I am now back to a BIOS that just has DOCP enabled and an almost identical OS install as to what I had before. No tricks anywhere else - just a plug and play configuration. I am unaware as to how the flash got to be the issue, as I secretly was agreeing with @ich777 in thinking it was something firmware related. For my case, it appears that it was not.

Hopefully this helps. I know re-installing the OS is not feasible for most but I knew that I could be back to where the system was within the hour. It totally beats seeing those yellow lines in the syslog ever again.

Yikes. I don’t trust myself enough to do that big of an overhaul. I’d be worried about losing my data not being able to restore things .

Hello,

Lately I have been getting this:
Jun 24 11:57:43 MediaTank kernel: NVRM: GPU 0000:02:00.0: RmInitAdapter failed! (0x22:0x51:884)
Jun 24 11:57:43 MediaTank kernel: NVRM: GPU 0000:02:00.0: rm_init_adapter failed, device minor number 0

Unraid version 7.1.4

Nvidia Info:

Nvidia Driver Version: 575.51.02

Open Source Kernel Module: No

Installed GPU(s):
0:
NVIDIA GeForce GTX 1650 SUPER
02:00.0

The error happens frequently and when erroring the card doesn't seem responsive. Sometimes restarting the TDARR docker helps release the card and fix the issue.

Any suggestions on what to try? I'm starting to think my card is dying :(

Thanks

UPDATE - I have upgraded the nvidia driver to 575.65 and removed the GPU statistics plugin, then rebooted. So far, no logs indicate issues, but it's too early to tell. Any other troubleshooting steps would be greatly appreciated.

Edited June 24, 20251 yr by musicking

7 hours ago, musicking said:

Hello,

Lately I have been getting this:
Jun 24 11:57:43 MediaTank kernel: NVRM: GPU 0000:02:00.0: RmInitAdapter failed! (0x22:0x51:884)
Jun 24 11:57:43 MediaTank kernel: NVRM: GPU 0000:02:00.0: rm_init_adapter failed, device minor number 0

Unraid version 7.1.4

Nvidia Info:

Nvidia Driver Version: 575.51.02

Open Source Kernel Module: No

Installed GPU(s):
0:
NVIDIA GeForce GTX 1650 SUPER
02:00.0

The error happens frequently and when erroring the card doesn't seem responsive. Sometimes restarting the TDARR docker helps release the card and fix the issue.

Any suggestions on what to try? I'm starting to think my card is dying :(

Thanks

UPDATE - I have upgraded the nvidia driver to 575.65 and removed the GPU statistics plugin, then rebooted. So far, no logs indicate issues, but it's too early to tell. Any other troubleshooting steps would be greatly appreciated.

See my messages on this page and the previous one as they may help you. I had the same issue and it was not related to my GPU dying, but something that was present on my flash device. I had no issues for years and then I had my log filled with these errors too. I think it has something to do with updating Unraid to 7.x.x as from 6.x.x as that was roughly when my issues started. I ended up reinstalling and the issue went away.

1 hour ago, tykateetee said:

See my messages on this page and the previous one as they may help you. I had the same issue and it was not related to my GPU dying, but something that was present on my flash device. I had no issues for years and then I had my log filled with these errors too. I think it has something to do with updating Unraid to 7.x.x as from 6.x.x as that was roughly when my issues started. I ended up reinstalling and the issue went away.

This gives me hope, but I rather not completely restore.

I can rebuild the flash, but I would be restoring my backup and might not help.

Would be nice to know what on the flash is causing it.

UPDATE - Without GPU statistics installed, system is faster and no more error messages. I will wait and see what happens.

Edited June 25, 20251 yr by musicking

I just got a RTX 5060ti 16gb to use some AI docker containers, I installed the Nvidia drivers, but the GPU isn't being detected in the Nvidia drivers settings page. The device is outputting a display, and I see it as an option to to passthrough to VMs (I haven't actually done this, I just see it's available). Running nvidia-smi results in no devices were found.

I have the following driver selected: Open Source Driver: v575.57.08 (rebooted to apply)

I'm seeing these errors repeatedly in the logs:

Jun 25 20:04:46 WHALE kernel: NVRM: _kgspBootGspRm: unexpected WPR2 already up, cannot proceed with booting GSP
Jun 25 20:04:46 WHALE kernel: NVRM: _kgspBootGspRm: (the GPU is likely in a bad state and may need to be reset)
Jun 25 20:04:46 WHALE kernel: NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
Jun 25 20:04:46 WHALE kernel: NVRM: GPU 0000:0a:00.0: RmInitAdapter failed! (0x62:0x40:1941)
Jun 25 20:04:46 WHALE kernel: NVRM: GPU 0000:0a:00.0: rm_init_adapter failed, device minor number 0

Things I've tried:

• Update mobo bios

• Update gpu vbios using the nvidia update tool

• Enable resizable bar

• updated Unraid to 7.1.4

• Verified I don't have the gpu stats plugin installed

whale-diagnostics-20250625-2042.zip

Edited June 26, 20251 yr by jm028

2 hours ago, jm028 said:

Things I've tried:

• Update mobo bios

• Update gpu vbios using the nvidia update tool

• Enable resizable bar

• updated Unraid to 7.1.4

• Verified I don't have the gpu stats plugin installed

what you see is

Jun 25 20:38:20 WHALE kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 239

Jun 25 20:38:20 WHALE kernel: NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:

Jun 25 20:38:20 WHALE kernel: NVRM: BAR1 is 0M @ 0x0 (PCI:0000:0a:00.0)

Jun 25 20:38:20 WHALE kernel: NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:

Jun 25 20:38:20 WHALE kernel: NVRM: BAR2 is 0M @ 0x0 (PCI:0000:0a:00.0)

assuming PSU is fine, may try a different pcie slot

10 hours ago, alturismo said:

what you see is

Jun 25 20:38:20 WHALE kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 239

Jun 25 20:38:20 WHALE kernel: NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:

Jun 25 20:38:20 WHALE kernel: NVRM: BAR1 is 0M @ 0x0 (PCI:0000:0a:00.0)

Jun 25 20:38:20 WHALE kernel: NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:

Jun 25 20:38:20 WHALE kernel: NVRM: BAR2 is 0M @ 0x0 (PCI:0000:0a:00.0)

assuming PSU is fine, may try a different pcie slot

PSU should be fine.

Any idea why this would be happening? The top pcie x16 slot is the appropriate slot for a GPU. I think any other slot will result in a performance hit.

1 hour ago, jm028 said:

Any idea why this would be happening? The top pcie x16 slot is the appropriate slot for a GPU. I think any other slot will result in a performance hit.

1st approach, exclude issue with pcie1 ...

about performance hit, well, you are using your GPU on the host for some dockers, lxc, whatever ... you wont see any real difference anyway, in a Gaming VM it would, but therefore we dont need this driver plugin ;)

so, to exclude some issue with your pcie slot and rbar reservation, give it a try ...

15 hours ago, jm028 said:

Any idea why this would be happening?

To me this seems like a firmware issue, however have you yet tried to limit the PCIe Generation to something like Gen4 or even lower to Gen3 to see if that fixes the issue.

About the performance I'm on the same page as @alturismo because for transcoding it doesn't matter much however for AI related stuff you would need as much bandwidth as possible but I'm not sure if dropping down the PCIe Generation would have such a significant impact (at least not to Gen4) in a real world scenario.

Hey all- I have been troubleshooting an issue were my server has been locking up randomly. I believe now it is because docker and my VM are fighting over the GPU. Anytime I stop my docker instances to backup and upgrade, the GPU seems to be taken by my VM. Does anyone in this thread know of a way to bind the GPU to docker only or a way to deny the GPU from the VM Manager? I have IOMMU on and would like to leave it on incase I want to do something in the future.. but would it be as easy as turning off IOMMU?

Edited June 27, 20251 yr by kbareis

47 minutes ago, kbareis said:

Does anyone in this thread know of a way to bind the GPU to docker only or a way to deny the GPU from the VM Manager?

Diagnostics would help a lot to troubleshoot your issue.

On 6/24/2025 at 9:26 PM, musicking said:

This gives me hope, but I rather not completely restore.

I can rebuild the flash, but I would be restoring my backup and might not help.

Would be nice to know what on the flash is causing it.

UPDATE - Without GPU statistics installed, system is faster and no more error messages. I will wait and see what happens.

Well, errors returned in logs and device vanishes off and on :(

Thought I fixed it.

I will try a new flash drive at some point.

1 hour ago, musicking said:

Well, errors returned in logs and device vanishes off and on :(

Are you sure that this isn't possibly a problem that a container causes?

I had many issues back in the days with Tdarr crashing servers for example.

Sorry if this has been asked/discussed already.

I have an RTX 4070 Super (ASUS Dual) running on my UnRaid Box and about every 6 or so days the GPU seems to disappear from the "Installed GPU(s)" list when viewing the NVidia-Driver plugin. Rebooting seems to fix it for another 6 or so days then I have to rinse and repeat. This has been an issue for the last few versions of UnRaid going back to I think maybe 7.0 or so. I am not running the OpenSource version (for no particular reason). I can switch if that's the actual fix. What the functional difference between the opensource version and the other one? I only use the GPU in docker containers. No VMs use/need the GPU. Am I the only one seeing this? Any suggestions?

Mobo : ASUS Pro-WS-X570-ACE (Workstation Motherboard)
GPU : ASUS Dual NVidia RTX 4070 Super (Single GPU)
UnRaid : 7.1.4
NVidia Driver : 575.64