Hey everyone, I'm at a loss..

 

My unraid gui keeps going unresponsive overnight. Everything else works and web end containers are accessible. I've managed to SSH onto it during an unresponsive time to dump out a diagnostics (attached) but I'm lost in the lines of stuff! Hoping someone can help. The only thing I keep seeing in the syslog.txt between midnight and 5am is this:

 

Nov 25 03:47:01 Tower kernel: NVRM: No NVIDIA devices probed.

Nov 25 03:47:01 Tower kernel: nvidia-nvlink: Unregistered Nvlink Core, major device number 240

Nov 25 04:47:01 Tower kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 240

Nov 25 04:47:01 Tower kernel: NVRM: The NVIDIA probe routine was not called for 1 device(s).

Nov 25 04:47:01 Tower kernel: NVRM: This can occur when a driver such as:

Nov 25 04:47:01 Tower kernel: NVRM: nouveau, rivafb, nvidiafb or rivatv

Nov 25 04:47:01 Tower kernel: NVRM: was loaded and obtained ownership of the NVIDIA device(s).

Nov 25 04:47:01 Tower kernel: NVRM: Try unloading the conflicting kernel module (and/or

Nov 25 04:47:01 Tower kernel: NVRM: reconfigure your kernel without the conflicting

Nov 25 04:47:01 Tower kernel: NVRM: driver(s)), then try loading the NVIDIA kernel module

Nov 25 04:47:01 Tower kernel: NVRM: again.

 

Over and over again. Anyone seen this before?

 

Hardware setup is fairly new so there's a chance I've missed a plugin or a driver off something??

 

i7-13700K

64 GB ram

RTX 3070 (but stubbed out and passed through to a VM - all video transcoding is done on the igpu)

 

Running the usual stack of arrs and plex. recently disabled tdarr in a hope it was that. Did have a win 11 VM running 10 cores and the 3070 but I've also disabled that recently to see if it was that too. So right now I'm running just the arrs and plex.

 

Thanks

 

diagnostics.zip

So - I had a GPU persistence script running every hour:
 

#!/bin/bash

nvidia-smi -pm 1

 

which probably wasn't helping as I have no unraid GPU in my machine anymore (only GPU in there is used for the VM and invisible to unraid).

 

Last night I stopped this script and for the first time my system didnt brick overnight.

 

The next few days will be slowly turning everything back on that I've stopped in a hope that script was what was causing me the issues.