fnhanks

After unplugging, still getting the errors. I haven't rebooted since unplugging, so still trying to 100% confirm what the culprit is. Wondering if plugging in the KVM causes an issue that doesn't get cleared up until a reboot, as it's still doing it even after being unplugged. One thing that's interesting, is I think it might be somewhat related to the display going to sleep, as the KVM will say no HDMI output. Haven't tried with a real monitor yet, as this is sitting inside a rack, so somewhat difficult to do. Will try and use one of my longer HDMI cables to test. Just rebooted the server to see if the issue comes back with the KVM unplugged. If it does, then I know it's something else. If it doesn't, then I'll try and plug a real monitor in to see if it comes back to confirm whether it's any display or just this KVM.

Plugged the KVM in and the issues seem to have resumed. Looking in dmesg, I see more of the same error: [ 529.708730] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 529.708785] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 529.709959] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer I've attached the diagnostics that include the KVM. goober-diagnostics-20250823-0724-WITH-KVM.zip

diagnostics-20250822-0831-WITHOUT-KVM.zip The box ran fine for 18 hours without the HDMI cord plugged in. I've attached the diagnostics pulled prior to plugging it back in. The PCIE gen is set to auto in the BIOS and is currently running gen4. I've plugged the HDMI cord back in and will continue to monitor to see when it does it again so I can grab diagnostics at that point. I can't reboot it at the moment to check resizable bar support and above 4g decoding, but will do that later today.

Are there known issues with the open source driver? I upgraded to 7.1.4 today and swapped out my 1080TI for a 5080 and my system is completely unusable. I keep having my CPU (5900x) pegged at 100% with the culprit being Xorg, which usually points to a display/driver issue. The webui, ssh, docker containers, etc. everything halts. I was previously running 6.X and everything was working flawlessly. At this point it seems to either be the opensource driver or 7.1.4, or both, but it's completely hosed my system. Looking at the output of dmesg I see: dmesg logs [ 627.215967] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 627.216010] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 627.216784] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 1021.247207] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 1021.247287] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 1021.253483] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 1490.317300] br-2c59c634a98f: port 1(veth9886930) entered disabled state [ 1490.317767] veth9886930 (unregistering): left allmulticast mode [ 1490.317772] veth9886930 (unregistering): left promiscuous mode [ 1490.317777] br-2c59c634a98f: port 1(veth9886930) entered disabled state [ 1770.432845] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 1770.432891] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 1770.460024] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 2248.983618] br-ef5b47530577: port 1(veth926cb67) entered disabled state [ 2248.983721] veth8d97af8: renamed from eth0 [ 2249.084737] br-ef5b47530577: port 1(veth926cb67) entered disabled state [ 2249.085149] veth926cb67 (unregistering): left allmulticast mode [ 2249.085154] veth926cb67 (unregistering): left promiscuous mode [ 2249.085158] br-ef5b47530577: port 1(veth926cb67) entered disabled state [ 2344.157509] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 2344.157540] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 2344.158139] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 2944.742794] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 2944.742830] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 2944.744048] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 3544.206283] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 3544.206321] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 3544.207614] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 4144.654801] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 4144.654834] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 4144.656173] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 4744.145701] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 4744.145739] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 4744.146981] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 5344.639279] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 5344.639313] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 5344.640567] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 5943.992658] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 5943.992696] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 5943.993961] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer [ 6460.826757] br0: port 2(vnet0) entered disabled state [ 6460.827439] vnet0 (unregistering): left allmulticast mode [ 6460.827443] vnet0 (unregistering): left promiscuous mode [ 6460.827448] br0: port 2(vnet0) entered disabled state [ 6464.808303] br0: port 2(vnet3) entered blocking state [ 6464.808309] br0: port 2(vnet3) entered disabled state [ 6464.808317] vnet3: entered allmulticast mode [ 6464.808441] vnet3: entered promiscuous mode [ 6464.808822] br0: port 2(vnet3) entered blocking state [ 6464.808825] br0: port 2(vnet3) entered forwarding state [ 6469.487830] br0: port 4(vnet2) entered disabled state [ 6469.488622] vnet2 (unregistering): left allmulticast mode [ 6469.488626] vnet2 (unregistering): left promiscuous mode [ 6469.488631] br0: port 4(vnet2) entered disabled state [ 6470.810524] br0: port 4(vnet4) entered blocking state [ 6470.810530] br0: port 4(vnet4) entered disabled state [ 6470.810539] vnet4: entered allmulticast mode [ 6470.810628] vnet4: entered promiscuous mode [ 6470.811169] br0: port 4(vnet4) entered blocking state [ 6470.811175] br0: port 4(vnet4) entered forwarding state [ 6544.504784] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065 [ 6544.504842] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT]) [ 6544.506123] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer I've tried uninstalling nvidia drivers, rebooting, then reinstalling the drivers but it's the same result. The box just hangs every few minutes for about 5 minutes pegged at 100% CPU usage and then it goes back to normal for a few minutes. I had a gl.inet comet KVM plugged into the GPU before. I have unplugged the HDMI cable and that may have fixed it? Will need to continue to do troubleshooting, but it would appear with these drivers that you can only operate everything completely headless otherwise it has these lockups?