December 24, 2025Dec 24 I removed one of my GPUs and so I have only one GPU in the motherboard. Then my VM quit working. After some research, this is what I found. I didn't see this information anywhere else on the forum so apologies if it is a repeat.Either the VM would be on but never boot or boot with no video, as I could still hear it. I have an Nvidia card and it was showing Code 43 in device manager. It seems like I could get it to work once after the server reboot, if I was lucky. I have my GPU passed through to the VM. Eventually, I found the Linux framebuffer is touching the GPU before vfio binds it so the GPU no longer resets cleanly. I have not yet tried this but the permanent fix is to add to this to the append line in your syslinux:video=efifb:off video=simpledrm:off video=vesafb:offIn the Unraid webui, go to the main tab and click on your flash drive. At the bottom is your syslinux config. Again, I have not tried this yet myself because I have not found the time to restart my server. There are other posts that detail this is well. You can use the following method if you don't want to reboot or as a troubleshooting step.What worked for me without rebooting was to force the video card to reset. I made this script but you can just use the individual commands. Make sure and change the bus appropriately to match your system:#!/bin/bash GPU="0000:21:00.0" AUDIO="0000:21:00.1" # Always force FLR echo flr > /sys/bus/pci/devices/$GPU/reset_method 2>/dev/null # Unbind GPU + audio echo "$GPU" > /sys/bus/pci/devices/$GPU/driver/unbind echo "$AUDIO" > /sys/bus/pci/devices/$AUDIO/driver/unbind # Perform FLR echo 1 > /sys/bus/pci/devices/$GPU/reset # Rebind to vfio echo "$GPU" > /sys/bus/pci/drivers/vfio-pci/bind echo "$AUDIO" > /sys/bus/pci/drivers/vfio-pci/bind echo "GPU reset (FLR) complete."I'll explain what it does. If you look at the output from lspci -vv -s 21:00.0 (change the bus to match your system), you will see what reset methods your GPU supports. It's a mess but it's in there somewhere under DevCap and DevCtl. Here is what mine looks like:root@Tower:~# lspci -vv -s 21:00.0 21:00.0 VGA compatible controller: NVIDIA Corporation AD104 [GeForce RTX 4070] (rev a1) (prog-if 00 [VGA controller]) Subsystem: Gigabyte Technology Co., Ltd Device 40ee Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 305 IOMMU group: 49 Region 0: Memory at c2000000 (32-bit, non-prefetchable) [size=16M] Region 1: Memory at a400000000 (64-bit, prefetchable) [size=16G] Region 3: Memory at a800000000 (64-bit, prefetchable) [size=32M] Region 5: I/O ports at 3000 [size=128] Expansion ROM at c3000000 [disabled] [size=512K] Capabilities: [60] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee00000 Data: 0000 Capabilities: [78] Express (v2) Legacy Endpoint, IntMsgNum 0 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ TEE-IO- DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend+ LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L1, Exit Latency L1 <4us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- FltModeDis- LnkSta: Speed 16GT/s, Width x16 TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp+ 10BitTagReq+ OBFF Via message, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- AtomicOpsCtl: ReqEn- IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq- 10BitTagReq- OBFF Disabled, EETLPPrefixBlk- LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS- LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+ EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported, FltMode- Capabilities: [b4] Vendor Specific Information: Len=14 <?> Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01 Status: NegoPending- InProgress- Capabilities: [258 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2- ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=255us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us L1SubCtl2: T_PwrOn=10us Capabilities: [128 v1] Power Budgeting <?> Capabilities: [420 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UncorrIntErr- BlockedTLP- AtomicOpBlocked- TLPBlockedErr- PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UncorrIntErr- BlockedTLP- AtomicOpBlocked- TLPBlockedErr- PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- UncorrIntErr+ BlockedTLP- AtomicOpBlocked- TLPBlockedErr- PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked- CESta: RxErr+ BadTLP+ BadDLLP+ Rollover- Timeout+ AdvNonFatalErr- CorrIntErr- HeaderOF- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF+ AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?> Capabilities: [900 v1] Secondary PCI Express LnkCtl3: LnkEquIntrruptEn- PerformEqu- LaneErrStat: LaneErr at lane: 14 Capabilities: [bb0 v1] Physical Resizable BAR BAR 0: current size: 16MB, supported: 16MB BAR 1: current size: 16GB, supported: 64MB 128MB 256MB 512MB 1GB 2GB 4GB 8GB 16GB BAR 3: current size: 32MB, supported: 32MB Capabilities: [c1c v1] Physical Layer 16.0 GT/s Phy16Sta: EquComplete+ EquPhase1+ EquPhase2+ EquPhase3+ LinkEquRequest- Capabilities: [d00 v1] Lane Margining at the Receiver PortCap: Uses Driver+ PortSta: MargReady- MargSoftReady- Capabilities: [e00 v1] Data Link Feature <?> Kernel driver in use: vfio-pci Kernel modules: nouveauMine only supports FLR. If you run cat /sys/bus/pci/devices/$GPU/reset_method (make sure to sub the $GPU variable) it will show you what the current reset method it set to. Mine was set to flr bus, meaning it tries FLR, then falls back to bus (which mine doesn't support). So this script will set the reset method to just FLR, then unbind the GPU, reset it, and then rebind to VFIO without having to reboot. You might want to comment out the "Always force FLR" line first and just try the default method. Your card might support other methods you can try as well. You just need to replace the flr after the echo in the "Always force FLR" command with another method that your card supports. I don't know what those would be but you can look it up if you need to. You don't need to worry about putting an invalid option and breaking things because if it doesn't support it or you put in an invalid option, it just will say it is an invalid option. For me, it didn't work the first time with the default method but it did when forcing FLR. Make sure the VM is off before you try this, then turn it on after and see if it worked. Edited December 24, 2025Dec 24 by bobbintb
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.