Edited for a better understanding
I have exactly the same issue as Encore's, but mine is a 12600K with B760M D4, it's upgraded from 9980HK with M4SMD (B150).
So my issue: I can passtrough a VF to a VM and the GPU is working fine in the VM, but once I shutdown the VM, all the GPU and VFs will disappear (from Tools - System Devices), unless I reboot Unraid.
When the issue is not appear, intel_gpu_top -d sriov layout:
intel-gpu-top: Intel Alderlake_s (Gen12) @ /dev/dri/card0 - 0/ 699 MHz
77% RC6; 0.04/26.65 W; 0 irqs/s
ENGINES BUSY MI_SEMA MI_WAIT
Render/3D 0.10% |▏ | 0% 0%
Blitter 0.00% | | 0% 0%
Video 0.00% | | 0% 0%
VideoEnhance 0.00% | | 0% 0%
Compute 0.00% | | 0% 0%
And dmesg | grep i915 layout:
[ 42.451359] i915 0000:00:02.0: Running in SR-IOV PF mode
[ 42.451628] i915 0000:00:02.0: [drm] VT-d active for gfx access
[ 42.451725] i915 0000:00:02.0: vgaarb: deactivate vga console
[ 42.451757] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[ 42.452346] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[ 42.454439] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[ 42.454532] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[ 42.476010] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/tgl_guc_70.bin version 70.13.1
[ 42.476014] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[ 42.478956] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads!
[ 42.479359] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[ 42.479359] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
[ 42.479733] i915 0000:00:02.0: [drm] GuC RC: enabled
[ 42.480181] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
[ 42.480249] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
[ 42.512448] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
[ 42.513433] i915 0000:00:02.0: 7 VFs could be associated with this PF
[ 42.540781] fbcon: i915drmfb (fb0) is primary device
[ 42.613281] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
[ 44.808474] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[ 44.808522] i915 0000:00:02.1: enabling device (0000 -> 0002)
[ 44.808535] i915 0000:00:02.1: Running in SR-IOV VF mode
[ 44.809295] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.4.1
[ 44.810804] i915 0000:00:02.1: [drm] VT-d active for gfx access
[ 44.810816] i915 0000:00:02.1: [drm] Using Transparent Hugepages
[ 44.811192] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.4.1
[ 44.811814] i915 0000:00:02.1: GuC firmware PRELOADED version 1.4 submission:SR-IOV VF
[ 44.811815] i915 0000:00:02.1: HuC firmware PRELOADED
[ 44.813559] i915 0000:00:02.1: [drm] Protected Xe Path (PXP) protected content support initialized
[ 44.813561] i915 0000:00:02.1: [drm] PMU not supported for this GPU.
[ 44.813602] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.1 on minor 1
[ 44.813738] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[ 44.813741] i915 0000:00:02.1: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[ 44.813766] i915 0000:00:02.2: enabling device (0000 -> 0002)
[ 44.813773] i915 0000:00:02.2: Running in SR-IOV VF mode
[ 44.814013] i915 0000:00:02.2: [drm] GT0: GUC: interface version 0.1.4.1
[ 44.814246] i915 0000:00:02.2: [drm] VT-d active for gfx access
[ 44.814253] i915 0000:00:02.2: [drm] Using Transparent Hugepages
[ 44.814604] i915 0000:00:02.2: [drm] GT0: GUC: interface version 0.1.4.1
[ 44.815104] i915 0000:00:02.2: GuC firmware PRELOADED version 1.4 submission:SR-IOV VF
[ 44.815105] i915 0000:00:02.2: HuC firmware PRELOADED
[ 44.816641] i915 0000:00:02.2: [drm] Protected Xe Path (PXP) protected content support initialized
[ 44.816643] i915 0000:00:02.2: [drm] PMU not supported for this GPU.
[ 44.816672] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.2 on minor 2
[ 44.816755] i915 0000:00:02.0: Enabled 2 VFs
[ 149.702760] i915 0000:00:02.0: VF1 FLR
[ 149.810657] i915 0000:00:02.0: VF1 FLR
Once I shutdown the VM, which has a VF configured, all GPU and VFs will disappear and there are some new lines with dmesg | grep i915:
[ 284.324075] i915 0000:00:02.0: VF1 FLR
[ 286.533005] i915 0000:00:02.1: Running in SR-IOV VF mode
[ 286.533827] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.4.1
[ 286.535312] i915 0000:00:02.1: [drm] VT-d active for gfx access
[ 286.535326] i915 0000:00:02.1: [drm] Using Transparent Hugepages
[ 286.535723] i915 0000:00:02.1: [drm] GT0: GUC: interface version 0.1.4.1
[ 286.536229] i915 0000:00:02.1: GuC firmware PRELOADED version 1.4 submission:SR-IOV VF
[ 286.536231] i915 0000:00:02.1: HuC firmware PRELOADED
[ 286.537712] i915 0000:00:02.1: [drm] Protected Xe Path (PXP) protected content support initialized
[ 286.537715] i915 0000:00:02.1: [drm] PMU not supported for this GPU.
[ 286.537760] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.1 on minor 1
[ 287.586734] i915 0000:00:02.1: [drm] *ERROR* tlb invalidation response timed out for seqno 23
[ 287.626701] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=io+mem
[ 287.626705] i915 0000:00:02.2: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[ 287.696644] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=io+mem:owns=io+mem
[ 288.918021] i915 0000:00:02.0: Disabled 2 VFs
And intel_gpu_top -d sriov layout:
Requested device sriov not found!
My go file:
#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &
Steps I have tried:
1. Remove VFs from VMs.
2. Remove Intel GPU TOP and Intel Graphics SR-IOV then reboot Unraid.
3. Install Intel Graphics SR-IOV and Intel GPU TOP,get VFs enabled then reboot Unraid.
4. Ensure the host GPU is not passthrough and VFs are not binded (no checks before VFs in System Devices), then add a VF to a VM.
5. Start the VM and ensure GPU (UHD770 in my case) is recognized in Device Management (in Windows).
6. Shutdown VM, GPUs are all disappeared in System Devices.
BTW, the Intel-GVT-g plugin has already been removed before I upgrade this NAS, Unraid version is 6.12.10, but in System Devices my GPU displays like:
[8086:4680] 00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
Without something like UHD770 in its name, will this be a symptom of my issue?
Enclosed is the log file, please help to see if there is any troubleshoot I can try, thanks in advance!
diagnostics-20240618-2109.zip