encryptedcurse

@knotgerman Since it sounds like you went through the process of tinkering with SCEHUB/SCEWIN, did you manage to get nvram.txt? If so, could you share it? Poked around the BIOS with UEFI Editor and found these somewhat interesting settings: Disable Turbo GT frequency CFG Lock Enable Configurable TDP Power Limit 3 /4 Package Power Limit MSR Lock Energy Efficient P-state Low Power S0 Idle Capability DLRM Setting for Storage

I see what you mean now. intel_idle.c is missing a arl_cstates[] struct and/or accompanying entry in the intel_idle_ids[] struct. 🙂 Really trivial to add...I've noticed Intel is very sloppy with their open source contributions. Interestingly, they recently added the ability to specify your own C-states table. However, I don't think we could just copy the Meteor Lake table 1:1 since the MWAIT instructions are different. 0x0 is trivial, but I don't know how you'd figure out the exit_latency and target_residency for the other two. Maybe something to look into if you're savvy, but I'm personally putting it on the backburner for now. grep . /sys/devices/system/cpu/cpu0/cpuidle/state*/desc /sys/devices/system/cpu/cpu0/cpuidle/state0/desc:CPUIDLE CORE POLL IDLE /sys/devices/system/cpu/cpu0/cpuidle/state1/desc:ACPI FFH MWAIT 0x0 /sys/devices/system/cpu/cpu0/cpuidle/state2/desc:ACPI FFH MWAIT 0x21 /sys/devices/system/cpu/cpu0/cpuidle/state3/desc:ACPI FFH MWAIT 0x60

@knotgerman Right, I understand that the current stable release of powertop (2.15) reports ACPI C-states, but 2.16-rc3 should have Arrow Lake support so there shouldn't be any more reporting issues, in theory. pcm should report correctly as well. Here's what I see with powertop 2.16-rc3... Package(HW) | C2 (pc2) 57.7% | C3 (pc3) 0.0% | C6 (pc6) 0.0% | C7 (pc7) 0.0% | C8 (pc8) 0.0% | C9 (pc9) 0.0% | C10 (pc10) 0.0% |...and pcm 202502-1: Core C-state residencies: C0 (active,non-halted): 0.15 %; C1: 0.32 %; C3: 0.00 %; C6: 57.70 %; C7: 41.83 %; Package C-state residencies: C0: 100.00 %; C2: 0.00 %; C4: 0.00 %; C6: 0.00 %; ┌───────────────────────────────────────────────────────────────────────────────┐ Core C-state distribution│6666666666666666666666666666666666666666666666777777777777777777777777777777777│ └───────────────────────────────────────────────────────────────────────────────┘ ┌────────────────────────────────────────────────────────────────────────────────┐ Package C-state distribution│00000000000000000000000000000000000000000000000000000000000000000000000000000000│ └────────────────────────────────────────────────────────────────────────────────┘I'm not convinced I'm hitting past C0 (or C2, if we're being generous — not sure why powertop doesn't show pc0). FYI, the 8-9 W I quoted is with practically nothing running. I get to 11-12 W regularly. Are you using the Realtek-provided r8125 driver or the generic r8169? Do you plan to edit the BIOS to enable S0ix? I looked into it very briefly, but I'm not sure I'm ready to take the risk yet. lspci output as you requested: root@pve:~# lspci -vv -k | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM |Kernel)' pcilib: sysfs_read_vpd: read failed: No such device 00:01.0 PCI bridge: Intel Corporation Meteor Lake-H PCIe Root Port (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #12, Speed 32GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ Kernel driver in use: pcieport Kernel modules: shpchp 00:06.0 PCI bridge: Intel Corporation Arrow Lake-HX PCIe Root Port #13 (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #13, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- Kernel driver in use: pcieport Kernel modules: shpchp 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] (prog-if 02 [NVM Express]) LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ Kernel driver in use: nvme Kernel modules: nvme 80:1b.0 PCI bridge: Intel Corporation 800 Series PCH PCIe Root Port #17 (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #17, Speed 16GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- Kernel driver in use: pcieport Kernel modules: shpchp 80:1c.0 PCI bridge: Intel Corporation 800 Series PCH PCIe Root Port #1 (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #1, Speed 16GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- Kernel driver in use: pcieport Kernel modules: shpchp 80:1c.2 PCI bridge: Intel Corporation Device 7f3a (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #3, Speed 16GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable+ CommClk+ Kernel driver in use: pcieport Kernel modules: shpchp 80:1c.3 PCI bridge: Intel Corporation Device 7f3b (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #4, Speed 16GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ Kernel driver in use: pcieport Kernel modules: shpchp 80:1d.0 PCI bridge: Intel Corporation 800 Series PCH PCIe Root Port #9 (rev 10) (prog-if 00 [Normal decode]) LnkCap: Port #9, Speed 16GT/s, Width x4, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- Kernel driver in use: pcieport Kernel modules: shpchp 84:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 0c) LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ Kernel driver in use: r8169 Kernel modules: r8169

@knotgerman You seem pretty active in this discussion. Are you certain intel_idle is the issue? Per https://github.com/torvalds/linux/blob/master/drivers/idle/intel_idle.c, Arrow Lake isn't explicitly mentioned, but shouldn't it use the Meteor Lake data? I'm running ASUS PRIME Z890M-PLUS WIFI (latest beta BIOS 3020) + Intel Core Ultra 5 245K (microcode 0x121) + Proxmox VE 9.1.11 (kernel 7.0.2-2-pve). I compiled the latest powertop and I'm seeing zero package C-state residency despite all cores being in C6 or C7 85-100% of the time. pcm confirms the same. I can enter package C2 if I stop all services on my server, but nothing deeper. This also drops me to around 8-9 W vs. 11 W normally. Wonder if I'm just chasing non-existent gains. I have all ASPM and C-state settings enabled and 'maxed out' in BIOS (besides native ASPM, so that ASPM is managed by BIOS) and confirmed with lspci, checked MSR register 0xE2, etc. Gone down multiple rabbit holes and I can't pinpoint any issues. Don't want to enable runtime PM for SATA controller + disks as suggested by powertop since I'm using ZFS. Doubtful it's causing any issues but I could very well be wrong. Maybe my integrated RTL8125 / r8169 driver is at fault (even though lspci reports that ASPM is enabled)...? The PSR stuff doesn't seem relevant for a headless server. I do see Low Power S0 Idle (V5) : 0 in FADT as others have pointed out, but I'm not quite sure if S0 is relevant here either?