January 16Jan 16 I am trying to figure out why my system is unable to reach higher c-states than C2.A couple of days ago I've installed a new CPU and an M.2 samsung ssd. With the CPU upgrade my system could now utilize the front m.2 slot on my ITX MB with LGA 1200 socket. Previously I had only one ssd in the back of the MB that's been managed by the PCH chip. With only one SSD in the back I reached max C3 (but it was at least 17w in idle). Now since the front m.2 slot is sitting directly in the 20 lanes in front of the CPU, it seems to prevent higher c-states whyever... After a lot of digging, I saw someone with the same board and configuration (but unknown OS), I knew it has to be possible for my system to get it to higher c-states ... at least c6. Unfortunately I can't reach out to him.. Nethertheless, I dug a bit through my system and analyzed my whole configs and logs. My BIOS ASPM and power relevant settings are fine. I see that my NVME's have ASPT=(Autonomous Power State Transition) enabled. ASPM L0-L1.2 is on all devices enabled and should work fine. Nothing disabled there.while checking dmesg, I noticed something unusual.root@NASty:~# dmesg | grep -i nvm [ 3.224540] nvme 0000:02:00.0: platform quirk: setting simple suspend ## <<<---- my newly added Samsung 980 SSD [ 3.224642] nvme nvme1: pci function 0000:02:00.0 [ 3.224651] nvme nvme0: pci function 0000:08:00.0 [ 3.234151] nvme nvme0: 16/0/0 default/read/poll queues [ 3.236697] nvme0n1: p1 [ 3.239673] nvme nvme1: D3 entry latency set to 8 seconds ## <<<---- thinks that it takes ages to come back online [ 3.249674] nvme nvme1: allocated 64 MiB host memory buffer. [ 3.297495] nvme nvme1: 12/0/0 default/read/poll queues [ 3.304596] nvme1n1: p1 [ 93.234922] nvme nvme0: using unchecked data buffer2 cache NVME's ... nvme0 being heavily used and nvme1 barely, but this setting is only for the samsung ssd?I've red mixed stuff about Samsung SSD's being/not the most reliable for power saving configurations.I've tried to add a kernel cmdline to the flash, but that didn't seem to prevent the platform quirk.[ 0.000000] Command line: BOOT_IMAGE=/bzimage initrd=/bzroot nvme_core.default_ps_max_latency_us=0 [ 0.053634] Kernel command line: BOOT_IMAGE=/bzimage initrd=/bzroot nvme_core.default_ps_max_latency_us=0 [ 3.376714] nvme 0000:02:00.0: platform quirk: setting simple suspendAny idea how to avoid the quirk being applied?Google AI (my personal AI server isn't currently working.. :D) said the following:1. What is a "Quirk"? In kernel development, a quirk is a workaround for hardware that does not strictly adhere to industry standards (like ACPI or PCIe specifications). The kernel identifies the device (via its Vendor/Device ID) and essentially says: "I know this hardware crashes with normal power management, so I am applying a special rule." 2. What does "Simple Suspend" mean? Normally, modern NVMe SSDs and PCIe controllers use complex transitions like APST (Autonomous Power State Transition) or D3hot/D3cold to save power. Simple Suspend forces a simplified method of putting the device to sleep. Often, this means the device is not put into the deepest possible sleep mode to ensure it doesn't "disappear" or freeze the system upon waking up. 3. Why does this prevent higher C-states? This is a direct indicator of why your server is not reaching deep C-states (C8–C10): For the CPU (the Package) to enter deep sleep, the entire PCIe chain must signal that it is ready. If the kernel enforces "Simple Suspend" for your NVMe or a controller, the link often remains in a state (e.g., L1 instead of L1.2) that consumes too much power to allow the CPU to transition into C8/C10.
January 16Jan 16 Community Expert What does smartctl -c /dev/nvmeXnX report (replace X's with the appropriate disk numbers)Edit: nvm youre talking about C- states not the reported wattage... my bad Edited January 16Jan 16 by MowMdown
January 16Jan 16 Author I know that it supports PS4 what's the highest with 0,005w. root@NASty:~# nvme get-feature -f 2 -H /dev/nvme1 get-feature:0x02 (Power Management), Current value:0x00000004 Workload Hint (WH): 0 - No Workload Power State (PS): 4 root@NASty:~# nvme get-feature /dev/nvme1 -f 0xc -H get-feature:0x0c (Autonomous Power State Transition), Current value:0x00000001 Autonomous Power State Transition Enable (APSTE): Enabled Auto PST Entries ................. Entry[ 0] root@NASty:~# smartctl -c /dev/nvme1n1 smartctl 7.5 2025-04-30 r5714 [x86_64-linux-6.12.54-Unraid] (local build) Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Firmware Updates (0x16): 3 Slots, no Reset required Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test Optional NVM Commands (0x0055): Comp DS_Mngmt Sav/Sel_Feat Timestmp Log Page Attributes (0x0f): S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Maximum Data Transfer Size: 512 Pages Warning Comp. Temp. Threshold: 82 Celsius Critical Comp. Temp. Threshold: 85 Celsius Namespace 1 Features (0x10): NP_Fields Supported Power States St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat 0 + 5.24W - - 0 0 0 0 0 0 1 + 4.49W - - 1 1 1 1 0 0 2 + 2.19W - - 2 2 2 2 0 6000 3 - 0.0500W - - 3 3 3 3 6000 1500 4 - 0.0050W - - 4 4 4 4 4000 9000 Supported LBA Sizes (NSID 0x1) Id Fmt Data Metadt Rel_Perf 0 + 512 0 But somehow it still seems prevent the CPU from getting into a deeper sleep.root@NASty:~# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:. ]+|ASPM |Disabled;|Enabled;)' 00:01.0 PCI bridge: Intel Corporation Device 4c01 (rev 01) (prog-if 00 [Normal decode]) LnkCap: Port #2, Speed 16GT/s, Width x16, ASPM L1, Exit Latency L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 00:06.0 PCI bridge: Intel Corporation Device 4c09 (rev 01) (prog-if 00 [Normal decode]) LnkCap: Port #5, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 00:1b.0 PCI bridge: Intel Corporation Tiger Lake-H PCIe Root Port #17 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #17, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- 00:1b.4 PCI bridge: Intel Corporation Device 43c4 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #21, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- 00:1c.0 PCI bridge: Intel Corporation Tiger Lake-H PCIe Root Port #1 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- 00:1c.4 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #5, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 00:1c.7 PCI bridge: Intel Corporation Device 43bf (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #8, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 00:1d.0 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #9 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #9, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <16us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 00:1d.4 PCI bridge: Intel Corporation Device 43b4 (rev 11) (prog-if 00 [Normal decode]) LnkCap: Port #13, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk- 01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0]) LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express]) LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 06:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 03) LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 07:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a) LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <8us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+ 08:00.0 Non-Volatile memory controller: Sandisk Corp SanDisk Extreme Pro / WD Black 2018/SN750/PC SN720 NVMe SSD (prog-if 02 [NVM Express]) LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <8us LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+edit:I want to mention: Before I've installed the Samsung NVME I reached C3 with ~17w consumption vs now ~24w. But allegedly the SSD only consumes 0,005w more. Edited January 16Jan 16 by W0nderW0lf
January 17Jan 17 Author I've become a little smarter again.There are 2 "power_states" that decide in which state the NVME is located.D3hot and D3cold, which almost says what the conditions stand for. Hot (is on) and cold(low sleep)My SSD's are kept constant in "D0" which ensures the highest consumption despite supposed PS4 stati. The SSD is thus deliberately kept warm because it is feared that it will not wake up in time.root@NASty:~# cat /sys/class/nvme/nvme1/device/power_state D0 ## You notice in dmesg D3 condition is almost never reached [ 3.239673] nvme nvme1: D3 entry latency set to 8 seconds Edited January 17Jan 17 by W0nderW0lf
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.