OS prevents SSD's to enter d3cold leaving it at d0 despite aspm and apst is enabled resulting in higher power draw

[7.2.3]

I am desperately trying to find out why my System is unable to pass higher states than PC2 or PC3.

I have tested 3 NVME's and all have the same issue in unraid. (I can only have 2 installed

The Firmware for the NVME's are up to date.

Researching similar behaviour online led to almost the same conclusion, that the kernel or other OS specific settings change the desired power_state for the SSD.

I cannot find anything related except for older threads mentioning a bugged samsung firmware and framework's desktop/notebooks or proxmox users...

However.. many people use those SSD's as OS or attached storage, but still being able to set the SSD in other states than "D0"

My system has ASPM and everything related to native power management enabled. It work's but it cannot reach it's full potential due to hardware not willing to get in to full sleep. If I check the Unraid WebUI or CLI for what's the current power draw, it tells me most of the time it's ~0,005w for each SSD, but being in reality 5-8w, because it's not reflecting the real power draw, but the configured power-state that the NVME is currently supporting.

For example:

The first line indicates that my Samsung 980 SSD is set to "simple suspend", which can circumvent lower power states, because it believes that it's not waking up fast enough. (See the line "D3 entry latency set to 8 seconds")

root@NASty:~# dmesg | grep -i "nvme"
[    3.207150] nvme 0000:02:00.0: platform quirk: setting simple suspend
[    3.207243] nvme nvme1: pci function 0000:02:00.0
[    3.207255] nvme nvme0: pci function 0000:08:00.0
[    3.216209] nvme nvme0: 16/0/0 default/read/poll queues
[    3.218728]  nvme0n1: p1
[    3.221735] nvme nvme1: D3 entry latency set to 8 seconds
[    3.231602] nvme nvme1: allocated 64 MiB host memory buffer.
[    3.279170] nvme nvme1: 12/0/0 default/read/poll queues
[    3.286783]  nvme1n1: p1
[   79.541831] nvme nvme0: using unchecked data buffer

Nethertheless it supports virtually lower power_states:

root@NASty:~# nvme get-feature -f 2 -H /dev/nvme1
get-feature:0x02 (Power Management), Current value:0x00000004
	Workload Hint (WH): 0 - No Workload
	Power State   (PS): 4         <------------------------ That's the highest level, see table below

root@NASty:~# smartctl -c /dev/nvme1n1
smartctl 7.5 2025-04-30 r5714 [x86_64-linux-6.12.54-Unraid] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0055):     Comp DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x0f):         S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     82 Celsius
Critical Comp. Temp. Threshold:     85 Celsius
Namespace 1 Features (0x10):        NP_Fields
Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     5.24W       -        -    0  0  0  0        0       0
 1 +     4.49W       -        -    1  1  1  1        0       0
 2 +     2.19W       -        -    2  2  2  2        0    6000
 3 -   0.0500W       -        -    3  3  3  3     6000    1500
 4 -   0.0050W       -        -    4  4  4  4     4000    9000

Ignoring the fact that my cache is not really in use, it still keeps holding it up at a higher power state

root@NASty:~# cat /sys/class/nvme/nvme1/device/power_state
D0

root@NASty:~# nvme get-feature /dev/nvme0 -f 0xc -H
get-feature:0x0c (Autonomous Power State Transition), Current value:0x00000001
	Autonomous Power State Transition Enable (APSTE): Enabled

In the WebUI I see constant powerdraw, without it being used by Containers, VM's or Plugins.

This value is an estimated guess, since without this NVME installed (just a single one that is not directly bound to the CPU) I consume ~6-8w less.

nasty-diagnostics-20260117-2257.zip

My System:

Z590i

11th gen i9-11900

16GB ram

2xSSD

1xASM1166

6x4TB Seagate

ASPM is also enabled on all devices:

08:00.0 Non-Volatile memory controller: Sandisk Corp SanDisk Extreme Pro / WD Black 2018/SN750/PC SN720 NVMe SSD (prog-if 02 [NVM Express])
        LnkCap:    Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <8us
        LnkCtl:    ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express])
        LnkCap:    Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
        LnkCtl:    ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+