On 3/11/2025 at 8:40 AM, TheLinuxGuy said:

Cool web interface, can you share how / where to get it?

See my post below

On 3/4/2025 at 12:21 AM, baonguyen84 said:

Can someone point me into the right direction to find what is causing the issue? 

Might be your NIC, i got a similar behavior with the I226-V. Maybe try to active all but your NICs in powertop (described the procedure some posts above) and see if it does the trick.

Edited March 13, 20251 yr by rorian

On 3/4/2025 at 2:35 PM, knotgerman said:

 

Thoughts on Cooler Master Gold 850 v3? (MWE Gold 850 V3)

 

https://www.cybenetics.com/evaluations/psus/2478/

 

This is currently the #2 PSU besting all others at low idle.  I assume Cooler Master didn't game these results or do component swaps.  Is it worth buying RM750x 2024 to compare or just listen to cybernetics results?

 

 

Answering my own question:

 

https://www.tomshardware.com/pc-components/power-supplies/cooler-master-mwe-gold-850-v3-power-supply-review

 

++ Confirmed efficiency at low idle.

+/- Peak is only 40% (340W)

+/- Efficiency drops quickly over 40% and even faster when hot

- Fan noise at max (I'm never going to ever hit that)

- Fan spins up sooner when hot

 

Recommended for up to 350W.  850W version is quieter and more efficient that 750W version.

 

Unknown whether cooler master does component swaps to cheapen PSU over time.  Perhaps someone else can comment on that.

I've just setup a new system based on an Odroid H4+ single board computer. 

 

Running a clean install of Ubuntu Server 24.10 headless with just a single SATA SSD connected and after powertop --auto-tune I get idle power consumption of around 3W.

 

After this initial test, I've disconnected the SSD and instead booted Unraid 7.0.1 from a Transcend JetFlash 720. After the initial setup (without any arrays, VMs or apps installed) and running powertop, the Unraid setup draws around 2,5W more during idle compared to the Ubuntu Server install. 

 

How do I get Unraid to run similarly efficient during idle? Any suggestions?

 

 

 

root@Unraid:~# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #3 (prog-if 00 [Normal decode])
		LnkCap:	Port #3, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
		LnkCap:	Port #4, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #7 (prog-if 00 [Normal decode])
		LnkCap:	Port #7, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCap:	Port #0, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

Does anyone here have an NVIDIA GPU in their system that can reach a C-State higher than C3, or am I searching for unicorns? 

Hi,

So I'm facing a new problem. When I run PowerTop AutoTune, the system crashes after a few hours.

Since I know a lot of people read this thread (I do myself too when researching new parts) I wanted to share my experience with the MSI MPG Z690 FORCE WIFI DDR5 ATX board. TL;DR no C10, C3 max.

I was looking for a motherboard that can support two x8/x8 PCIE x16 slots at full bandwidth PCIE 5.0 with a lot of nvme storage. The MSI MPG Z690 FORCE WIFI first this requirement nicely, the bad news is that it isn't power efficient at all.

I unlocked the hidden BIOS menus, changed many settings and enabled the usual suspects of ASPM and all other settings needed. Nothing will bright this board past C3 Pkg.

This is not a total loss for me since I am going to use it for my 56Gbps Connectx-3 Pro experiment and I also have a 12 disk enclosure that will connect to a broadcom 9500-8i - so this is why I needed dual x8/x8.

On 3/16/2025 at 3:13 PM, TOMillr-old said:

After this initial test, I've disconnected the SSD and instead booted Unraid 7.0.1 from a Transcend JetFlash 720. After the initial setup (without any arrays, VMs or apps installed) and running powertop, the Unraid setup draws around 2,5W more during idle compared to the Ubuntu Server install. 

Any suggestions how I might optimize power consumption in Unraid further to get closer to the Ubuntu Server values?

Yesterday I upgraded to 7.0.1 (stable) and here is power consumption (Odroid H3 with 3x SSD) with running 1 docker container:

 

Edited March 21, 20251 yr by bagican

Hi, i wanted to mention this here, as even though i don't use Unraid, this thread is one of the biggest resources about C-state issues and powertop.

I recently built a homelab server with a B760M DS3H DDR4 mobo, and was unable to reach package C-states higher than pc3.
The solution turned out switching M.2 slots to the non-CPU one.

Note: This MOBO also has the problematic Realtek 2.5 Gb port, for which i switched to the r8125 driver, and i haven't tried switching back to the default (in-kernel) r8169 driver.
I hope this can help someone else avoid some headaches.

On 2/23/2025 at 5:16 PM, rorian said:

Update:

 

It's the somewhat broken Intel I226-V NIC which causes unraid to become not responding anymore aka the faulty device:

 

Runtime PM for PCI Device Intel Corporation Ethernet Controller I226-V 	echo 'auto' > '/sys/bus/pci/devices/0000:01:00.0/power/control';
Runtime PM for PCI Device Intel Corporation Ethernet Controller I226-V 	echo 'auto' > '/sys/bus/pci/devices/0000:02:00.0/power/control';

 

Pkg does go do C6 not higher. ASPM Settings in Bios all set manually to L1 oder L0&L1 Options. Not sure yet if auto does a better job.

My Setup:

2 x 16TB Toshiba MC Drives @ Addon m.2 -> 6x SATA Adapter ASM1166 Controller

2 x 1TB Crucial SATA SSDs @ Board ASM1064 Controller

With everything running i get around 18W in idle. On Drive SpinDown i hovers around 7-8 Watts.

 

I guess i can live with this, but i will see if there is anything left to get it lower. Will update if i find something.

The manual way off adding single devices to the /boot/config/go files i mentioned above should be mentioned in the startup Post by @mgutt

 

Would be cool if anybody has tipps share.

 

On 2/24/2025 at 4:49 PM, white-orb said:

When I was experimenting with Minisforum MS-01 (it also has i226-V) running Ubuntu 24.04 (kernel v6.8) I also faced the same weird behavior: every time I did `powertop --auto-tune` a system hanged entirely. Initially, I blamed a Chinese hardware, but later I stumbled across a powertop's bug-tracker with a report on the i226-V bug, (also Odroid).

 

It seems it was fixed in later kernel versions as I didn't experience it on Fedora 41 (kernel 6.11) and Asus B760-I STRIX Gaming WiFi which also has i226-V. You should try some other system with a latest kernel.

 

Hey guys, i also had the issue back than in this thread what with an Intel I226T1BLK card system freezed at some point with older unraid 6 & v7.

Now that Unraid OS 7.1.0-beta.1 (with kernel 6.12.19-Unraid x86_64) is released, can anyone of you retest if the issue is gone and PM now working with the intel cards?

Ps.: In the mean time, is there any other 2,5 GBE card (except realtek 8125B chipset and intel I225 or I226) with low power consumption and where we can reach higher c-states?

thx

On 2/24/2025 at 4:49 PM, white-orb said:

When I was experimenting with Minisforum MS-01 (it also has i226-V) running Ubuntu 24.04 (kernel v6.8) I also faced the same weird behavior: every time I did `powertop --auto-tune` a system hanged entirely. (...)

 

It seems it was fixed in later kernel versions (...)

I can second this. Kernel versions 6.11 and above seems to have introduced a fix for this issues. Running powertop --auto-tune no longer causes the system to hang.

I got new hardware, a Lenovo thinkcentre M70Q gen3 with a 12500T. I enabled all c states in bios, disabled hyper threading, removed the wifi card, disabled audio. did the powertop command. but it refuses to go in a c state lower then c2

 

I changed over all other hardware, from my previous build and that worked all fine...

oke started eliminating some options, first went and installed ubuntu server, then I reached C10 and 1,7watts! then I studied the behavior of the different io and I found out that some usb ports have weird behavior and cause high usage en preventing the system to reach lower c-states and identified a usb port that doesn't seem to have that effect. 

 

I prepared a fresh usb stick with unraid 7.1 beta and now it idles at 3-3,5watt spending most of the time in C10. Bit more then ubuntu bit still very acceptable. I then tried a back-up of my "production" unraid server that is running 7.0.1 and it was idling at 10watts not going lower then C7. I think ill just transfer the key set-up unraid from scratch.

9 hours ago, Jess3 said:

I found out that some usb ports have weird behavior and cause high usage en preventing the system to reach lower c-states and identified a usb port that doesn't seem to have that effect. 

 

Interesting findings. I've also struggled with getting idle power consumption in Unraid closer to Ubuntu or Debian. 

 

I've spend the day testing different OS installs and how they impact c-states and idle power consumption compared to Unraid:

 

Unraid 7.1 beta

c10: no

idle: 4.5 W

 

Debian 12 

c10: yes

idle: 2,9 W

 

TrueNAS Scale 25.04 beta

c10: yes

idle: 3,4 W

 

Ubuntu Desktop 22.04

c10: yes

idle: 2,9 W

 

Ubuntu Server 22.04

c10: no

idle: 3,1 W

 

Ubuntu Server 24.10

c10: no

idle: 3,1 W

 

Ubuntu Desktop 24.10

c10: no

idle: 3,4 W

 

 

So Unraid basically is worst when it comes to idle floor in my tests. Eventhough that setup is missing the SATA SSD the other OS were running from and can power down the PCI-to-SATA chip the Odroid H4+ is using.

 

What's Unraid running underneath? Ubuntu? I find it curious, that newer releases of Ubuntu fail to reach c10 states on this system.

 

 

HW configuration Unraid 7.1 beta test
 

Quote

 

Unraid 7.1 beta 1.                                     (Leike 12V / 60W /5A)
┌───────────────────────────────────────────────────────────────────────────┐
│                              Odroid H4+                                   │
│                                                                           │
│  Intel N97                                                                │
│  1x Crucial 16GB DDR4                                                     │
│                                                                           │
│                           1Gbit/s                                         │
│     SATA SATA SATA SATA   LAN   LAN#2              USB2  USB2  USB3  USB3 │
│M.2   #1   #2   #3   #4    #1   unused               #1    #2    #1    #2  │
└─┬────┬────┬─────┬───┬──────┬─────────────────────────┬─────┬─────┬─────┬──┘
  │    │    │     │   │      │                         │     │     │     │
  │    │    │     │   │      └──► Fritzbox 7590        │   unused  │     │
  │    │    │     │   │                                │         unused  │
  │    │    │     │   └► unused                        │               unused
  │    │    │     │                                    │
  │    │    │     └────► unused                        │   ┌────────────────────────┐ 
  │    │    │                                          └──►│ Transcent JetFlash 700 │  
  │    │    │                                              └────────────────────────┘   
  │    │    │                                               
  │    │    └──────────► unused      
  │    │                  
  │    └───────────────► unused                        
  │ 
  └────────────────────► unused            
 

 

 

Quote

 

____________________________________________________________________
			P o w e r T O P

____________________________________________________________________
 *  *  *   System Information   *  *  *

PowerTOP Version;2.15 ran at Wed Mar 26 14:13:57 2025

Kernel Version;Linux version 6.12.19-Unraid
System Name;HARDKERNELODROID-H41.0
CPU Information;4 Intel(R) N97
OS Information;Unraid OS 7.1 x86_64

Target: 1 units/s;System:  194.6 wakeup/s;CPU:  2.2% usage;GPU: 0 ops/s;GFX: 0 wakeups/s;VFS: 0 ops/s;

____________________________________________________________________
 *  *  *   Top 10 Power Consumers   *  *  *

Usage;Events/s;Category;Description
  0.1%;139.8;Timer;tick_nohz_handler
  0.0%;  9.5;kWork;bond_mii_monitor
  0.0%;  7.0;Process;[PID 17] [rcu_preempt]
  0.0%;  5.3;Timer;inactive_task_timer
  0.0%;  3.9;Interrupt;[7] sched(softirq)
  0.0%;  3.8;kWork;usb_giveback_urb_bh
  0.3%;  1.1;Process;[PID 3124] /usr/libexec/unraid/emhttpd 
  0.0%;  1.5;kWork;delayed_vfree_work
  0.0%;  1.4;kWork;srcu_invoke_callbacks
  0.0%;  1.2;Process;[PID 2424] /usr/libexec/unraid/emhttpd 

____________________________________________________________________
 *  *  *   Processor Idle State Report   *  *  *

Package;0
C2 (pc2);  0.9%
C3 (pc3);  0.0%
C6 (pc6);  0.2%
C7 (pc7);  0.0%
C8 (pc8); 91.8%
C9 (pc9);  0.0%
C10 (pc10);  0.0%

 

 

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #3 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #7 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

 

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave

cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference
balance_performance

 

HW configuration Ubuntu / Debian / TrueNAS test

 

Quote

 

Debian 12                                             (Leike 12V / 60W /5A)
┌───────────────────────────────────────────────────────────────────────────┐
│                              Odroid H4+                                   │
│                                                                           │
│  Intel N97                                                                │
│  1x Crucial 16GB DDR4                                                     │
│                                                                           │
│                           1Gbit/s                                         │
│     SATA SATA SATA SATA   LAN   LAN#2              USB2  USB2  USB3  USB3 │
│M.2   #1   #2   #3   #4    #1   unused               #1    #2    #1    #2  │
└─┬────┬────┬─────┬───┬──────┬─────────────────────────┬─────┬─────┬─────┬──┘
  │    │    │     │   │      │                       unused  │     │     │
  │    │    │     │   │      └──► Fritzbox 7590            unused  │     │
  │    │    │     │   │                                          unused  │
  │    │    │     │   └► unused                                        unused
  │    │    │     │
  │    │    │     └────► unused
  │    │    │     
  │    │    └──────────► unused      
  │    │
  │    │        (SSD)   ┌─────────────────────┐    
  │    └───────────────►│ WD Red SA400 500 GB │   
  │                     └─────────────────────┘    
  │ 
  └────────────────────► unused            
 

 

 

 

Quote

 

____________________________________________________________________
			P o w e r T O P

____________________________________________________________________
 *  *  *   System Information   *  *  *

PowerTOP Version;2.14 ran at Wed Mar 26 18:45:48 2025

Kernel Version;Linux version 6.1.0-32-amd64
System Name;HARDKERNELODROID-H41.0
CPU Information;4 Intel(R) N97
OS Information;Debian GNU/Linux 12 (bookworm)

Target: 1 units/s;System:  28.6 wakeup/s;CPU:  0.2% usage;GPU: 0 ops/s;GFX: 0 wakeups/s;VFS: 0 ops/s;

____________________________________________________________________
 *  *  *   Top 10 Power Consumers   *  *  *

Usage;Events/s;Category;Description
  0,0%;  9,2;Timer;tick_sched_timer
  0,0%;  4,8;kWork;fb_flashcursor
  0,0%;  3,8;Interrupt;[3] net_rx(softirq)
  0,0%;  1,9;Process;[PID 44] [kcompactd0]
  0,0%;  1,5;kWork;psi_avgs_work
  0,0%;  1,0;Process;[PID 15] [rcu_preempt]
  0,0%;  1,0;Timer;watchdog_timer_fn
  0,1%;  0,6;Process;[PID 920] powertop -C 
  0,0%;  0,6;Interrupt;[4] block(softirq)
  0,0%;  0,5;kWork;igc_watchdog_task

____________________________________________________________________
 *  *  *   Processor Idle State Report   *  *  *

Package;0
Powered On;  0,0%
C1E;  0,1%
RC6;  0,0%
RC6pp;  0,0%
C8;  0,0%
C10; 99,7%
;
;

 

 

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+

 

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave

cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference
balance_performance

 

Edited March 26, 20251 yr by TOMillr-old

38 minutes ago, TOMillr-old said:

What's Unraid running underneath? Ubuntu?

if i'm not wrong, it should be Slackware (https://en.wikipedia.org/wiki/Slackware) - at least for Unraid 6.x

 

• Quote

  

   

Edited March 26, 20251 yr by bagican

13 hours ago, bagican said:

if i'm not wrong, it should be Slackware (https://en.wikipedia.org/wiki/Slackware)

Ah, thanks. I first through was, Unraid might use Ubuntu underneath and that might explain why I'm not getting the system down to c10 states at all.

Does anyone have any suggestions how I might get Unraid closer to the Debian 12 or TrueNAS idle values? 

 

EDIT:

Hardkernel used the following script to establish the idle baseline of around 2,7 W for the Odroid H4+

 

#!/bin/bash

echo '0' > /proc/sys/kernel/nmi_watchdog

dir='/sys/bus/pci/devices/'
# dirarr=`ls -1 ${dir}`
bus=`lspci | grep -v Ethernet | awk -F' ' '{print $1}'`
for slot in ${bus}
do
	# echo ${dir}0000:${slot}/power/control
	echo 'auto' > ${dir}0000:${slot}/power/control
done

slotSATA=`lspci | grep ASMedia | awk -F' ' '{print $1}'`
 
echo 'auto' > ${dir}0000:${slotSATA}/ata1/power/control
echo 'auto' > ${dir}0000:${slotSATA}/ata2/power/control
echo 'auto' > ${dir}0000:${slotSATA}/ata3/power/control
echo 'auto' > ${dir}0000:${slotSATA}/ata4/power/control

dirSCSI='/sys/class/scsi_host/'
dirSCSIarr=`ls -1 ${dirSCSI}`

for slotSCSI in ${dirSCSIarr}
do
	if [ -e ${dirSCSI}${slotSCSI}/link_power_management_policy ]; then
		# echo ${dirSCSI}${slotSCSI}/link_power_management_policy
		echo 'med_power_with_dipm' > ${dirSCSI}${slotSCSI}/link_power_management_policy
	fi
done

dirMMC='/sys/block/'
dirMMCarr=`ls -1 ${dirMMC}`

for slotMMC in ${dirMMCarr}
do
	if [ -e ${dirMMC}${slotMMC}/power/control ]; then
		# echo ${dirMMC}${slotMMC}/power/control
		echo 'auto' > ${dirMMC}${slotMMC}/power/control
	fi
done

echo "powersupersave" > /sys/module/pcie_aspm/parameters/policy

 

 

Edited March 27, 20251 yr by TOMillr-old

For those who want to choose power efficient SSD, chceck this:

https://www.techpowerup.com/review/samsung-9100-pro-2-tb/7.html

There are measurements with ASPM enabled

 

I just extracted text from the images (on the macOS it's very easy - just open the image and select all text and copy - Cmd+C)

 

Power Draw: Idle Mobile / PCle L1 ASPM Enabled:

 

' Samsung 980 1 TB            ': 0.023 W
' Samsung 980 Pro 1 TB        ': 0.032 W
' Lexar NM710 1 TB            ': 0.033 W
' MSI Spatium M450 1 TB       ': 0.041 W
' Lexar NM790 2 TB            ': 0.049 W
' Seagate FireCuda 510 1 TB   ': 0.052 W
' Samsung 970 EVO Plus 1 TB   ': 0.071 W
' Kioxia Exceria Plus G4 2 TB ': 0.074 W
' Kingston KC3000 2 TB        ': 0.077 W
' Corsair MP400 2 TB          ': 0.079 W
' Corsair MP600 Elite 2 TB    ': 0.086 W
' Crucial T500 2 TB           ': 0.087 W
' Kioxia Exceria Plus G3 2 TB ': 0.087 W
' Crucial PS Plus 1 TB        ': 0.091 W
' Kingston NV2 1 TB           ': 0.094 W
' Neo Forza NFP495 2 TB       ': 0.099 W
' Seagate Barracuda 120 1 TB  ': 0.103 W
' Kingston NV3 2 TB           ': 0.111 W
' Crucial PS 1 TB             ': 0.131 W
' Corsair MP700 Pro 2 TB      ': 0.133 W
' Samsung 9100 Pro 2 TB       ': 0.201 W
' Samsung 870 QVO 1 TB        ': 0.209 W
' Samsung 860 EVO 1 TB        ': 0.210 W
' XPG Atom 50 2 TB            ': 0.246 W
' Solidigm P44 Pro 1 TB       ': 0.256 W
' Hynix Gold P31 2 TB         ': 0.350 W
' Crucial P1 1 TB             ': 0.357 W
' ADATA SX8200 Pro 1 TB       ': 0.395 W
' Crucial MX500 1 TB          ': 0.464 W
' Samsung 990 Pro 2 TB        ': 0.534 W
' Crucial BX500 1 TB          ': 0.778 W
' WD Blue SN570 1 TB          ': 0.929 W
' WD Blue SN580 1 TB          ': 1.015 W
' WD Black SN770 1 TB         ': 1.036 W
' WD Black SN7100 2 TB        ': 1.058 W
' XPG Gammix S70 Blade 2 TB   ': 1.282 W
' Team Group G50 2 TB         ': 1.490 W
' WD Black SN850 1 TB         ': 1.507 W

 

Power Draw: Idle Desktop / PCle L1 ASPM Disabled:

• Quote

  Power Draw: Idle
  Desktop / PCle L1 ASPM Disabled
  
  ' Seagate Barracuda 120 1 TB  ': 0.103 W
  ' Samsung 870 QVO 1 TB        ': 0.209 W
  ' Samsung 860 EVO 1 TB        ': 0.211 W
  ' Hynix Gold P31 2 TB         ': 0.350 W
  ' Seagate FireCuda 510 1 TB   ': 0.463 W
  ' Crucial MX500 1 TB          ': 0.464 W
  ' Corsair MP400 2 TB          ': 0.469 W
  ' Samsung 980 1 TB            ': 0.510 W
  ' Kingston NV3 2 TB           ': 0.576 W
  ' Samsung 970 EVO Plus 1 TB   ': 0.602 W
  ' Crucial P1 1 TB             ': 0.680 W
  ' ADATA SX8200 Pro 1 TB       ': 0.721 W
  ' Samsung 980 Pro 1 TB        ': 0.740 W
  ' Kingston KC3000 2 TB        ': 0.776 W
  ' Solidigm P44 Pro 1 TB       ': 0.776 W
  ' Lexar NM710 1 TB            ': 0.778 W
  ' Crucial BX500 1 TB          ': 0.779 W
  ' XPG Atom 50 2 TB            ': 0.786 W
  ' Crucial T500 2 TB           ': 0.790 W
  ' Lexar NM790 2 TB            ': 0.800 W
  ' Samsung 990 Pro 2 TB        ': 0.865 W
  ' MSI Spatium M450 1 TB       ': 0.893 W
  ' Crucial P5 1 TB             ': 0.911 W
  ' Neo Forza NFP495 2 TB       ': 0.920 W
  ' WD Blue SN570 1 TB          ': 0.929 W
  ' WD Blue SN580 1 TB          ': 1.015 W
  ' WD Black SN770 1 TB         ': 1.036 W
  ' WD Black SN7100 2 TB        ': 1.059 W
  ' Corsair MP600 Elite 2 TB    ': 1.235 W
  ' Crucial P5 Plus 1 TB        ': 1.302 W
  ' Kioxia Exceria Plus G3 2 TB ': 1.302 W
  ' Kingston NV2 1 TB           ': 1.377 W
  ' Team Group G50 2 TB         ': 1.490 W
  ' WD Black SN850 1 TB         ': 1.508 W
  ' Samsung 9100 Pro 2 TB       ': 1.558 W
  ' XPG Gammix S70 Blade 2 TB   ': 1.927 W
  ' Corsair MP700 Pro 2 TB      ': 2.151 W
  ' Kioxia Exceria Plus G4 2 TB ': 2.183 W
  
  TECHPOWERUP
  Lower is Better

   

  - source: https://www.techpowerup.com/review/samsung-9100-pro-2-tb/7.html

 

It's a shame that there are no WD Red drives there.

 

Did you notice?

• Samsung 980 Pro 1 TB: 0.032 W vs WD Black SN850 1 TB: 1.507 W - 47x difference.
• SATA disk like Crucial MX500 (I own 2pcs) consumes 0.464 W in both cases with ASPM enabled and disabled.
• other SATA disks from Samsung consume ~2x less power consumption comparing to Crucial, ~0.21W

 

My favorite one is Samsung 980 Pro. ❤️

Edited March 27, 20251 yr by bagican

On 10/21/2020 at 1:05 PM, mgutt said:

Powertop [Support auf Deutsch]

 

Powertop is an Intel tool (yes, works for AMD, too) to check power consumption states of sata, pcie, usb, etc devices. You can download powertop as follows, then restart your server and it will be automatically installed:

mkdir /boot/extra
cd /boot/extra
wget https://github.com/mgutt/unraid-packages/raw/main/6.11.0/powertop-2.15-x86_64-1.txz

 

Then open your terminal and execute:

powertop

 

Now press "TAB" until you reach "Idle Stats" and check your CPU Pkg (HW). Depending of the brand and model of your CPU you can reach different C-States. If your system isn't optimized, you will see this:

 

If you press "TAB" you will see additional information about standby states of different hardware components of your system.

 

In the tab "Tunables" you will see multiple devices which have the state "Bad", which means powertop found devices which do not allow standby/sleep states:

 

Now you can execute the following command to optimize this:

 

WARNING

Many SATA controllers (JMB xxx or ASM xxxx) do not support link power management and your disks loose connection after reaching spindown! Read further before executing --auto-tune!

 

 

powertop --auto-tune &>/dev/null

 

By that all compents will return "Good":

 

Now, your server should reach better C-States.

 

Package C-States

This table shows the dependency between CPU Core and CPU Package C-States:

 

Several things are important to reach Package (Pkg) C-States C3, C7, C9 or even C10:

 

Accurate power measurement

- do some research how accurate your power meter is (UPS integrated power meters are usually bad)

- close your Unraid WebGUI and wait 1 minute! (this closes many background jobs, which cost your energy, use your PC's terminal and "ssh root@tower" to monitor your C-States with powertop)

- don't use Unraid in GUI mode (you should avoid this mode, if you want low power consumption)

 

Powertop shows only C1, C2 and C3 Pgk-States

This happens if powertop does not recognize your CPU (happens for most AMD CPUs and recent Intel CPUs). Your CPU could still reach C9 or even C10, but powertop is not able to obtain this and shows only C3.

 

BIOS

- only Gigabyte 8th or 9th Intel: enable CEC2019

- enable ASPM (Active-State Power Management) in the PCI settings

- enable all C-States like C7/C9/C10 in the CPU settings ("auto" usually means "disabled", so select "enable")

- disable Audio

- disable RGB LED controller

- disable Turbo in the CPU settings

 

PCIe cards

Some PCIe cards prevent states other than C0 or C3 (Intel). As an example my Qnap Aquantia 10G card allowed only C3 while all my Intel 10G cards allowed C7. Without any installed card I'm able to reach C10.

 

iGPU

Install the iGPU driver plugin through unRAID Apps if your CPU has an iGPU

 

hpet

Set hpet=disable as a Kernel setting if you have an 8th, 9th or 10th Intel CPU as there is a bug which can cause crashes (shouldn't be necessary anymore since unRAID 6.10)

 

CPU governor

Use the tips & tweaks Plugin to set the CPU governor to "powersave" or use the below commands in the Go file.

 

10G networking

SFP+ DAC consumes less power than SFP+ SR/LR and both less than 10G RJ45. Depending on your internet connection the most efficient scenario is not to use a 10G switch. Instead directly connect your client with your server and use a second 1G cable to connect with your internet router. By that your 10G controller/card goes to sleep while your client is powered off. In comparison an active connection to a 10G switch never allows sleep states.

 

MTU

Surprisingly a user was not able to reach any C-State other than C1E after setting the MTU to 9000.

 

SATA Controller

Some mainboards or pcie cards provide SATA ports through an additional SATA controller. Sadly some of the most popular and usually well supported JMB xxx and ASM xxxx controllers have an incompatibility to "powertop --auto-tune" because it sets all SATA ports to "med_power_with_dipm". To bypass this problem, you should not use "powertop --auto-tune". Instead read the "Commands" paragraph to set the commands manually. Then you can remove the line with the "med_power_with_dipm" setting or you can set each hostX individually. Example:

# Enable SATA link power management (this skips SATA host3 and host4)
echo med_power_with_dipm | tee /sys/class/scsi_host/host1/link_power_management_policy
echo med_power_with_dipm | tee /sys/class/scsi_host/host2/link_power_management_policy
echo med_power_with_dipm | tee /sys/class/scsi_host/host5/link_power_management_policy
echo med_power_with_dipm | tee /sys/class/scsi_host/host6/link_power_management_policy

 

If you are using the ASM1166 you could even try to update the firmware.

 

NVMe

A user found out that a specific NVMe SSD allowed only C2, while with a Samsung Evo Plus the system reached C10.

 

Power supply

An unnecessary high-power and/or inefficient power supply can raise the power consumption by 5 to 15W. Choose wisely:

- as low power (watt) as possible

- as efficient as possible (like Titanium grade)

- avoid dual power supply setups (server grade hardware is not build for efficiency)

 

If you plan to build a setup with extremely low power consumption, consider these power supplies:

- PicoPSU (up to 200 watts, but is limited to ~4 SSDs as it has low 5V output)

- Corsair RM550x (2021) (this looses only 1 watt more than a PicoPSU at 10W to 20W load!)

- BeQuiet Pure Power 11 FM 550W (similar to the Corsair, but 2 watts more than a PicoPSU, but cheaper than the Corsair)

- Bequiet 12M only ~1W more than a PicoPSU

 

Notes:

- As of writing this post all other power supplies are much more inefficient. So don't think for example a Corsair SF450 is more efficient than the Corsair RM550x (2021) although the former is Platinum grade . It's not. I have both and the Platinum looses 4W more at 10W load. The reason is that some of the most recent power supplies include a new technology to reach the future ATX12VO standard, which has the requirement of being extremely efficient at low loads. For example Seasonic told me in 06/2022 that power supplies for ATX12VO are still in development.

- An HDD can consume up to 30W if you switch on your server, but in this state the other server components do not reach their maximum load. So summarize wisely to obtain the correct power supply size.

- Do not buy the Corsair / BeQuiet if you don't reach a power consumption under 30W: Above this, usual Platinum / Titanium power supplies are more efficient.

 

Commands

Instead of using "powertop --auto-tune" you could add the following power-saving commands to your /boot/config/go file (stop your array and execute them through your terminal to test them first) with the Config File Editor Plugin:

 

# -------------------------------------------------
# Set power-efficient CPU governor
# -------------------------------------------------
/etc/rc.d/rc.cpufreq powersave

# -------------------------------------------------
# Disable CPU Turbo
# -------------------------------------------------
[[ -f /sys/devices/system/cpu/intel_pstate/no_turbo ]] && echo "1" > /sys/devices/system/cpu/intel_pstate/no_turbo
[[ -f /sys/devices/system/cpu/cpufreq/boost ]] && echo "0" > /sys/devices/system/cpu/cpufreq/boost

# -------------------------------------------------
# Enable power-efficient ethernet
# -------------------------------------------------

# enable IEEE 802.3az (Energy Efficient Ethernet): Could be incompatible to LACP bonds!
for i in /sys/class/net/eth?; do dev=$(basename $i); [[ $(echo $(ethtool --show-eee $dev 2> /dev/null) | grep -c "Supported EEE link modes: 1") -eq 1 ]] && ethtool --set-eee $dev eee on; done

# Disable wake on lan
for i in /sys/class/net/eth?; do ethtool -s  $(basename $i) wol d; done

# -------------------------------------------------
# powertop tweaks
# -------------------------------------------------

# Enable SATA link power management
echo med_power_with_dipm | tee /sys/class/scsi_host/host*/link_power_management_policy

# Runtime PM for I2C Adapter (i915 gmbus dpb)
echo auto | tee /sys/bus/i2c/devices/i2c-*/device/power/control

# Autosuspend for USB device
echo auto | tee /sys/bus/usb/devices/*/power/control

# Runtime PM for disk
echo auto | tee /sys/block/sd*/device/power/control

# Runtime PM for PCI devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/power/control

# Runtime PM for ATA devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/ata*/power/control

 

Note: EEE can cause problems with 2.5G Intel Ethernet.

 

Config File Editor:

 

 

Debugging

 

Here are some helpful terminal commands.

 

Spin down all disks (Unraid only)

for dev in /dev/sd?; do /usr/local/sbin/emcmd cmdSpindown="$(grep -zoP "(?<=name=\")[a-z0-9]+(?=\"\ndevice=\"${dev: -3})" /var/local/emhttp/disks.ini | tr -d '\0')"; done

 

Obtain ASPM status of all devices:

lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'

 

Watch CPU frequency:

watch -n3 "cpufreq-info | grep 'current CPU'"

 

So I set up powertop autotune on my 13500 (I also have updated ASM1106) I see that the C states in the packets are not very big, could this be improved in any way?

 

@Nozle Please provide more information so we can help you. You probably made a typo and meant ASM1166, which is a well-known controller that supports ASPM (with updated firmware). To achieve higher C-states, the PC/server/NAS needs to be unplugged from the keyboard/mouse and LCD monitor. Maybe there is a device (like a PCI card, GPU, or disk, etc.) that prevents going to a deeper C-state. It's hard to tell if we know nothing about your hardware (except for the 13500 and ASM11...).

Edited March 28, 20251 yr by bagican

3 hours ago, bagican said:

@Nozle Please provide more information so we can help you. You probably made a typo and meant ASM1166, which is a well-known controller that supports ASPM (with updated firmware). To achieve higher C-states, the PC/server/NAS needs to be unplugged from the keyboard/mouse and LCD monitor. Maybe there is a device (like a PCI card, GPU, or disk, etc.) that prevents going to a deeper C-state. It's hard to tell if we know nothing about your hardware (except for the 13500 and ASM11...).

Sorry, my configuration was a few messages above:

 

 

13500 Seasonic SPX650 Asrock ITX PG TB4 Z790 64GB DDR5 6600 Corsair ASM1166 with latest firmware 6 fans total: 1 CPU, 2 HDDs at the back, 2 at the top of the motherboard, and one on the side 3 Toshiba N200 HDDs 2 NVME WD SN850X Everything is up to date (BIOS, NVME firmware, ASM, etc.)

 

 

PowerTop Autotune running

 

20 Dockers ( authentik, wallos, npm, inmich, dockge, paperless, etc)
 

 

 

2 hours ago, Nozle said:

20 Dockers ( authentik, wallos, npm, inmich, dockge, paperless, etc)

have you tried running powertop with 0 running dockers?

2 hours ago, bagican said:

have you tried running powertop with 0 running dockers?

No, because some are critical, like Authentik, npm, and Vaultwarden. The rest are disabled. Also, the HomeAssistant vm is running.

Followed up on my test and also installed OpenMediaVault for comparison purposes. 

 

OpenMediaVautl 7.4.17

c10: yes

idle: 3,2 W

 

Any idea how I can get Unraid down to c10 states and idle-values comparable to TrueNAS or OMV? 

 

Unraid 7.1 beta

c10: no

idle: 4.5 W

 

Debian 12 

c10: yes

idle: 2,9 W

 

TrueNAS Scale 25.04 beta

c10: yes

idle: 3,4 W

 

On 3/28/2025 at 8:05 PM, bagican said:

have you tried running powertop with 0 running dockers?

This is what I got by stopping all Docker programs except Authentik, npm, and Vaultwarden. I also ran the HomeAssistant virtual machine.