Reduce power consumption with powertop

April 3, 20251 yr

Running Unraid 7.1.0 beta 2. Managed to compile and run the latest version of powertop from github which has support for Arrow Lake. With a 245k on an ASUS PRIME Z890M-PLUS WIFI (latest bios and Intel ME as of 3 April) I can successfully have some cores in C6 to C7, which is better than I expected running 47 docker containers. I have managed to get a percentage displayed at the package c states before, but usually it displays 0.0%. I think I can still have some gains somewhere in the bios. Might update this post later with some numbers with 0 docker containers or vms running.

Currently drawing about 39 Watts from the wall for anyone interested in watts.

2x32gb RAM
4x sata HDD not spinning

1x 4TB SN850x

Zigbee USB

JetKVM

Quote

April 3, 20251 yr

Author

33 minutes ago, 12323r said:

running 47 docker containers. I have managed to get a percentage displayed at the package c states before, but usually it displays 0.0%

Did you try while docker service is disabled? If you can verify that powertop is able to display them, you could then try to stop one container after another to find the "bad" ones. For example java based applications are really bad for packages states as of my experience.

Quote

April 3, 20251 yr

41 minutes ago, mgutt said:

Did you try while docker service is disabled? If you can verify that powertop is able to display them, you could then try to stop one container after another to find the "bad" ones. For example java based applications are really bad for packages states as of my experience.

I have disabled the docker service and can reach C2 on the package, I should be able to reach higher right?

One of my docker containers is biglybt (java torrent client) which I cannot replace by a different client unfortunately.

Quote

April 3, 20251 yr

Author

Maybe your problem is the Realtek 2.5Gb Ethernet Controller. Not sure how the current state is regarding the realtek driver and low power states. I checked a bigger photo of your board and maybe it helps to disable the aura chip (led controller, check the bios) as well.

Quote

April 4, 20251 yr

Hey guys, me again.

Was rocking my H610i system with onboard i219V 1GBE controller + picoPSU @ ~14 watt idle.
Now that managed 2,5GBE switches on aliexpress are cheap as hell, i have ordered 4x + an pcie I226-V card.
Back a few months i was trying a I226-T1 which was horrible back than.
I sufferd from the ASPM issue so that the link was down short after it was booted.
Now that kernel 6.12 is used in unraid 7.1.0-beta 2 i was giving it a try.

Read on the kernel github issues that there was some work regarding freezing in the igc driver.

First it looked good with the pcie I226-V card, but after deeper performance tests, i had an issue with speed.
iperf3 ran great a few runs (~2,37 GBE/s) but than i just dropped to under 100 MBIT/s.

Also coping files to the cache (nvme) over SMB dropped from stable 1 GBE with the i219v to sometimes stalling during copy!

So i digged deeper and after hours of tests (with other clients), BIOS restore defaults and powertop + BIOS setting hunting i found the issue.

When "PEG ASMP" in BIOS is on, the speed issue is here (but it reaches pkg pwr C6, with aspm enabled on the i226-V). With this turned OFF, the speed issue is gone (but package pwr doesnt even reach C1 and also ASPM is disabled for the i226-V).

Here is the lspci output when the issue is gone:

Quote

00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
LnkCap: Port #2, Speed 16GT/s, Width x16, ASPM not supported
LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk-
00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express])
LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

Sadly, because the pkg power reaches no higher cstates, my power consumption is up to ~22 Watt.

After some googling about "PEG ASPM" it turned out that this blog about a low power NAS is mentioning it to leave it enabled: https://mattgadient.com/7-watts-idle-on-intel-12th-13th-gen-the-foundation-for-building-a-low-power-server-nas/

If i leave it enabled, i can go back to the stable onboard i219V anyways

@ all here
Anyone with an I226-V and similar issues?
Maybe it's because an cheap one from aliexpress.

Any news on the realtek 8125b and cstates?
Last time i tested and stated above by @mgutt i also dont know the current state.

@mgutt
Which 2,5 GBE card do you rocking now?
Is it worth to try a genuineIntel I226-T1 again?

Thank you

pOpY

Edited April 4, 20251 yr by pOpYRaid

Quote

1

April 5, 20251 yr

I've also now tried the latest stable of OpenMediaVault and the system is able to hit c10 with idle power values of around 3.1 W.

Is there any chance, I can get Unraid to match at least TrueNAS Scale in terms of power consumption? Otherwise I'm going to use OVM, TrueNAS Scale or Debian + Docker instead. Shame for the lifetime Unraid license though...

Unraid 7.1 beta

c10: no

idle: 4.5 W

OpenMediaVault 7.4.17

c10: yes

idle: 3.1 W

Debian 12

c10: yes

idle: 2,9 W

TrueNAS Scale 25.04 beta

c10: yes

idle: 3,4 W

Ubuntu Desktop 22.04

c10: yes

idle: 2,9 W

Ubuntu Server 22.04

c10: no

idle: 3,1 W

Ubuntu Server 24.10

c10: no

idle: 3,1 W

Ubuntu Desktop 24.10

c10: no

idle: 3,4 W

Quote

April 5, 20251 yr

Author

20 minutes ago, TOMillr-old said:

Is there any chance, I can get Unraid to match at least TrueNAS Scale in terms of power consumption?

There is a chance. Log the output of lspci as for example:

lspci -vv >/mnt/user/lspci_unraid.txt

Do the same with truenas. Then upload the files and we compare which pcie devices / bridges have aspm enabled and active.

We then could try to set aspm through a script. Several people posted a this solution in this thread with other hardware setups.

Quote

April 5, 20251 yr

1 hour ago, mgutt said:

Do the same with truenas

What would be a good OS for comparing the LSPCI output? OMV and Debian 12 have even lower idle values. But would that be a realistic comparison?

Quote

April 5, 20251 yr

Author

Of course. Finally you could create the output of any OS. Maybe we then see a main difference between those and Unraid.

Quote

April 5, 20251 yr

I've previously already did the following readings:

Unraid 7.1 beta 1

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #3 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #7 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

Ubuntu Desktop 22.04.

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #3 (prog-if 00 [Normal decode])
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #7 (prog-if 00 [Normal decode])
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+

Debian 12

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+

TrueNAS Scale 25.04 rc1

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+

Quote

April 5, 20251 yr

On 10/19/2024 at 9:20 PM, spyrosj said:

Any suggestions on low power 10g SFP+ nics that are the best for direct connection? I'm contemplating on getting one for each of my servers so that backup jobs can run faster and potentially save power.

The backup job would probably be:

Wake-on-lan or wake on RTC my backup server every day at night

Run a CRON rsync job on reboot that transfers from the main server's cache to the backup server's cache

After the rsync job run mover on the backup server to move cache contents the main array which is JBOD

Let the S3 Sleep plugin or the script shutdown the backup server once the job completes and the drives are idle.

Did you get 10g SFP+ card yet? Any recommendations?

Quote

April 7, 20251 yr

On 4/5/2025 at 10:30 PM, TOMillr-old said:

I've previously already did the following readings:

@mgutt Are my old readings in any way conclusive? Or should I do some more?

Quote

April 11, 20251 yr

Hey Guys,

running Unraid 7.0.1

MB: MSI pro z690-A

Intel i5-12400

32GB DDR5

3HDD WD Red 8Tb - array

3x WD black sn770 nvme - zfs cache pool

I checked every possible setting in bios. I've reached a point where the powersave setting turned off one of the nvme and zfs pool got in degraded status. I've added a command to avoid ASPM for nvme in my go file. You can find it here below.

As you can see I cannot go higher than c3. with other tries, I reached C6.

When idle my server is 20-22W with all disk down. scaling_governor and pcie_aspm: both set to powersave.

Quote

April 12, 20251 yr

On 4/7/2025 at 3:14 PM, TOMillr-old said:

@mgutt Are my old readings in any way conclusive? Or should I do some more?

Anyone? I can't believe that a clean and idle installation of Unraid draws 26% more power compared to a beta release of TrueNAS scale or 48% more than a basic Debian server.

Quote

1

April 12, 20251 yr

On 4/4/2025 at 9:50 PM, pOpYRaid said:

Hey guys, me again.

Was rocking my H610i system with onboard i219V 1GBE controller + picoPSU @ ~14 watt idle.
Now that managed 2,5GBE switches on aliexpress are cheap as hell, i have ordered 4x + an pcie I226-V card.
Back a few months i was trying a I226-T1 which was horrible back than.
I sufferd from the ASPM issue so that the link was down short after it was booted.
Now that kernel 6.12 is used in unraid 7.1.0-beta 2 i was giving it a try.

Read on the kernel github issues that there was some work regarding freezing in the igc driver.

First it looked good with the pcie I226-V card, but after deeper performance tests, i had an issue with speed.
iperf3 ran great a few runs (~2,37 GBE/s) but than i just dropped to under 100 MBIT/s.

Also coping files to the cache (nvme) over SMB dropped from stable 1 GBE with the i219v to sometimes stalling during copy!

So i digged deeper and after hours of tests (with other clients), BIOS restore defaults and powertop + BIOS setting hunting i found the issue.

When "PEG ASMP" in BIOS is on, the speed issue is here (but it reaches pkg pwr C6, with aspm enabled on the i226-V). With this turned OFF, the speed issue is gone (but package pwr doesnt even reach C1 and also ASPM is disabled for the i226-V).

Here is the lspci output when the issue is gone:

Sadly, because the pkg power reaches no higher cstates, my power consumption is up to ~22 Watt.

After some googling about "PEG ASPM" it turned out that this blog about a low power NAS is mentioning it to leave it enabled: https://mattgadient.com/7-watts-idle-on-intel-12th-13th-gen-the-foundation-for-building-a-low-power-server-nas/

If i leave it enabled, i can go back to the stable onboard i219V anyways

@ all here
Anyone with an I226-V and similar issues?
Maybe it's because an cheap one from aliexpress.

Any news on the realtek 8125b and cstates?
Last time i tested and stated above by @mgutt i also dont know the current state.

@mgutt
Which 2,5 GBE card do you rocking now?
Is it worth to try a genuineIntel I226-T1 again?

Thank you

pOpY

Guys, i have found a settings combination which is stable for my setup and saves another ~5 Watt.
I am down to ~15.8 Watt (avg on idle) now with the I226-V and an stable 2,5 GBE speed.
With no montior, disks spun down, not accessing webgui, nvme cache running.
Tests performed with iperf3 and large SMB file copy to cache.
On all tests it has a stable 2,5 GBE connection with no speed issues.

The key was to enable "PEG ASPM" again, set the max cstate in the BIOS to C10 (this disables ASPM Control in BIOS completely).
Than enable "Native ASPM" in BIOS, this gives ASPM control to linux and you can enable/disable ASPM per device basis.

Now you can find the pcie bus address of your ethernet adapter with the command:

lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'

In my case it's 01:00.0.

Look in /sys/bus/pci/devices/ directory for a similar sounding directory, it's 0000:01:00 in my case.
do an:

ls /sys/bus/pci/devices/0000:01:00.0/link/

In there are the available ASPM modes, in my case it's just one file named:

l1_aspm

You can now manually disable/enable a ASPM mode for a particular device by writing into this file:

Enable ASPM:

echo 1 > /sys/bus/pci/devices/0000:01:00.0/link/l1_aspm

Disable ASPM:

echo 0 > /sys/bus/pci/devices/0000:01:00.0/link/l1_aspm

After writing to this file, check your changes by the lspci command from above.

It should say "disabled" for this particular device (in my case also for the PCIe controller because my NIC card depends on it).

Here is the output after i disabled ASPM for just the I226-V NIC:

root@MediaServer:~# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
                LnkCap: Port #2, Speed 16GT/s, Width x16, ASPM L1, Exit Latency L1 <16us
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk-
00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <4us
                LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express])
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
root@MediaServer:~#

As a bonus, if you want to have it persitent (after boot) add the following to your /boot/config/go file:

# disable ASPM for I226-V, when this is enabled the network speed drops often to <300 MB/s (instead of full performance of 2500 MB/s)
# with this workaround the system can reaches at least c-state c3 (~15,8 Watt) and have full ethernet performance
echo 0 > /sys/bus/pci/devices/0000:01:00.0/link/l1_aspm

Here are my powertop stats:



           Pkg(HW)  |            Core(HW) |            CPU(OS) 0   CPU(OS) 1
                    |                     | C0 active   0.7%        6.6%
                    |                     | POLL        0.0%    0.1 ms  0.0%    0.0 ms
                    |                     | C1E         2.0%    0.2 ms  3.6%    0.1 ms
C2 (pc2)    5.3%    |                     |
C3 (pc3)   47.0%    | C3 (cc3)    0.0%    |
C6 (pc6)    0.0%    | C6 (cc6)    6.5%    | C6          4.3%    0.9 ms  9.1%    0.9 ms
C7 (pc7)    0.0%    | C7 (cc7)   64.8%    |
C8 (pc8)    0.0%    |                     | C8          1.0%    0.9 ms  0.9%    0.9 ms
C9 (pc9)    0.0%    |                     |
C10 (pc10)  0.0%    |                     |

I am happy with the result.

Hope this helps someone.
pOpY

Quote

2

April 16, 20251 yr

On 4/12/2025 at 11:56 AM, TOMillr-old said:

Anyone? I can't believe that a clean and idle installation of Unraid draws 26% more power compared to a beta release of TrueNAS scale or 48% more than a basic Debian server.

Just upgrade my test system to the final release of TrueNAS Scale 25.04. Idle power consumption of an empty system is still 26% lower compared to the latest Unraid build. 👎

Any chance to get Unraid down to that level as well? The hardware is clearly capable of reaching c10.

Quote


 TrueNAS SCALE 25.04                                   (Leike 12V / 60W /5A)
┌───────────────────────────────────────────────────────────────────────────┐
│                              Odroid H4+                                   │
│                                                                           │
│  Intel N97                                                                │
│  1x Crucial 16GB DDR4                                                     │
│                                                                           │
│                           1Gbit/s                                         │
│     SATA SATA SATA SATA   LAN   LAN#2              USB2  USB2  USB3  USB3 │
│M.2   #1   #2   #3   #4    #1   unused               #1    #2    #1    #2  │
└─┬────┬────┬─────┬───┬──────┬─────────────────────────┬─────┬─────┬─────┬──┘
  │    │    │     │   │      │                       unused  │     │     │
  │    │    │     │   │      └──► Fritzbox 7590            unused  │     │
  │    │    │     │   │                                          unused  │
  │    │    │     │   └► unused                                        unused
  │    │    │     │
  │    │    │     └────► unused
  │    │    │     
  │    │    └──────────► unused      
  │    │
  │    │        (SSD)   ┌─────────────────────┐    
  │    └───────────────►│ WD Red SA400 500 GB │   
  │                     └─────────────────────┘    
  │ 
  └────────────────────► unused

Quote

____________________________________________________________________
                        P o w e r T O P

____________________________________________________________________
 *  *  *   System Information   *  *  *

PowerTOP Version;2.14 ran at Wed Apr 16 07:08:26 2025

Kernel Version;Linux version 6.12.15-production+truenas
System Name;HARDKERNELODROID-H41.0
CPU Information;4 Intel(R) N97
OS Information;Debian GNU/Linux 12 (bookworm)

Target: 1 units/s;System:  126.0 wakeup/s;CPU:  2.7% usage;GPU: 0 ops/s;GFX: 0 wakeups/s;VFS: 0 ops/s;

____________________________________________________________________
 *  *  *   Top 10 Power Consumers   *  *  *

Usage;Events/s;Category;Description
  0.1%; 44.0;Timer;tick_nohz_handler
  0.2%;  7.3;Process;[PID 4236] /usr/bin/python3 /usr/lib/netdata/plugins.d/python.d.plugin 1 
  0.0%;  7.7;Timer;inactive_task_timer
  0.3%;  4.5;Process;[PID 4234] /usr/bin/python3 /usr/lib/netdata/plugins.d/python.d.plugin 1 
  0.0%;  5.2;Process;[PID 3057] /usr/bin/containerd 
  0.0%;  4.5;kWork;psi_avgs_work
  0.2%;  3.4;Process;[PID 4232] /usr/bin/python3 /usr/lib/netdata/plugins.d/python.d.plugin 1 
  0.1%;  3.5;Interrupt;[7] sched(softirq)
  0.1%;  3.0;Process;[PID 3172] /usr/bin/python3 /usr/bin/cli --menu --pager 
  0.0%;  3.0;Process;[PID 17] [rcu_preempt]

____________________________________________________________________
 *  *  *   Processor Idle State Report   *  *  *

Package;0
Powered On;  0.0%
C1E;  1.1%
RC6;  0.0%
RC6pp;  0.0%
C8;  0.8%
C10; 96.5%
;

Quote

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port (prog-if 00 [Normal decode])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial AT

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave

cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference
balance_performance

Quote

1

April 17, 20251 yr

Got myself ASM1166 card for hopes to reduce power but my gpu started throwing errors when passing through to VM if pci aspm is enabled so got maybe max -5W from switching from HBA to that card.

All in all i have come down from 100w idle to 55-60w idle now mainly by setting 3700x to 45w eco mode and setting governow to on demand. Powersave governor makes makes windows VM sluggish.

10 containers. 1 linux VM and 1 windows VM with gpu passtrough.

Quote

April 19, 20251 yr

Hi there,

I am trying to get started with my unraid server. However, I am failing to get to a satisfying idle power consumption.

Here is my setup:

- ASRock HDV/M.2 Intel B760 with RTL8125BG 2.5 Gbit LAN

- Intel i5 12500

- 16GB Kingston ValueRAM DDR5-5600

- 500GB Samsung SSD 980 M.2 PCIe 3.0

- 20 TB Exos X20 (currently only HDD)

Except for a few plugins, my Unraid is clean and I don't even run an Array yet.

Relevant BIOS settings have been done (activating C states, disabling Turbo, disabling other peripherals).

Tunables are all on good after autotune.

I set Normal CPU Scaling Governor to Power Save via Tips and Tweaks plugin.

After some trouble with the Realtek chip, I managed to activate ASPM with ASPM-Helper:

00:06.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0 (rev 05) (prog-if 00 [Normal decode])
		LnkCap:	Port #5, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <16us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
00:1c.0 PCI bridge: Intel Corporation Raptor Lake PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
		LnkCap:	Port #1, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
		LnkCap:	Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 980 (DRAM-less) (prog-if 02 [NVM Express])
		LnkCap:	Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

I only reach C2 states (checked via SSH) and measured an idle power consumption of 25W with Tapo P110M.

What are things that I can check?

Quote

April 21, 20251 yr

On 4/3/2025 at 1:24 PM, 12323r said:

Running Unraid 7.1.0 beta 2. Managed to compile and run the latest version of powertop from github which has support for Arrow Lake. With a 245k on an ASUS PRIME Z890M-PLUS WIFI (latest bios and Intel ME as of 3 April) I can successfully have some cores in C6 to C7, which is better than I expected running 47 docker containers. I have managed to get a percentage displayed at the package c states before, but usually it displays 0.0%. I think I can still have some gains somewhere in the bios. Might update this post later with some numbers with 0 docker containers or vms running.

Currently drawing about 39 Watts from the wall for anyone interested in watts.

2x32gb RAM
4x sata HDD not spinning

1x 4TB SN850x

Zigbee USB

JetKVM

1. Latest bios drives up power consumption on that board. Downgrade:

2. SN850x has high idle power consumption; move this to chipset m.2 slots if you haven't already.

3. Use usb attached to chipset (USB ports with "*" in bios tell you which to use)

4. powertop is broken and likely won't get fixed anytime soon.

5. 2.5gbe must have aspm manually enabled

Quote

April 22, 20251 yr

After installing the RTL8125 Drivers plugin, I was able to reach C3 and around 17W with spun down HDD but running some Dockers. I read about unlocking some hidden Asrock BIOS settings, but I would have to install Windows in this machine to achieve it and this sounds like a lot of hassle for another 2-5 Watts possibly. Or is there any other way?

Quote

April 30, 20251 yr

On 4/12/2025 at 11:56 AM, TOMillr-old said:

Anyone? I can't believe that a clean and idle installation of Unraid draws 26% more power compared to a beta release of TrueNAS scale or 48% more than a basic Debian server.

Anyone got any ideas how to fix this?

Quote

May 2, 20251 yr

On 4/4/2025 at 9:50 PM, pOpYRaid said:

Now that managed 2,5GBE switches on aliexpress are cheap as hell, i have ordered 4x + an pcie I226-V card.
Back a few months i was trying a I226-T1 which was horrible back than.

@pOpYRaid, a couple of months ago I ordered some Chinese i226-V cards from Amazon to test them out. They are not the same as Intel i226-T1 card although sellers advertise them like that. The most significant difference between Intel’s and Chinese cards: chips and firmwares.

Chinese cards can’t get firmware updates through Intel’s NVM updater. These cards simply aren’t recognized. They also use custom/different version naming, not compatible with what Intel is using. What firmwares are used in these Chinese cards and how old they are we can only guess. I tried to cross-flash them (I had a successful cross-flashing experience with X710 card in past), but I didn’t succeed this time. Just for a note, i226-V controllers are often used in consumer mobos and they do have the same problem: motherboard vendors use some unknown/custom firmware that isn’t recognized by Intel’s NVM updater as well. Also, Intel denies to provide any support for such controllers asking to reach out a mobo’s vendor. I read somewhere that stubborn guys did a cross-flashing for some mobo’s and controllers just stopped being recognized by motherboards, so there is a risk.

The second difference is that Intel I226-T1 card is based on i226-LM chip, not i226-V one like Chinese cards. According to datasheets the LM chip is a “commercial and server version with long life supply” and it supports AMT (Active Management Technology). The V version is described as a “non-commercial version”, without AMT. The rest looks the same (features, power consumption, etc.)

Basically, Intel’s card could be a totally different thing due to a differences in chips and firmwares, especially considering how Intel is constantly improving it. Personally, I would bet on Intel’s card.

Edited May 2, 20251 yr by white-orb

Quote

May 6, 20251 yr

Hi I am new to unraid was trying to get my GMKtec G9 nas (Intel N150 with 2x intel i226 ethernet) wattage down since its currently 25 watts idle. I tried the commands from page 1 here

Ran the commands from inside unraid terminal, and ran:

powertop --auto-tune &>/dev/null

This just causes my system to hang, have to hard reset the GMKtec G9 nas. I am just wondering do I need to do some bios tweaking or enable some bios features before running that powertop command?

tuntables option shows bad states still. The only one that worked was the setting powersave on the cpu governer, but that did nothing for saving power.

Quote

May 8, 20251 yr

Tried the final version of 7.1 and power consumption during idle is still way worse compared to OMV, Debian or even TrueNAS scale.

In OMV, even with some containers like Plex running, power consumption is still 22% lower than a clean install of Unraid 7.1 with nothing running at all. ☹️ What's idle Unraid doing to justify the additional power need?

Quote


Unraid 7.1                                             (Leike 12V / 60W /5A)
┌───────────────────────────────────────────────────────────────────────────┐
│                              Odroid H4+                                   │
│                                                                           │
│  Intel N97                                                                │
│  1x Crucial 16GB DDR4                                                     │
│                                                                           │
│                           1Gbit/s                                         │
│     SATA SATA SATA SATA   LAN   LAN#2              USB2  USB2  USB3  USB3 │
│M.2   #1   #2   #3   #4    #1   unused               #1    #2    #1    #2  │
└─┬────┬────┬─────┬───┬──────┬─────────────────────────┬─────┬─────┬─────┬──┘
  │    │    │     │   │      │                         │     │     │     │
  │    │    │     │   │      └──► Fritzbox 7590        │   unused  │     │
  │    │    │     │   │                                │         unused  │
  │    │    │     │   └► unused                        │               unused
  │    │    │     │                                    │
  │    │    │     └────► unused                        │   ┌────────────────────────┐ 
  │    │    │                                          └──►│ Transcent JetFlash 700 │  
  │    │    │                                              └────────────────────────┘   
  │    │    │                                               
  │    │    └──────────► unused      
  │    │                  
  │    └───────────────► unused                        
  │ 
  └────────────────────► unused

Quote

lspci -vvPPDq | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:./]+|:\sASPM (\w+)? ?((En|Dis)abled)?)';

0000:00:1c.0 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #3 (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #4 (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6 PCI bridge: Intel Corporation Alder Lake-N PCI Express Root Port #7 (prog-if 00 [Normal decode])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.0/01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.3/02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
0000:00:1c.6/03:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave

cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference
balance_performance

Quote

powertop -C &> /dev/null && cat powertop.csv | head -n 42

____________________________________________________________________
                        P o w e r T O P

____________________________________________________________________
 *  *  *   System Information   *  *  *

PowerTOP Version;2.15 ran at Wed May  7 13:07:17 2025

Kernel Version;Linux version 6.12.24-Unraid
System Name;HARDKERNELODROID-H41.0
CPU Information;4 Intel(R) N97
OS Information;Unraid OS 7.1 x86_64

Target: 1 units/s;System:  530.7 wakeup/s;CPU:  9.4% usage;GPU: 0 ops/s;GFX: 0 wakeups/s;VFS: 0 ops/s;

____________________________________________________________________
 *  *  *   Top 10 Power Consumers   *  *  *

Usage;Events/s;Category;Description
  0.4%;413.2;Timer;tick_nohz_handler
  0.1%; 19.9;Process;[PID 18] [rcu_preempt]
  0.1%; 11.9;Interrupt;[7] sched(softirq)
  0.0%;  9.8;Interrupt;[3] net_rx(softirq)
  0.0%;  9.6;kWork;bond_mii_monitor
  0.0%;  7.5;Timer;inactive_task_timer
  0.0%;  4.4;kWork;delayed_vfree_work
  0.0%;  3.8;Process;[PID 7771] docker
  0.0%;  3.6;kWork;usb_giveback_urb_bh
  0.6%;  0.9;Process;[PID 7232] /usr/bin/php -q /usr/local/emhttp/webGui/nchan/update_3 

____________________________________________________________________
 *  *  *   Processor Idle State Report   *  *  *

Package;0
C2 (pc2);  1.3%
C3 (pc3);  0.0%
C6 (pc6);  0.2%
C7 (pc7);  0.0%
C8 (pc8); 75.6%
C9 (pc9);  0.0%
C10 (pc10);  0.0%
;
;

Edited May 10, 20251 yr by TOMillr-old

Quote

1

May 10, 20251 yr

Unraid 7.1.1

Intel UItra 265K

TUF GAMING B860M-PLUS WIFI (RTL8125)

Pretty sure I've exhausted all combinations of relevant BIOS settings (native ASPM, C10, turbo off, etc) as well as disabling hardware features up to disabling the RTL8125 and swapping in an Intel I340-T2. This is a test install of Unraid so it has next to nothing going on or plugged in otherwise.

powertop seems to be having issues representing states higher than C3, but tuneables are good. I did notice this is showing pkg(OS) while @12323r is getting pkg(HW) from 2.15 so not sure if I'm missing another aspect of Arrow Lake compatibility or some other factor.

PowerTOP 2.15     Overview   Idle stats   Frequency stats   Device stats   Tunables   WakeUp                            

           Pkg(OS)  |            CPU(OS) 0
Powered On  0.0%    | POLL        0.0%    0.0 ms
C1_ACPI     3.3%    | C1_ACPI     0.8%    0.2 ms
C2_ACPI    13.4%    | C2_ACPI    23.1%    0.9 ms
C3_ACPI    72.7%    | C3_ACPI    71.0%    1.8 ms
RC6pp       0.0%    |

In corefreq it does acknowledge that the core are going into higher C states (oddly P cores reaching higher states than E cores).

--- Freq(MHz) Ratio - Turbo -- C0 ---- C1 -- C2:C3 - C4:C6 --- C7 -- Min TMP Max 
000    3.31 ( 0.03)   0.09%   0.44%   0.25%   0.00%   1.23%  97.94%  32 / 32/ 36                                                                 
001    3.50 ( 0.04)   0.09%   0.25%   0.10%   0.00%   1.68%  97.89%  32 / 34/ 36                                                                 
...                                                              
006    2.05 ( 0.02)   0.05%   0.12%   0.00%   0.00%   0.00%  99.84%  32 / 32/ 36                                                                 
007    2.38 ( 0.02)   0.06%   0.17%   0.02%   0.00%   0.00%  99.77%  34 / 34/ 36                                                                 
008    8.88 ( 0.09)   0.23%   1.11%  12.43%   0.00%  86.27%   0.00%  32 / 32/ 34                                                                 
009   12.79 ( 0.13)   0.33%   1.60%   3.33%   0.00%  94.74%   0.00%  32 / 32/ 34                                                                 
...

lspci looks good with all devices having ASPM L1 enabled.

root@Tower:~# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:06.0 PCI bridge: Intel Corporation Device ae4d (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #13, Speed 32GT/s, Width x16, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk-
80:1b.0 PCI bridge: Intel Corporation Device 7f44 (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #21, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
80:1c.0 PCI bridge: Intel Corporation Device 7f38 (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk-
80:1c.4 PCI bridge: Intel Corporation Device 7f3c (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
80:1d.0 PCI bridge: Intel Corporation Device 7f36 (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #15, Speed 16GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable+ CommClk+
80:1d.7 PCI bridge: Intel Corporation Device 7f37 (rev 10) (prog-if 00 [Normal decode])
                LnkCap: Port #16, Speed 16GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable+ CommClk+
81:00.0 Non-Volatile memory controller: Micron/Crucial Technology P5 Plus NVMe PCIe SSD (prog-if 02 [NVM Express])
                LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <8us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
83:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
                LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <4us, L1 <8us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+
83:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
                LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <4us, L1 <8us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk+

Basically sitting at about 25W with a power supply that is probably only 70% efficient at that load. Not terrible but I didn't really gain much over Bios default and feel like something is still hung up at lower package states. Going to try ubuntu to see what I get, any suggestions in the interim?

Quote

Reduce power consumption with powertop

Featured Replies

Top Posters In This Topic

Popular Days

Most Popular Posts

genesisdoeswhatnintendont

mgutt

TheLinuxGuy

Posted Images

Join the conversation

Top Posters In This Topic

Popular Days

Most Popular Posts

genesisdoeswhatnintendont

mgutt

TheLinuxGuy

Posted Images

Account

Navigation

Search

Configure browser push notifications

Chrome (Android)

Chrome (Desktop)

Safari (iOS 16.4+)

Safari (macOS)

Edge (Android)

Edge (Desktop)

Firefox (Android)

Firefox (Desktop)