Reduce power consumption with powertop

[mgutt]

6 hours ago, Erka53 said:

After experiencing this i turned off the plugin from within Nerdtools but it still keeps happening.
 

Powertop does nothing if you install it. Only --auto-tune enables power saving features and those are lost on the next reboot.

[noVem]

Hello, thank for taking the time to write this all up @mgutt

 

It'd be appreciated if someone could look at my BIOS settings and see where I'm going wrong with enabling C-States. Since taking the photo I have disabled turbo mode. I also tried changing the 'Package C State Limit' from Auto to C7 with no luck.

 

Running a Asus Z170-WS with an i7-6700

 

I've also enabled something called 'EPU Power Saving Mode' in the BIOS which is described as "The ASUS EU (energy processing unit) applies settings for minimum system power consumption, Enabling this will apply lower EPU Core/Cache voltage and heip save energy consumption, Leave disabled if overclocking the system"

 

 

The manual for my motherboard has the options listed as such;

Spoiler

  

 

Edited October 26, 2022 by noVem

[mgutt]

4 hours ago, noVem said:

It'd be appreciated if someone could look at my BIOS settings and see where I'm going wrong with enabling C-States.

Please post your Powertop results.

 

And your ASPM results (see "Debugging" section in first post).

 

 

[J05u]

Just curious, which PSU will be more efficient for SFX build?

I want to move my 5 drives, 2 ssd and 2 nvme to small form factor with HBA card.  (i5 10400)

I see only SF450 from Corsair being small and good rating

[mgutt]

2 hours ago, J05u said:

Just curious, which PSU will be more efficient for SFX build?

https://forums.unraid.net/topic/98070-reduce-power-consumption-with-powertop/?do=findComment&comment=1182653

 

 

2 hours ago, J05u said:

I see only SF450 from Corsair

I have it and it consumes 3 to 4W more compared to the PicoPSU or Corsair RM550x (2021) on low loads.

[jesta]

Got another tip that works quite well for me!

Switching my CPU (Ryzen 7 5700g) to the "powersave" governor saves me about 10W under normal load because all cores will run at 1.4GHz. I only need higher clocks when gaming on my Win 10 VM so I automated governor switching depending on whether I'm logged in to Windows:

 

• install Putty and set up ssh key authentication to the UnRAID host, save a Putty profile for the server that utilises ssh key authentication
• open Group Policy Editor by hitting Win+R, entering gpedit.msc and hitting enter
• in Computer Configuration > Windows Settings > Scripts for Shutdown and User Configuration > Windows Settings > Scripts for logout create new entries for Start/Login and Shutdown/Logout:
• for script name input path to plink.exe (C:\Program Files\PuTTY\plink.exe)
• for script parameters input -batch <Putty profile> /etc/rc.d/rc.cpufreq powersave/ondemand

So you get a total of 4 entries (one each for start/shutdown/login/logout) that set the governor to powersave or ondemand (or whatever you prefer).

[J05u]

16 hours ago, mgutt said:

https://forums.unraid.net/topic/98070-reduce-power-consumption-with-powertop/?do=findComment&comment=1182653

 

 

I have it and it consumes 3 to 4W more compared to the PicoPSU or Corsair RM550x (2021) on low loads.

issue is that case can accep[t only SFX  Maybe i will need to move to Node 304 in the end of day

[ThomasEg]

Hi,

 

Been reading a bit on this topic because i got my hands on a new SuperMicro X12SDV-4C-SPT4F board which has a fairly new 4 core Xeon D 1700 cpu... i thought it might be a good low power unraid server... but i'm not quite seeing the "low power"-part which i had hoped :-/

 

My setup is basically:

- X12SDV-4C-SPT4F (Xeon D-1718T, 4-cores)
- 2x 64Gb RDIMM (Samsung M393A8G40AB2-CWE)

- Small samsung NVMe, no drives otherwise
- 10G SPF+ connected + BMC/IPMI RJ45 1Gbe connected, 10G Gbe disabled via jumper on motherboard
- PicoPSU 160XT with LEICKE 150W brick

 

The Shelly plug on the wall tells me it's drawing 36W idle... and peaking at around 53W during boot... i would have expected a lot lower than 36W, but i might be wrong here?

 

First off, i'm only seeing C1, C1E and C6 states....and nothing else...is this normal for Xeon-Ds? I see most other Intel-cpus in this thread having quite a lot more possible states, but this might be a xeon-d thing?

 

I've disabled turbo and a couple of other things in BIOS. But it doesn't really do anything to the powerconsumption...

 

I'm a missing something important here or is Xeon D just not as efficient in idle as i had expected?

[mgutt]

14 minutes ago, ThomasEg said:

X12SDV-4C-SPT4F + Xeon D-1718T

Not a good choice regarding efficiency and unraid:

- server hardware is not build for efficiency, that's why it does not even support all C-States 

- SFP25 seems to be provided through a controller which is inside the CPU, which means 8 permanently active PCIe 4.0 Lanes I think (inside the CPU of course)

- even though there is a jumper, I'm not sure the X550 controller is cut of power (please touch it after the server is running for a while)

- although it's a recent CPU, it is not really powerful and it's missing an iGPU (it makes dGPU passthrough harder and you can't use Plex or similar applications without installing a dGPU)

 

Finally a "good" example that T-CPUs do not safe energy and TDP values are useless: My recent W480M with Xeon W-1290E consumes 10W in idle. And this is a 10-core / 20-thread CPU with 95W TDP. And I did not use a PicoPSU.

45 minutes ago, ThomasEg said:

I've disabled turbo

You missed the last page if powertop. Are all settings "good"? Did you enable ASPM (L1) in the BIOS?

[screwbox]

Are those instructions from the first page relevant and actualized since 2020? Going to tune some powersavings on my Core i3 8100 Asus P11c-i Unraidserver and just want to make shure that im not applying old settings which are not any good these days...

[mgutt]

12 minutes ago, screwbox said:

actualized since 2020?

Asking for a CPU, which was released 2017 🤔 Ignoring "Edited October 14 by mgutt" 🤪

[screwbox]

So okay, there is something i'm missing obviously. The only thing that i was reading was "Posted October 21, 2020" in the top bar of your post. Literally searched the first page of this thread and can't even find "Edited October 14 by mgutt" so i think i'm too dumb to see this information.

But anyways, now that at least i got attention: The question was more in the direction if those informations are compatible and valid in combination of a 2017 CPU BUT with the newest stable release of Unraid since there were so many changes in the background like loosing the original nerdpack and nertools are missing some functionality etc. that i'm not sure if everything is compatible or relevant anymore.

Many how to's and tutorials got irrelevant these days beacuse of this.

A simple "yes, should be fine" or "no not working anymore because of this and that" would be sufficient. Never intended to throw a bad light on your efforts/tutorial and call them "outdated" etc., if you got it that way i'm sorry. Was not intentional!

Edited October 30, 2022 by screwbox

[J05u]

When I run autotune i am getting 

powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failed
Loaded 0 prior measurements
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be load

 

In tunables all showing as good

 

But in Pkg(HW) as on screenhot bad, all zeros

 

What i can do to fix it?

[mgutt]

Two interesting things for power saving nerds:

 

1.) This user found out that SATA cards with ASM1166 controller are compatible to powertop --auto-tune if a firmware update is installed:

https://forums.unraid.net/topic/102010-recommended-controllers-for-unraid/?do=findComment&comment=1185707

 

2.) Unraid 6.10 and 6.11 has a bug, which leaves multiple running scripts forever open (users are reporting 2 to 5W reduction in idle by executing the stated commands):

https://forums.unraid.net/bug-reports/stable-releases/power-consumption-is-not-stable-since-unraid-610-611-r2148/?do=findComment&comment=21046

 

[jaminben]

On 10/26/2022 at 4:05 PM, mgutt said:

Please post your Powertop results.

 

And your ASPM results (see "Debugging" section in first post).

 

 

 

Hello,

 

I've got more or less the same setup (ASUSTeK COMPUTER INC. Z170-A  + Intel® Core™ i7-6700K) and I'm also struggling to get mine to use anything over C3... I also don't appear to be able to run powertop --auto-tune without the following error:

 

Quote

powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failed
Loaded 0 prior measurements
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Leaving PowerTOP

 

 

Unraid Version: 6.10.3

 

Any help greatly appreciated please

 

 

 

 

 

Power usage has gone down from around 70-100W to 40-50W on average which is good.

 

Edited November 7, 2022 by jaminben

[BeersTeddy]

On 10/22/2022 at 12:24 AM, Christopher W. said:

Could you share your bios setting? I only get to c3

Not really. Don't have a monitor attached to it.

As far as I remember:

Latest bios + factory reset.

Can't remember changing any settings. Even if it would be only the obvious ones.

[cpu]

New build to replace my power hungry Ryzen 1600 AF + B450 mATX -> I used Fujitsu D3402-B2 + Core i5-7500 + 16 GB RAM and Corsair RM550x PSU currently idling at 7.2 W average after running powertop tune. Will report back once ASM1166 will arrive and 970 Pro Evo NVME. I've seen that ASM1166 requires firmware upgrade to properly support sleep states. 

Summary I've reduced usage from 70 W to 7.2 W so far but realistically I'm aiming at 11-12 W with fully deployed system. 

[crowdx42]

So I am getting the error below when I run auto tune. The system is a brand new setup with a 12600k and MSI Z690 motherboard. I have two Dell Perc 310 SAS controllers and a marvel SATA card in the machine. There are 20 hard drives and I am getting around 100 watts idle which I would love to get lower. 
root@UNRAID:~# powertop --auto-tune
powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failed
Loaded 0 prior measurements
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
Leaving PowerTOP

[dsmonteiro]

Hey everyone,

As much as I've tried to figure it out on my own, I can't see to sort it. I have a Z590 Aorus Master board + 11600 and no matter what I try, I can't seem to enter any C-state (please bear with me, I'm a self-taught noob).





Can anyone help me understand what I'm missing here? I'll gladly provide more information.

[mgutt]

17 hours ago, dsmonteiro said:

I can't seem to enter any C-state

Your CPU load (C0) is permanently at 4% and C1 is at least 10%. In addition your Aquantia network Controller does not support ASPM. At first lower your load.

 

And then post which steps you've already done from the first post of this thread.

[Interstellar]

So after mutt pointed me to this, I've had a go with powertop. To check BIOS settings requires connecting a monitor and I can't be bothered with that right now...

 

Running --auto-tune gives be a 6W(!) drop from 30W to 24W...

 

As expected my NIC prevents package C10, but the cores are going into C7 the majority of the time.

 

Not done hpet=disable yet

 

i5-8600T (50mV under-volt)

16GB DDR4 (non-ECC)

H310CM-HDV/M.2

1TB M2

Intel Quad-Port NIC

Four 14TB WD (Shucked)

320GB 2.5" (CCTV Stream)

 

OPNSense VM running on cores 4/5 (isolated)

 

Multiple Dockers include Home Assistant, Node Red, Plex, Grafana, InfluxDB, Telegraf, MairaDB, Mosquitto plus others...

 

           Pkg(HW)  |            Core(HW) |            CPU(OS) 0
                    |                     | C0 active   6.7%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          0.2%    0.1 ms
C2 (pc2)    6.7%    |                     |
C3 (pc3)   44.8%    | C3 (cc3)    2.5%    | C3          1.8%    0.1 ms
C6 (pc6)    0.0%    | C6 (cc6)   12.1%    | C6         14.6%    0.5 ms
C7 (pc7)    0.0%    | C7 (cc7)   69.7%    | C7s         0.1%    1.8 ms
C8 (pc8)    0.0%    |                     | C8         12.4%    0.9 ms
C9 (pc9)    0.0%    |                     | C9          0.3%    6.7 ms
C10 (pc10)  0.0%    |                     |
                    |                     | C10        58.4%    5.7 ms
                    |                     | C1E         4.0%    0.1 ms

                    |            Core(HW) |            CPU(OS) 1
                    |                     | C0 active   7.5%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          0.7%    0.1 ms
                    |                     |
                    | C3 (cc3)    2.6%    | C3          1.9%    0.1 ms
                    | C6 (cc6)    8.4%    | C6         12.3%    0.3 ms
                    | C7 (cc7)   63.2%    | C7s         0.1%    0.4 ms
                    |                     | C8          8.5%    0.6 ms
                    |                     | C9          1.5%    0.7 ms
                    |                     |
                    |                     | C10        58.6%    2.2 ms
                    |                     | C1E         7.8%    0.1 ms

                    |            Core(HW) |            CPU(OS) 2
                    |                     | C0 active   8.2%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          1.1%    0.0 ms
                    |                     |
                    | C3 (cc3)    1.7%    | C3          1.3%    0.1 ms
                    | C6 (cc6)   10.4%    | C6         13.0%    0.3 ms
                    | C7 (cc7)   62.5%    | C7s         0.2%    1.2 ms
                    |                     | C8         10.9%    0.6 ms
                    |                     | C9          3.1%    1.1 ms
                    |                     |
                    |                     | C10        52.0%    2.3 ms
                    |                     | C1E         9.2%    0.1 ms

                    |            Core(HW) |            CPU(OS) 3
                    |                     | C0 active   6.9%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          0.7%    0.1 ms
                    |                     |
                    | C3 (cc3)    2.8%    | C3          2.7%    0.1 ms
                    | C6 (cc6)    9.4%    | C6         13.3%    0.3 ms
                    | C7 (cc7)   63.7%    | C7s         0.0%    1.1 ms
                    |                     | C8          7.3%    0.5 ms
                    |                     | C9          1.0%    0.7 ms
                    |                     |
                    |                     | C10        59.0%    4.0 ms
                    |                     | C1E         7.9%    0.1 ms

                    |            Core(HW) |            CPU(OS) 4
                    |                     | C0 active  10.8%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          0.5%    0.0 ms
                    |                     |
                    | C3 (cc3)    4.2%    | C3          2.5%    0.1 ms
                    | C6 (cc6)    5.4%    | C6          8.8%    0.2 ms
                    | C7 (cc7)   67.1%    | C7s         0.0%    0.0 ms
                    |                     | C8         17.4%    0.7 ms
                    |                     | C9          0.5%    3.6 ms
                    |                     |
                    |                     | C10        52.1%    4.9 ms
                    |                     | C1E         6.3%    0.1 ms

                    |            Core(HW) |            CPU(OS) 5
                    |                     | C0 active  13.9%
                    |                     | POLL        0.0%    0.0 ms
                    |                     | C1          1.0%    0.0 ms
                    |                     |
                    | C3 (cc3)    0.9%    | C3          1.2%    0.1 ms
                    | C6 (cc6)    4.3%    | C6          5.5%    0.2 ms
                    | C7 (cc7)   70.7%    | C7s         0.0%    0.0 ms
                    |                     | C8          6.9%    0.7 ms
                    |                     | C9          2.5%    1.1 ms
                    |                     |
                    |                     | C10        62.9%    4.6 ms
                    |                     | C1E         6.5%    0.1 ms

                    |             GPU     |
                    |                     |
                    | Powered On  0.2%    |
                    | RC6        99.8%    |
                    | RC6p        0.0%    |
                    | RC6pp	  0.0%    |
                    |                     |

 

 

Will test over the next few weeks...!

Edited December 4, 2022 by Interstellar

[mgutt]

27 minutes ago, Interstellar said:

Not done hpet=disable yet

 

Shouldn't be necessary since Unraid 6.10. I think it was fixed in the Linux Kernel itself.

 

28 minutes ago, Interstellar said:

gives be a 6W(!) drop from 30W to 24W..

A nice step.

 

What I see:

- your iGPU is 0.2% on?! Did you connect a monitor or are you running Unraid in GUI mode or maybe the GPU is used through a container?

- your cores show a load between 7 and 10%, which is much. Are you having a VM running? As long the CPU does its work, the package itself can't reach deeper C-States (see the table in the first post, which shows the dependency between CPU and package C-States).

 

 

[crowdx42]

3 minutes ago, mgutt said:

Shouldn't be necessary since Unraid 6.10. I think it was fixed in the Linux Kernel itself.

 

A nice step.

 

What I see:

- your iGPU is 0.2% on?! Did you connect a monitor or are you running Unraid in GUI mode or maybe the GPU is used through a container?

- your cores show a load between 7 and 10%, which is much. Are you having a VM running? As long the CPU does its work, the package itself can't reach deeper C-States (see the table in the first post, which shows the dependency between CPU and package C-States).

 

Am I correct in thinking running Dockers also prevent the cpu from going into a deeper C state?

 

[Interstellar]

1 hour ago, mgutt said:

Shouldn't be necessary since Unraid 6.10. I think it was fixed in the Linux Kernel itself.

 

A nice step.

 

What I see:

- your iGPU is 0.2% on?! Did you connect a monitor or are you running Unraid in GUI mode or maybe the GPU is used through a container?

- your cores show a load between 7 and 10%, which is much. Are you having a VM running? As long the CPU does its work, the package itself can't reach deeper C-States (see the table in the first post, which shows the dependency between CPU and package C-States).

 

 

 

No idea - completely headless. None of my dockers need it as far as I'm aware, I don't pass it through to Plex for example. Don't even have the drivers installed - is it a problem its being used briefly? Up to 1.5% if I look now. Edit: Back to 0.3%

 

Is there a way to find out what is using the GPU?

 

I have about 15 dockers running and a OPNSense VM running on cores 4 and 5 (isolated), stopping all the dockers/VMs drops the average usage to low single digit CPU usage and drops 1-2W off the power.

 

 

Edited December 4, 2022 by Interstellar

[mgutt]

1 hour ago, crowdx42 said:

Am I correct in thinking running Dockers also prevent the cpu from going into a deeper C state?

Depends on the container and how much load it produces. As for example a minecraft container produces a "huge" load through its java processes. But usual containers like plex, nextcloud, etc do not. A VM is different from that. It comes with a huge overhead because of its emulation part. So using VMs usually prevents reaching the deepest C-States.

 

22 minutes ago, Interstellar said:

stopping all the dockers/VMs drops the average usage to low single digit CPU usage and drops 1-2W off the power.

Only 1W is strange. If I start minecraft or a Windows VM, the power consumption raises by around 10W.

 

23 minutes ago, Interstellar said:

Don't even have the drivers installed

For the iGPU? You should, but I think they are, because as far as I know powertop does not show the iGPU as long no drivers are installed.