Reduce power consumption with powertop

[mgutt]

6 hours ago, sasbro97 said:

Zigbee socket with energy meter before the USV. You think its okay

Yes, but only Aqara has a good accuracy.

 

6 hours ago, sasbro97 said:

I don't need to worry about the warnings, right

Correct.

[phatass]

Hi,

 

Have my "new" server up and running and would like to reduce power consumption.

Have installed powertop as described in #1, but am a little scared using "-autotune" due to the SATA concerns described.

I am using an MSI Pro Z790 motherboard and a sata extension card.

They show up as follows in Tools -> System Devices:

- [8086:7a62] 00:17.0 SATA controller: Intel Corporation Device 7a62 (rev 11)

- [197b:0585] 02:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller (NVMe Drive)

 

I am an absolute Unraid/linux rookie and am scared to mess up something important.

 

Should I run autotune or not?

 

Where will the -autotune config be stored after having run the command, and can I revert settings if something goes wrong?

 

Thanks!

[mgutt]

1 hour ago, phatass said:

Where will the -autotune config be stored

It's nothing like a config file. It's only saying the device to use standby. After reboot this setting is lost.

 

1 hour ago, phatass said:

JMB58x

Disks connected to this SATA Controller could loose connection. You should stop the array and execute autotune. After that you should use this to access several times some data from the different disks:

 

dd if=/dev/sdX of=/dev/null bs=$bs iflag=count_bytes count=1G

 

Replace sdX against the correct device name as mentioned on the main page.

 

After that open Tools > syslog and check the last entries if there are any sdX related messages / SATA disconnect messages.

 

Repeat this step after spindown. But I'm not sure if the spindown button is available as long the array is stopped. If not, try this command:

 

for dev in /dev/sd?; do /usr/local/sbin/emcmd cmdSpindown="$(grep -zoP "(?<=name=\")[a-z0-9]+(?=\"\ndevice=\"${dev: -3})" /var/local/emhttp/disks.ini | tr -d '\0')"; done

 

After that execute dd again to wake it up. If your JMB controller is incompatible to standby, you will see SATA disconnects in the syslog. If no messages appear in the logs, it probably is unaffected. Finally the risk is low as it does not "destroy" your array. It "only" could be possible that a disks is removed from the array and emulated. But finally it still contains all its data.

[Stubbs]

I am hitting between 50W and 54W idle with 8 HDDs spun down, and running powertop --quiet --auto-tune.

I am using a Seasonic Focus Gold 550W PSU.

 

To clarify, would there be no benefit of changing to an ATX12VO PSU like an RM550x? The guide says to avoid buying one unless you can get down to 30W, but would using that PSU literally help me get closer to 30W from 50W?

I could also go for a SeaSonic FOCUS Plus Platinum 550 W, but I'm not sure simply going to platinum is worth it.

[mgutt]

4 hours ago, Stubbs said:

The guide says to avoid buying one unless you can get down to 30W, but would using that PSU literally help me get closer to 30W from 50W?

As long your current power supply does not have an extremely bad efficiency, no. The benefit depends on the load. Usually the power supplies efficiency curve has a "logistic growth", but the newer ATX12VO standard (which is still not official) caused the usage of new technologies which added an additional "bump" at the beginning of the efficiency curve, which is easily visible for the RM1000x (2021):

https://www.tweakpc.de/hardware/tests/netzteile/corsair_rmx_2021_550_750_1000/s02.php

 

This might lead you to the conclusion that it can be useful to change the power supply even up to 50W, but if you compare the RM550x (2021) at 25W:

 

with for example the SF450 Platinum, you see a similar efficiency:

https://www.tweakpc.de/hardware/tests/netzteile/corsair_sf750_platinum/s03.php

 

I mean yes, the RM550x (2021) is even better up to 60W, but the benefit is with 0.x to 1.x W extremely low. So the lesser the load, the better the benefit of the RM550x (2021).

 

 

[dopeytree]

Has anyone else encouraged a problem with array drives not staying spun down in 6.12.x?

ZFS is only on cache drives. Array is XFS. All drives are sata -> motherboard. There is a LSI HBA card but currently unused as waiting for cage to arrive.

Edited July 9, 2023 by dopeytree

[Graxo]

Hi,

 

Does anyone have experience with this in combination with an AMD Ryzen 5 4600G?

Please PM me

Because my server hangs now when i boot it.. it looks broken orsomething.

 

Kind regrads,

Graxo

Edited July 12, 2023 by Graxo

[TheLinuxGuy]

On 7/1/2023 at 6:33 PM, mgutt said:

Kontron Mainboard (2x NVMe + various PCIe Slots + 6 SATA ports)...already consumes less then 5W in idle. 

Do you mind sharing which specific Kontron motherboard is this?

[JaimeUriarte]

Hello everyone.

I just finish building my first home media server, everything works fine. Now I'm trying to reduce power consumption now it idles at 33W with HDD spin down and 43 with disk spin up.

I just followed the suggestions in this post but while executing powertop I'm not getting any readings for CPU Pkg (HW).

 

My build is the following:

Intel i3 10th Gen 10100

Gigabyte H410M S2H V2 with F3 BIOS

Cooler Master MWE 80+ Bronze V2 650W

WD_Blue_SN570_1TB nvme cache

WDC WD101EDBZ-11B1DA0 10TB for Array

Patriot Memory 32GB Boot Device

The case is a BitFenix Nova MESH M-ATX ARGB with an extra Be Quiet! 80mm Pure Wings 2 PWM for the drives area

 

So what I am missing to make powertop able to improve my power consumption?

*EDIT:

It seems I was missing an option in the BIOS to allow the OS to make power saving changes, so now its achieving pc8 state idling 21W and stable 34W with dockers and HDD spinning. Lets see if I can reduce it further.

Edited July 13, 2023 by JaimeUriarte

[TheLinuxGuy]

While debugging C states on my other test box I came across an intel C state test tool: https://github.com/intel/S0ixSelftestTool/tree/main - looks interesting.

 

root@ubuntu:/home/ubuntu/S0ixSelftestTool# ./s0ix-selftest-tool.sh -r on

---Check Runtime PC10 Residency during Screen ON---:

The system OS Kernel version is:
Linux ubuntu 6.2.0-20-generic #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  6 07:48:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

modprobe cpufreq_stats failedLoaded 0 prior measurements
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only
RAPL device for cpu 0
RAPL Using PowerCap Sysfs : Domain Mask d
RAPL device for cpu 0
RAPL Using PowerCap Sysfs : Domain Mask d
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only
Leaving PowerTOP

The system will keep idle for 40 seconds then check runtime PC10 state:


The CPU runtime PC10 residency when screen ON: 0.00%
The CPU runtime PC8 residency when screen ON: 0.00%


Turbostat log:
30.002281 sec
CPU%c1  CPU%c6  CPU%c7  GFX%rc6 Pkg%pc2 Pkg%pc3 Pkg%pc6 Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10
0.82    0.00    98.88   100.03  27.48   68.35   0.00    0.00    0.00    0.00    0.00
0.51    0.00    98.86   100.03  27.48   68.35   0.00    0.00    0.00    0.00    0.00
1.10
1.14    0.00    98.56
0.95
0.73    0.00    99.13
0.61
0.61    0.00    99.05
0.86
0.36    0.00    99.50
0.48
1.70    0.00    98.16
0.83


Your system did not achieve the runtime PC10 state during screen ON


Your system Intel graphics DMC FW loaded status is:yes


Intel graphics i915 DMC FW is loaded. Will re-check the deeper
Package C-state by ignoring PCI Devices LTR value:

All the PCI devices LTR values ignore is done!

The system will keep idle for 40 seconds then check runtime PC10 state:


The CPU runtime PC10 residency when screen ON: 0.00%
The CPU runtime PC8 residency when screen ON: 0.00%


Turbostat log:
30.003155 sec
CPU%c1  CPU%c6  CPU%c7  GFX%rc6 Pkg%pc2 Pkg%pc3 Pkg%pc6 Pkg%pc7 Pkg%pc8 Pkg%pc9 Pk%pc10
0.92    0.00    98.72   100.03  28.93   66.27   0.00    0.00    0.00    0.00    0.00
0.42    0.00    99.19   100.03  28.93   66.27   0.00    0.00    0.00    0.00    0.00
0.78
0.90    0.00    98.69
1.00
0.98    0.00    98.56
1.05
0.62    0.00    98.99
0.89
0.67    0.00    99.07
0.77
1.99    0.00    97.81
0.93


Your system did not achieve the runtime PC10 state during screen ON


PCI devices LTR value ignore does not help the PC10, will
check PCIe Link PM states:

Available bridge device: 0000:00:1d.0

The PCIe bridge link power management state is:
0000:00:1d.0 Link is in L1

The link power management state of PCIe bridge: 0000:00:1d.0 is not expected.
which is expected to be L1.1 or L1.2, or user would run this script again.


The L1SubCap of the failed 0000:00:1d.0 is:


The L1SubCtl1 of the failed 0000:00:1d.0 is:



Checking PCI Devices tree diagram:
-[0000:00]-+-00.0  Intel Corporation 11th Gen Core Processor Host Bridge/DRAM Registers
           +-02.0  Intel Corporation TigerLake-H GT1 [UHD Graphics]
           +-08.0  Intel Corporation GNA Scoring Accelerator module
           +-0a.0  Intel Corporation Tigerlake Telemetry Aggregator Driver
           +-0d.0  Intel Corporation Tiger Lake-H Thunderbolt 4 USB Controller
           +-14.0  Intel Corporation Tiger Lake-H USB 3.2 Gen 2x1 xHCI Host Controller
           +-14.2  Intel Corporation Tiger Lake-H Shared SRAM
           +-15.0  Intel Corporation Tiger Lake-H Serial IO I2C Controller #0
           +-15.2  Intel Corporation Device 43ea
           +-15.3  Intel Corporation Device 43eb
           +-16.0  Intel Corporation Tiger Lake-H Management Engine Interface
           +-17.0  Intel Corporation Device 43d3
           +-19.0  Intel Corporation Device 43ad
           +-19.1  Intel Corporation Device 43ae
           +-1d.0-[01]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           +-1e.0  Intel Corporation Device 43a8
           +-1e.3  Intel Corporation Device 43ab
           +-1f.0  Intel Corporation Tiger Lake-H LPC/eSPI Controller
           +-1f.4  Intel Corporation Tiger Lake-H SMBus Controller
           \-1f.5  Intel Corporation Tiger Lake-H SPI Controller

 

[rikdegraaff]

Hi @mgutt,

 

Wanna download and use powertop on my unraid server.

From your first post, I've done the first three commands, but I still cannot get powertop installed.
FYI Github.com doesn't work anymore, maybe that's why?

 

Then I reboot and put powertop in the terminal and get the following code:

bash: powertop: command not found

 

What is going on?

Because I really need to get the power consumption lower, it costs me about 450 euro's per year for electricity, that's very inefficient.

 

Thanks in advance!

 

EDIT: When I look at /boot/extra/ I find a file called powertop. It is ~14 Kb big. Is that OK? 

Edited July 16, 2023 by rikdegraaff
See Edit

[luzankia]

It should be "powertop-2.15-x86_64-1.txz" and 2.17MB in size.

 

EDIT : I just checked https://github.com/mgutt/unraid-packages/raw/main/6.11.0/powertop-2.15-x86_64-1.txz and the link work for me.

Edited July 22, 2023 by luzankia
link test

[fwiler]

On 7/1/2023 at 3:33 PM, mgutt said:

or Lenovo Tiny PC (eg M920x supports 4x NVMe + 1x SATA) already consumes less then 5W in idle. 

 

Tried searching for 4 nvme + 1 sata on M920x. Not seeing anything.  And especially not less than 5w in idle with 5 drives.  Do you have a link or know what card was used in the pcie slot?  I can't find one small enough to house 2 drives and that won't overheat.

[rikdegraaff]

3 hours ago, luzankia said:

It should be "powertop-2.15-x86_64-1.txz" and 2.17MB in size.

 

EDIT : I just checked https://github.com/mgutt/unraid-packages/raw/main/6.11.0/powertop-2.15-x86_64-1.txz and the link work for me.

 

Thanks, I just downloaded it to my pc and via my pc copied to unRAID flash drive.
I'll hope that by the next reboot it works.

And indeed now is the file 2.17MB in size...

 

If it doesn't work you'll hear me in this topic.
Thanks again.

 

Kind Regards!

[Lucascoco]

Hello, must i ran the command

powertop --auto-tune

after every reboot ?

[rikdegraaff]

Does anyone knows if the Supermicro AOC-SAS2LP-MV8 support link power management (https://www.supermicro.com/en/products/accessories/addon/AOC-SAS2LP-MV8.php).
The card is based on the Marvell 9480 host controller...

 

Can't find anything about it on Google, so maybe you guys know some more?

 

Kind regards!

[dopeytree]

Does your motherboard support ASPM? you want to set to L 1 for max power saving on pcie.

 

L0 – All PCI Express transactions and other operations are enabled.

L0s – A low resume latency, energy saving “standby” state. 

L1 – Higher latency, lower power “standby” state (optional).

L0sL1 - Activate both L0s and L1 support.

 

https://www.supermicro.com/support/faqs/faq.cfm?faq=22413#:~:text=L0 – All PCI Express transactions,standby” state (optional).

Edited July 25, 2023 by dopeytree

[rikdegraaff]

8 hours ago, dopeytree said:

Does your motherboard support ASPM? you want to set to L 1 for max power saving on pcie.

 

L0 – All PCI Express transactions and other operations are enabled.

L0s – A low resume latency, energy saving “standby” state. 

L1 – Higher latency, lower power “standby” state (optional).

L0sL1 - Activate both L0s and L1 support.

 

https://www.supermicro.com/support/faqs/faq.cfm?faq=22413#:~:text=L0 – All PCI Express transactions,standby” state (optional).

I have a Gigabyte B560M Aorus ELITE, as mobo, so don't know if that motherboard supports ASPM.

Dunno what ASPM is at all... 🙈🙈

 

EDIT: Yes my motherboard supports ASPM.
So does my card does so too?

Edited July 25, 2023 by rikdegraaff
See EDIT

[mgutt]

3 hours ago, Lucascoco said:

must i ran the command

powertop --auto-tune

after every reboot

Yes. You could add the command to the /boot/config/go file.

[dopeytree]

22 hours ago, rikdegraaff said:

I have a Gigabyte B560M Aorus ELITE, as mobo, so don't know if that motherboard supports ASPM.

Dunno what ASPM is at all... 🙈🙈

 

EDIT: Yes my motherboard supports ASPM.
So does my card does so too?

 

All my motherboards by default  have come with the setting of L0 for PCIE ASPM (pcie power management)

 

Anyway boot into bios then pcie options or similar and it should be there. 

 

Usually you have individual choices for about x8 pcie options.

Set them to L1 and hit save and reboot.

 

Then also make sure your unraid go file has the pcie setting: 

 

# Runtime PM for PCI devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/power/control

 

Here's is my full go file as an example.

 

#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &

# -------------------------------------------------
# Console Alias
# -------------------------------------------------

# show cpu core frequencies
echo "alias cpu=\"watch -n3 \\\"cpufreq-info | grep 'current CPU'\\\"\"">>/etc/profile


# -------------------------------------------------
# powertop tweaks
# -------------------------------------------------

# Enable SATA link power management
echo med_power_with_dipm | tee /sys/class/scsi_host/host*/link_power_management_policy

# Runtime PM for I2C Adapter (i915 gmbus dpb)
echo auto | tee /sys/bus/i2c/devices/i2c-*/device/power/control

# Autosuspend for USB device
echo auto | tee /sys/bus/usb/devices/*/power/control

# Runtime PM for disk
echo auto | tee /sys/block/sd*/device/power/control

# Runtime PM for PCI devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/power/control

# Runtime PM for ATA devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/ata*/power/control
 
# ------------------------------------------------
# Force Nvidia GPU idle
# ------------------------------------------------

# nvidia-smi --persistence-mode=1 nvidia-persistenced

# fuser -v /dev/nvidia*

# ------------------------------------------------
# Force Wake On Lan
# ------------------------------------------------

ethtool -s eth0 wol g

ethtool -s eth1 wol g

# -------------------------------------------------
# Enable power-efficient ethernet
# -------------------------------------------------

# enable IEEE 802.3az (Energy Efficient Ethernet): Could be incompatible to LACP bonds!
for i in /sys/class/net/eth?; do dev=$(basename $i); [[ $(echo $(ethtool --show-eee $dev 2> /dev/null) | grep -c "Supported EEE link modes: 1") -eq 1 ]] && ethtool --set-eee $dev eee on; done

 

Edited July 26, 2023 by dopeytree

[rikdegraaff]

1 hour ago, dopeytree said:

 

All my motherboards by default have come with the setting of L0.

 

Any boot into bios then pcie options or similar and it should be there. 

Usually you have individual choices for about x8 pcie options.

Set them to L1 and hit save and reboot.

Then also make sure your unraid go file has the pcie setting: 

 

# Runtime PM for PCI devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/power/control

 

Here's is my full go file as an example.

 

#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &

# -------------------------------------------------
# Console Alias
# -------------------------------------------------

# show cpu core frequencies
echo "alias cpu=\"watch -n3 \\\"cpufreq-info | grep 'current CPU'\\\"\"">>/etc/profile


# -------------------------------------------------
# powertop tweaks
# -------------------------------------------------

# Enable SATA link power management
echo med_power_with_dipm | tee /sys/class/scsi_host/host*/link_power_management_policy

# Runtime PM for I2C Adapter (i915 gmbus dpb)
echo auto | tee /sys/bus/i2c/devices/i2c-*/device/power/control

# Autosuspend for USB device
echo auto | tee /sys/bus/usb/devices/*/power/control

# Runtime PM for disk
echo auto | tee /sys/block/sd*/device/power/control

# Runtime PM for PCI devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/power/control

# Runtime PM for ATA devices
echo auto | tee /sys/bus/pci/devices/????:??:??.?/ata*/power/control
 
# ------------------------------------------------
# Force Nvidia GPU idle
# ------------------------------------------------

# nvidia-smi --persistence-mode=1 nvidia-persistenced

# fuser -v /dev/nvidia*

# ------------------------------------------------
# Force Wake On Lan
# ------------------------------------------------

ethtool -s eth0 wol g

ethtool -s eth1 wol g

# -------------------------------------------------
# Enable power-efficient ethernet
# -------------------------------------------------

# enable IEEE 802.3az (Energy Efficient Ethernet): Could be incompatible to LACP bonds!
for i in /sys/class/net/eth?; do dev=$(basename $i); [[ $(echo $(ethtool --show-eee $dev 2> /dev/null) | grep -c "Supported EEE link modes: 1") -eq 1 ]] && ethtool --set-eee $dev eee on; done

 

Thanks @dopeytree for all the info.
Sorry but one last question, in your go file, do you need to replace the question marks with the actual device?

I know that an asterix is a wildcard, but what does a question mark do?

 

Kind Regards and thanks in advance!

[mgutt]

45 minutes ago, rikdegraaff said:

I know that an asterix is a wildcard, but what does a question mark do?

It has a similar meaning. So no, you don't need to replace it.

[rikdegraaff]

1 minute ago, mgutt said:

It has a similar meaning. So no, you don't need to replace it.

 

Okay, thanks for the fast answer!

[kiwijunglist]

On 7/14/2023 at 4:02 AM, JaimeUriarte said:

Hello everyone.

I just finish building my first home media server, everything works fine. Now I'm trying to reduce power consumption now it idles at 33W with HDD spin down and 43 with disk spin up.

I just followed the suggestions in this post but while executing powertop I'm not getting any readings for CPU Pkg (HW).

 

My build is the following:

Intel i3 10th Gen 10100

Gigabyte H410M S2H V2 with F3 BIOS

Cooler Master MWE 80+ Bronze V2 650W

WD_Blue_SN570_1TB nvme cache

WDC WD101EDBZ-11B1DA0 10TB for Array

Patriot Memory 32GB Boot Device

The case is a BitFenix Nova MESH M-ATX ARGB with an extra Be Quiet! 80mm Pure Wings 2 PWM for the drives area

 

So what I am missing to make powertop able to improve my power consumption?

*EDIT:

It seems I was missing an option in the BIOS to allow the OS to make power saving changes, so now its achieving pc8 state idling 21W and stable 34W with dockers and HDD spinning. Lets see if I can reduce it further.

what was the option in bios?  I've been unable to get a good powerstate in Pkg(hw). I'm sure i've added all the options i need in bios.

[AmokK]

Same here, been using powertop for a month, all good in tunables, I think I checked all possible options in bios, but I still get 0% in every Pkg(Hw) values.