Reduce power consumption with powertop


Recommended Posts

Hi all, 

 

Very interesting thread. Looking for a new power supply to replace my TX650 Bronze from 2013. However, the Corsair RM550X is nearly out of stock in all reliable manufacturers in my country and extremely expensive +150€. On the first post it lists the BeQuiet Pure Power 11 FM 550W as an alternative (priced at around 89€ so more reasonable). However, it seems that the new version PURE POWER 12 M 550W has come out, although I am not able to find any review that tests low loads. 

 

So my question is, what are the alternatives to the Corsair RM550X? Still the Be Quiet as the first post says?

 

Thanks

Edited by SP67
Link to comment

Hi All

 

I wonder if people can offer me some advice.
I have a unraid server (been running for years) and comsumes around 50w , 24/7
Would love to lower the number

This is mainly a plex server, so doesnt do much, performance is not a main consideration.
I currently spin down the disks. (30min) which doesnt appear to make any diff.

The server has 4x8tb drives. (Tosibiha)

Server spec (this is a soldered on CPU)
 

Model: Custom

M/B: ASRock J4105-ITX Version - s/n: M80-B3004200454

BIOS: American Megatrends Inc. Version P1.30. Dated: 05/04/2018

CPU: Intel® Celeron® J4105 CPU @ 1.50GHz

HVM: Enabled

IOMMU: Enabled

Cache: 224 KiB, 4096 KiB

Memory: 8 GiB DDR2 (max. installable capacity 256 GiB)

Network: bond0: fault-tolerance (active-backup), mtu 1500
 eth0: 1000 Mbps, full duplex, mtu 1500
 eth1: interface down
 

I saw about auto tune, but is that a good idea on this server?  It is a live unit.

Is 50w decent, or can i get it into some sort of sleep state, as its not used a huge amount.

Uses----
Plex
File storage

 


Powertop Reads:
image.thumb.png.6eb70aec4eaff1c83f187924760323f5.png

 

image.thumb.png.cb88e909d12cca40c1500f969524a27c.png

 

 

Really appreciate any advice anyone can give! :)

Thabks

Link to comment
2 hours ago, virtualP said:

I currently spin down the disks. (30min) which doesnt appear to make any diff.

Doesn't make sense. A spinning disk consumes usually 6 to 8W and a non spinning disk 0.5 to 1.5W. This difference should be noticeable on your power meter.

 

2 hours ago, virtualP said:

I saw about auto tune, but is that a good idea on this server? ... Powertop Reads:

Did you even read the first post? As you are having "bad" entries it does not seem so.

 

2 hours ago, virtualP said:

as its not used a huge amount.

Aha, and why is your CPU having a load of 50%? Check your "C0 active" values. This is far away of an "not used" server. Execute "top" and check what is causing this huge load.

 

Link to comment
14 minutes ago, mgutt said:

Doesn't make sense. A spinning disk consumes usually 6 to 8W and a non spinning disk 0.5 to 1.5W. This difference should be noticeable on your power meter.

 

Did you even read the first post? As you are having "bad" entries it does not seem so.

 

Aha, and why is your CPU having a load of 50%? Check your "C0 active" values. This is far away of an "not used" server. Execute "top" and check what is causing this huge load.

 

image.png.7818342e0ad5b29b6d4ce6c4dbb67a1d.png

 

It wasdoing a partity check.  Ill let it calm and wait for the spins to spin down

 

For a server of this type with disks spun down, what sort of wattage would be decent?

 

I did read first post , but it warned about using auto tune in certain cases.....  Sorry im n00b!

Edited by virtualP
Link to comment
4 minutes ago, pkoci said:

I have asrock j4105 mb with 2 sata ssd and 3 HDDs Ironwolf 4TBs+ m.2 to sata expansion card and seasonic gold 500w ATX PSU, 16GB RAM. The system idles at 16w and never exceeds 40w.

Thanks

 

Im currently at around 35w - the disks should be spun down by now.
not sure what fans are in it, nor the PSU, so assuming i might have more than you, it seems about right????

Link to comment
2 minutes ago, virtualP said:

m currently at around 35w - the disks should be spun down by now.
not sure what fans are in it, nor the PSU, so assuming i might have more than you, it seems about right????

Are you serious? Sorry, but it seems you are giving a shit on the topic itself. I mean running a parity check and then asking about power consumption and now you even don't know which hardware you are using AND you don't know if your disks are spun down or not 🤨

Link to comment

Greetings,

 

first of all thanks for all the tips, I already managed to lower power consumption of my rig a bit. I have a couple of questions though, maybe someone will be able to help me.

 

I built a new PC to host unraid on, these are the specs:

MB: Asus PRIME H610M-R D4

CPU: i3-12100

HDD: 2x Seagate Ironwolf 4TB

SSD: Samsung 850 EVO 500GB (SATA)

PSU: Corsair RM550x (2018 revision sadly)

 

When running unRAID and 2 containers (Photoprism and nextcloud) I measured about 20W power usage. CPU is reaching C8 I believe. Is this good result? I thought I could run a little lower. I have disks set up to spin down but they never do that, I don't know why.

 

Main issue is when I start VM with home assistant and when I attach Sonoff Zigbee 3.0 USB stick. Power usage is now 30W and CPU never reaches any C state. I tried to apply all the tweaks but nothing seems to be helping. I'm attaching a couple of screenshots.

 

 

powertop_device_stats.png

aspm_status.png

powertop_idle_stats.png

powertop_tunables.png

Link to comment

First of all, many thanks for all the tips gathered throughout this topic and points given by @Mgutt! 

 

My setup is  hovering between 27 and 32 watt when disks are idling. Which I think is a great feat on it's own, but with my continuous quest to bring down power consumption, I would like to ask some pointers whether more is possible with current usage. I have applied all recommended bios settings tweaks and Powertop tuning as per instruction.

 

My setup is:

MB: Gigabyte C246M-WU4

CPU: Intel® Xeon® E-2288G

HDD: 6x WD ULTRASTAR 18TB

SSD: Samsung 970 EVO 1TB (NVME)

        Samsung 970 EVO Plus 2TB (NVME)

PSU: Corsair RM550x (2021)

RAM: 2x KINGSTON 2666 MHz ECC 32 GB

 

C states reach C9, but I always seems to have CPU's stuck in C2.

 

image.png.f5aea9fb47f913de6b0772bdf6cb282f.png

 

I am not running VM's, but believe Home-Assistant docker is to blame. CPU usage seems to continuously around 1 - 2%.

Is anyone experiencing the same and having ideas whether this could be lowered by e.g. tweaking Home-Assistant ?

 

image.thumb.png.c2c28485738a9b50679e3cc5c69c8cdf.png

 

Link to comment
57 minutes ago, bmdegraaf said:

Is anyone experiencing the same and having ideas whether this could be lowered by e.g. tweaking Home-Assistant ?

I checked my server and I don't see a process "hass", although I'm using Home Assistant, too. Please press "c". It will show the full command then. Or execute:

 

pgrep -af hass

 

Your shfs load is high as well. It seems you didn't change docker containers to the path /mnt/cache/appdata. Anytime something writes to /mnt/cache/user, it adds a little bit shfs process overhead. Some German information about this tweak:

https://forums.unraid.net/topic/99393-häufig-gestellte-fragen/#comment-1223263

 

Is fail2ban present all the time, too?

Link to comment
4 hours ago, berben said:

Main issue is when I start VM with home assistant and when I attach Sonoff Zigbee 3.0 USB stick. Power usage is now 30W and CPU never reaches any C state

This is pretty usual. VMs produce load. Load stops the CPU from going to sleep. But you could try to give the vm only the very first CPU core #0 and then check again. Another variant could be to isolate the last cpu core and use this exclusively for HA. Then reduce the maximum frequency for this single core:

 

Lets say your last core is #7 as you are having an 8-core CPU. Then do this to limit this core to 2 Ghz:

 

echo "2000000" > /sys/devices/system/cpu/cpu7/cpufreq/scaling_max_freq

 

Obtain all frequencies of all cores:

 

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq

 

Note: This is reset after each reboot. So you have to add it to a script or the /boot/config/go File to make it permanent.

 

Obtain minimum and maximum of your CPU:

 

cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_*_freq

 

 

Link to comment
5 hours ago, mgutt said:

This is pretty usual. VMs produce load. Load stops the CPU from going to sleep. But you could try to give the vm only the very first CPU core #0 and then check again. Another variant could be to isolate the last cpu core and use this exclusively for HA. Then reduce the maximum frequency for this single core:

 

Lets say your last core is #7 as you are having an 8-core CPU. Then do this to limit this core to 2 Ghz:

 

echo "2000000" > /sys/devices/system/cpu/cpu7/cpufreq/scaling_max_freq

 

Obtain all frequencies of all cores:

 

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq

 

Note: This is reset after each reboot. So you have to add it to a script or the /boot/config/go File to make it permanent.

 

Obtain minimum and maximum of your CPU:

 

cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_*_freq

 

 

Thank you for tips. Unfortunately it didn't help at all, I tried both approaches - CPU won't reach any C state. I tried replacing Home Assistant VM with docker container. I set up Home Assistant, Mosquito and Zigbee2MQTT containers and the story is almost the same. The is no CPU overhead that VM had but only C2 is reachable and there is no difference in power usage (maybe ~1W). I think Sonoff Zigbee USB stick is the culprit here. I can set up multiple different containers but only Zigbee2MQTT (it uses Sonoff stick directly) blocks CPU from entering better power states.

Link to comment
On 3/12/2023 at 10:40 PM, mgutt said:

Test different USB Ports and Ethernet ports if your board has multiple of them. For example the C246M-WU4 reaches only C9 if a specific ethernet and usb port is used:

https://forums.unraid.net/topic/119197-gigabyte-c246m-wu4-komme-nicht-über-c2-state-hinaus/page/4/#comment-1232565

 

It didn't help at all. I tried both USB 2.0 and 3.0 ports. This MB has only one ethernet port. Do you have any experience with this Sonoff USB stick? I think I saw this device on one of your screenshots.

Link to comment

CPU: 9900k underclocked to 2500 MHz

Motherboard: E3C246D4U

RAM: Corsair 4*16 GB

Cache: 2x Samsung 500 GB M.2 NVME, one in a PCI card

Array: 2 exos 18 TB and 2 WD purple 10 TB

Additional drive: WD purple 10 TB passed through

 

Always on dockers include plex and wyze bridge, there is also an always on windows VM running Blue Iris.

 

With disks spun down (except for the one passed through to the VM) I am at 40-45 W at the plug (with a 1500 UPS in series).

 

I have spent time using powertop and trying to get ASPM working, but these don't seem to get me past C3.

 

more details here: ASPM for E3C246D4U

 

I guess my power consumption is not bad for what the system is doing.

Edited by tdatta
added info
Link to comment
1 hour ago, tdatta said:

I am at 40-45 W at the plug (with a 1500 UPS in series).

And without? Which power supply do you use? Fans?

 

1 hour ago, tdatta said:

there is also an always on windows VM running Blue Iris.

With a VM it is nearly impossible to reach the lowest states. What is your CPU load? (Execute "uptime" in terminal)

Link to comment
19 hours ago, mgutt said:

And without? Which power supply do you use? Fans?

Readings without UPS:

40-50W standard idle, (plex and wyze and windows VM with passthrough HDD and Blue Iris)

38-39W with dockers off but BI VM running (seems strange that docker would use so much power)

24-27W with docker and VM off. I guess this is "true" idle.

 

Power supply is an EVGA from ~2018, probably 550-650 watts, I need to check to be sure.

Fans are two Noctua NF-A8 @ ~1700 RPM (each max 1W), and one Noctua industrial NF-12 iPPC @ ~1500 RPM (max 3.6W)

 

19 hours ago, mgutt said:

With a VM it is nearly impossible to reach the lowest states. What is your CPU load? (Execute "uptime" in terminal)

 

output from "uptime":

20:34:00 up 5:32 , 0 users, load average: 3.20 3.22, 3.00 

You can see in my previous linked post what the c states are like with and without the VM.

With the VM off it can spend close to ~50% in package C3

 

FYI- running powertop auto-tune saves about 10W at idle (the numbers above are after auto-tune)

 

Edited by tdatta
add info
  • Upvote 1
Link to comment

Hi,

 

First of all thanks to @mgutt and for everybody sharing their different feedback.

It helped me a lot to lower the enegergy consumption of my server.

I wanted to share my own feedback and maybe get some advices to further reduce my idle power consumption.

 

My unraid server is made of :

  • CPU : 5800X with 4C/8T isolated to the gaming vm
  • RAM : 4*16
  • MTB : Asus TUF Gaming x570-PRO
  • 7 3.5" HDD for data and parity : 4 * Seagate IronWolf 8 TB + 2 * Seagate SkyHawk 10 TB + 1 * Toshiba N300 10 TB
  • 1 2.5" SSD for cache : MX300 1TB
  • 1 NVME SSD (VM passthrough)
  • GPU nvidia 3090 TI (VM passthrough), no screen attached I use a dummy HDMI plug
  • GPU nvidia P620 for Plex transcoding
  • PSU RM1000X
  • 2.5GB PCIE ethernet card (VM passthrough)
  • 8 FANs (it's kind of hot in the summer)

Running unraid 6.11.5.

 

Of course this server was built before the energy crisis and I did not pick the components for their energy efficiency.

But when we received the first energy bill this year, well, I had to do something before my wife unplugged the server.

 

I measure the power consumption with a Shelly Plug S, there's a slight overhead due to a router.

If I look back in the past in the Shelly Cloud UI, before I started to search for power optimization, I was idling at 230 Wh (with 5 HDD instead of 7).

 

Here's what I did so far :

  • powertop --auto-tune at startup
  • Make it so my vm shut down state when idling --> done with a small tool installed in the VM
  • User script to wake up my VM from paused/pmsuspended/shutdown state with a WOL magic packet that trigger virsh commands
  • Put in place a script to change CPU governor (powersave/performance) depending of my vm state
  • Enabling CPPC/amd-pstate to reach 550 Mhz at idle
  • Change the P620 gpu power state to P8 with a script if nothing is using it
  • Look for docker containers that may access the disks and spin them up when it's not needed

 

Now if all my 7 disks are spinned down and my Gaming VM is shut down I'm currently at ~ 93 W.

My cache drive never goes to sleep but with 30 running docker I suppose this has to be expected.

It's still a lot compared to most of you but it's a good start.

 

Here's what I still need to do :

  • Disable onboard RGB led controller, audio, wifi and bluetooth as I don't use them anyway as my server is my garage
  • Dive in the BIOS manual of my MTB for ASPM/CSTATE/Turbo/Whatever and check if everything is OK in the BIOS
  • (WIP) Try to put the 3090 TI to P8 once the VM has been shutdown (https://forums.unraid.net/topic/118039-strom-sparen-ans%C3%A4tze/), but I need it to be more dynamic to work with my WOL script and play nice with the user script I use to put my P620 to P8.

 

Regarding that last point I found this comment :

 

As I have 2 gpus I could adapt it to target only my 3090 TI.

 

 

I also have some hope that unraid 6.12.x with kernel 6.1 will bring some improvements for AMD CPU users.

Edited by hot22shot
Link to comment

Hi all,

i switched from AsRock H110M-ITX , i5-7400 to a AsRock H570M-ITX , i5-11400 (Rocket Lake), 16GB RAM.

When using powertop, only ACPI states show up (C1 to C3).

Idle consumption is also quite high with 30W compared to 19W with my old setup.

I have installed the INTEL Plugin and used the CLI-commands.

Further, deepl sleep and the c-states are activated in the BIOS.

Any ideas, what might be the reason, why i can't see the C1-C10 states?

Thanks for the help!!

 

Link to comment
2 minutes ago, Aldamir said:

What's the --auto-tune command for then?

It enables the power saving setting on all pcie slots, USB and SATA ports. This is a one-time setting (which gets lost on every reboot). It is the same as the custom commands in the first post. So it's nothing like a background process watching and reducing power consumption if you thought about that.

 

4 minutes ago, Aldamir said:

Right now, i can reach no more than C3... i have no PCIe Device but 2 M.2 NVMe SSDs (SN570).

Do you have any idea?

The bios settings are looking good to me. Did you place the NVMe into the CPU M.2 slot? Try a Chipset M.2 slot then.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.