Reduce power consumption with powertop

[dopeytree]

I went into bio last night and changed all the tweaker cpu settings to (enabled rather than auto). I didn't touch the settings where you can completely control the cpu just enabled all the c states and disabled tubo boost. It now shows its reaching c3.

 

You can also force the gpu to idle in p8 rather than p0 by typeing in terminal 'nvidia-smi --persistence-mode=1' This has brought the gpu from 25w down to 12w.

 

To make it permanent you could make a little script that launches on array start.

 

#!/bin/bash

#nvidia-smi --persistence-mode=1
nvidia-persistenced

fuser -v /dev/nvidia*

 

 

Edited August 30, 2022 by dopeytree

[Aldamir]

20 hours ago, dopeytree said:

Runnng unraid 6.10.3

Can run powertop but not the autotune it gives the following:

 

powertop --auto-tune
powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failedCannot load from file /var/cache/powertop/saved_results.powertop
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
Leaving PowerTOP

 

Any tips to fix? is it a permissions problem?

 

I have a similar output.. 
Help is highly appreciated! 

[dopeytree]

@Aldamir I couldnt get the autotune thing to work so I did it manually...

Which motherboard have you got? 

 

On mine you press f12 as it boots up on the first screen. 

This brings up the boot loader. 

At the bottom is an option for setup (bios menu)

Loads bios.

Choose advanced menu.

Tweaker.

Change most things from auto to enabled.

Turn turbo boost to disabled.

Anything that has more settings than auto/disabled/enabled leave on default settings.

etc.

 

Other setting is under sata settings in the bios menu. Choose enable devslip for all ports.

 

Then save and exit.

PC reboots.

Load unraid.

Login and run pftop again and you should see most of the settings are now good rather than bad under the 'tuning' heading. 

Edited August 30, 2022 by dopeytree

[BusterDogg]

 

Newest update to the Tips & Tricks plugin is quite interesting. I would argue if the power savings are minimal. 

[HubertusPrime]

On 8/21/2022 at 1:23 PM, HubertusPrime said:

Thank you @mgutt for the great work!

 

I gave this a shot after a year (back in the days it just didn't work to get the CPU off turbo mode) and it's insane. Maybe my readings are wrong with powertop. But if not... wow

 

Before "tuning" this my power consumption with some steam servers runing, plex and home security runing was around 25-45Watts. Setup is this:

 

CPU: i5 - 10600k

Mainboard: Gigabyte Z490i Aorus Ultra

RAM: 64GB 3200mhz

NVME: 1 x Evo Plus 1TB

HDD: 2x8TB 2x4TB WD Red

Cooling: 4x Noctua 120mm

 

It came down to 6-12Watts! How is this possible? Only because entering C-State and avoiding turbo mode?

 

My CPU never clocked under 4.1Ghz before tuning. Now it's runing at 1.2Ghz until load is applied.

 

This is so amazing. Even if it's not turbo boosting anymore. It's still enough power for all my daily tasks. Even the Fans stopped. So it's a passive cooled server in the living room right now

 

And with the raising costs for power in europe, i really appreciate this money saving per Year.

 

Updated my Bios to F21 version because of a problem with my 10G intel NIC. Now there is no "ondemand" or "conservative" option anymore in govenor settings.

CPU runs way hotter than before (stuck to 4.5Ghz) even while set to powersafe mode and power consumption in idle with Array up is about 90Watts!

 

This is like 70Watts more than before. 

Any ideas to troubleshoot this?

On 8/29/2022 at 5:17 PM, dopeytree said:

Runnng unraid 6.10.3

Can run powertop but not the autotune it gives the following:

 

powertop --auto-tune
powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failedCannot load from file /var/cache/powertop/saved_results.powertop
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
Leaving PowerTOP

 

Any tips to fix? is it a permissions problem?

 

and this problem came up after updating BIOS too.... crazy

[dopeytree]

Are you actually changing the settings in the bios? There’s a menu with lots of options?

[Fredrick]

Hey guys,

Looking to get some insight. I've got a very power hungry setup that I need to tweak, and obviously want to spend my money on the most efficient changes. 

 

In total my rack pulls about 350-370W at the wall. 

 

Network gear and PoE-cameras: 65W. This is a UDM Pro SE, Unifi Switch 24 Pro, 2 APs and 4 cameras via PoE. I don't think theres much to be done with this

 

Server: 90-130W

• Ryzen 7 3800X
• 32Gb RAM
• LSI SAS 9202-16E (or something similar)
• Intel 4xgbit NIC
• PCI USB-controller (previously used to pass through to VM)
•  

DAS: 180W

• Array of 12 drives

 

My plan is:

1. Buy new chassis and more efficient PSU to ditch the DAS and move drives into same chassis as server
2. Drop the USB-controller and NIC-controller. I can manage without redundant NIC
3. Investige what makes my disks spin up. Currently I run about 24 dockers so I think pinpointing the culprits could be a pain
4. Turn off some dockers for most of the day . I.e. I don't need Plex and arr's during the night and so on.
5. Actually swapping the Ryzen I think is too expensive (?) to make sense in an effort to reduce my power bill

 

Anything I've missed or should do different?

 

[mgutt]

1 hour ago, Fredrick said:

Network gear and PoE-cameras: 65W. This is a UDM Pro SE, Unifi Switch 24 Pro, 2 APs and 4 cameras via PoE. I don't think theres much to be done with this

Ubiquiti switches and routers are very inefficient products as of my experience. I'm using their APs only (unifi docker is stopped most of the time).

 

1 hour ago, Fredrick said:

Server: 90-130W

• Ryzen 7 3800X
• 32Gb RAM
• LSI SAS 9202-16E (or something similar)
• Intel 4xgbit NIC
• PCI USB-controller (previously used to pass through to VM)
•  

DAS: 180W

• Array of 12 drives

Any Intel Setup up to 10th Gen + 6-Port SATA card, would be much more efficient.

 

I'm using the C246N-WU2 combined with a Xeon and it consumes less then 20W (with 8 HDDs, 10G Ethernet, 64GB RAM, ...).

 

1 hour ago, Fredrick said:

Buy new chassis and more efficient PSU to ditch the DAS and move drives into same chassis as server

You save even more energy If you use a case without backplane.

 

A more efficient power PSU does not help much as long your consumption is so high (maybe 5W if you upgrade from gold to platinum).

 

1 hour ago, Fredrick said:

Investige what makes my disks spin up. Currently I run about 24 dockers so I think pinpointing the culprits could be a pain

Should be an easy task by checking the share location (shares > show content > view LOCATION column). VM Disks, libvirt, docker.img and appdata should be completely on cache. If not: Set cache to prefer, disable docker and VM and start the mover.

 

1 hour ago, Fredrick said:

Turn off some dockers for most of the day

Should be unnecessary. Containers like plex idle at 0% most of the time. But a gaming server can use more CPU of course. Check the advanced docker view!

 

1 hour ago, Fredrick said:

Actually swapping the Ryzen I think is too expensive (?) to make sense in an effort to reduce my power bill

Depends on your energy costs. In Germany it would be an easy decision. ^^

 

PS Installing a more recent Ryzen model does not save a lot (or even nothing).

 

[Fredrick]

5 hours ago, mgutt said:

Ubiquiti switches and routers are very inefficient products as of my experience. I'm using their APs only (unifi docker is stopped most of the time).

 

Yea but I'm pretty much "all in" at this point so I'm not gonna turn. Just gonna remove the drive from UDM which I'm not using, that could possible save a penny.

 

5 hours ago, mgutt said:

You save even more energy If you use a case without backplane.

 

Yes that's what I'm gonna get I'm looking at Inter-Tech 4F28

 

More efficient PSU is mostly cause I'm gonna need a new one when moving all drives to a single one. Currently most drives are fed by the dual PSU DAS. 

 

Any tip on a good SATA-card? I'd still need some room for expansion

 

5 hours ago, mgutt said:

Should be an easy task by checking the share location (shares > show content > view LOCATION column). VM Disks, libvirt, docker.img and appdata should be completely on cache. If not: Set cache to prefer, disable docker and VM and start the mover.

 

I didn't quite understand this. I know what I have on my different shares, but I don't know which docker keeps accessing and spinning up disks. Thats also the idea behind shutting dockers down tbh, but I see most of them are ery inactive like you say. And I have no VMs, docker and appdata completely on cache

 

5 hours ago, mgutt said:

Depends on your energy costs. In Germany it would be an easy decision. ^^

 

Yea but getting used parts is kind of a pain. setting up an intel machine would probably set me back 300€. Even with high energy prices we do get a lot subsidised in Norway atm. Saving 20-30W is only 50€ or so per year.

 

Thank you for answering!

Edited September 3, 2022 by Fredrick

[mgutt]

1 hour ago, Fredrick said:

but I don't know which docker keeps accessing and spinning up disks.

The reason why docker spins up your disk is simple: Your docker reads/writes a file which is located on a disk. ^^

 

Follow the steps (shares tab > open Content....). unRAID shows if files are located on cache AND array:

 

A share like appdata should show in the column "LOCATION" only "cache" (note: do not use "only" as your cache setting, instead use "prefer" as the mover does not move files if you choose "only" and some files are left on the array).

Another reason could be a docker like Plex, which monitors the content of shares, which are located on your array. That is the reason why I disabled auto-indexing in Plex (so it does not check for new movie files) and I used an additional tweak by passing every disk manually:

 

But this tweak is usually only needed if you notice that multiple disks spin up while starting only one movie (I had this).

[dopeytree]

Has anyone had a problem with DEVSLP enabled in BIOS?

My disk1 got disabled by unraid and it now wants to rebuild it.

There's doesn't seem to be a way to re-enable a drive once Unraid thinks its bad.

Even if you can mount and view the drive outside of the array...

Looking at the logs it was a power/link issue which may have tripped it.

 

[mgutt]

2 hours ago, dopeytree said:

DEVSLP

Simply do not connect 3.3V. Easiest fix: Use Molex to PowerSATA.

[dopeytree]

Am using a nas case with sata backplane. it uses molex split to 4x sata drives.

Turned DEVSLP off anyway and seems fixed apart from the 13 hours to rebuild the 'disabled' drive 😆

[mgutt]

2 hours ago, dopeytree said:

it uses molex split to 4x sata drives.

The DEVSLP was never active:

https://en.m.wikipedia.org/wiki/DevSlp

[TOMillr]

Just a quick thought: Wouldn't it make sense to ship Unraid with powertop and include an optional setting that runs -- auto-tune at startup? 

[dopeytree]

Imagine its because traditionally servers are about being powerful as possible rather than energy efficiency.

 

I ran into problems with using sata powersaving functions.

Everything else is good advice tho mostly done through BIOS.

The in tips & tweaks turn to powersaving profile - you can choose to enable or disable boost but unraid isn't very good at reducing cores.

Edited October 9, 2022 by dopeytree

[mgutt]

2 hours ago, TOMillr said:

Wouldn't it make sense to ship Unraid with powertop and include an optional setting that runs -- auto-tune at startup? 

 

No, because many users have problems with specific commands which are set through powertop --auto-tune. Some of them need to be set manually to avoid problems. So this is only something for advanced users.

[ph0b0s101]

Hello together,

 

I tried to get better idle energy usage by following you guide. Installed powertop and restarted the server, see that the cstates are not used. Tried the auto tune command and got the following:

 

powertop --auto-tune
powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
modprobe cpufreq_stats failed
Loaded 0 prior measurements
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
RAPL device for cpu 0
RAPL device for cpu 0
Devfreq not enabled
glob returned GLOB_ABORTED
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only 
Leaving PowerTOP

One User also mentioned that powertop was removed by Tips & Tweaks plugin because of more problems then rewards. So is it still a thing? Furthermore, how can I get the config editor in Version 6.11.1? Seems there is no app to install and I cannot find it in the tools section of the webgui.

 

Any tipps?

[mgutt]

4 hours ago, ph0b0s101 said:

got the following:

All messages can be ignored.

 

4 hours ago, ph0b0s101 said:

So is it still a thing?

--auto-tune can have side effects like disabled usb devices or disconnected HDDs. Finally it's more for advanced users so T&T dev decided to remove it.

 

4 hours ago, ph0b0s101 said:

can I get the config editor in Version 6.11.1?

Install file explorer plugin. It includes an editor (that's why the editor plugin has been removed).

 

 

[mgutt]

I added a new paragraph to the first post:

Quote

Power supply

An unnecessary high-power and/or inefficient power supply can raise the power consumption by 5 to 15W. Choose wisely:

- as low power (watt) as possible

- as efficient as possible (like Titanium grade)

- avoid dual power supply setups (server grade hardware is not build for efficiency)

 

If you plan to build a setup with extremely low power consumption, consider these power supplies:

- PicoPSU (up to 200 watts, but is limited to ~4 SSDs as it has low 5V output)

- Corsair RM550x (2021) (this looses only 1 watt more than a PicoPSU at 10W to 20W load!)

- BeQuiet Pure Power 11 FM 550W (similar to the Corsair, but 2 watts more than a PicoPSU, but cheaper than the Corsair)

- Seasonic 600W/650W Titanium (any model, as efficient as a PicoPSU, but expensive)

 

Note: A single HDD consumes up to 30W on spin up)

 

[ph0b0s101]

On 10/14/2022 at 7:00 PM, mgutt said:

Install file explorer plugin. It includes an editor (that's why the editor plugin has been removed).

Hello @mgutt,

thank you for your reply. So far so good, but I cannot edit the go file because I did not find the file explorer plugin / app. I have installed the file manager plugin von dynamic but there is no option to directly edit files. So did I oversee something? Thank you for updating the start post with some power supply recommandations. This is great and I put them on my watchlist. Cheers

[fwiler]

Has anyone made an sfx power supply that is efficient at low wattage?  450w or under.

[mgutt]

On 10/18/2022 at 8:51 PM, fwiler said:

Has anyone made an sfx power supply that is efficient at low wattage? 

 

Sadly this website is the only one which tests power supplies at 5W, 10W, etc:

https://www.tweakpc.de/hardware/tests/netzteile/corsair_rmx_2021_550_750_1000/s02.php

 

The low load "kings" were all released since 2021. Through this German website it is possible to filter for recent SFX power supplies: 

https://geizhals.de/?cat=gehps&xf=3314_2021~4174_SFX&sort=p&hloc=at&hloc=de&v=e

 

Maybe the Seasonic SPX 650W is worth a try?!

[Christopher W.]

On 3/20/2022 at 9:05 PM, BeersTeddy said:

 

This is just brilliant.

Power usage went down by 50% on pretty fresh UNRAID. This is some serious result!!!

 

Currently, only 13W in IDLE (HDD's in standby) used to be 26W, usually closer to 30 as it was fluctuating.

For comparison:

Same PC, Windows 10, only one SSD connected, "power saving mode" and yet was going down to 19W only after a very long while.

 

Full spec:

HPE Proiliant ML10 gen9

Xeon E3-1225v5

4x 8GB DDR4

1x nvme connected to PCIe port

2x 3.5" parity + array

2x 3.5" unassigned

1x 2.5" hdd

 

Right now RAM using more power than CPU

 

 

Could you share your bios setting? I only get to c3

[Erka53]

On 8/24/2022 at 9:25 PM, HubertusPrime said:

*UPDATE*
 

Since i did all this power saving settings, my server looses network connection. Even when in use.

when this happens it‘s not even listed in my router. Is the NIC going to sleep? 
i edited the go file back to default, but it keeps happening. Lost a whole day of my life trying to solve this. 
 

any ideas or experiences with this behavior

Is there any way how you solved this?
I'm experiencing the same right now.
After experiencing this i turned off the plugin from within Nerdtools but it still keeps happening.