Reduce power consumption with powertop


Recommended Posts

I took out the disk from where I moved it, instead of reading the serial from Unifi (where I moved the disk into and read serial from the UI... sorry)

 

WD500AAKS

WD Caviar SE16

 

MDL: WD5000AAKS - 00A7B0

 

Bit different identifiers then the previous I posted....

Edited by Chopsting
Link to comment

Hi,

 

i have finally got it working.

 

Asrock C236 WSI

Xeon E3-1275 V6

32 GB ECC

Intel® X710 DUAL 10 GB SFP+

2x 2,5" Samsung 850 2 TB SSD

4x 2,5" Samsung 860 EVO 250 GB SSD

2x 20 TB Western Digital Ultrastar DC HC560

 

IDLE C7: OK, Mainboard cant go deeper 26,3 Watt

Parity and Full Smart Tests of all SSDs 42,2 Watt

 

Im Happy :-).

 

Without HDDs 14,7 Watt

 

Edited by cuddles
Link to comment

Ok thanks for clarification.

 

intel i9 11-900t

 

In intel chips we seem to have general boost all cores (performance governer) which keeps all cores (8c 8t) cpu at 3.6ghz with turbo boosts to 4.5ghz or Keep as low as possible (power saving) which actually changes the cores from 0.8 - 1.1ghz 

 

Is there no middle ground or a better way for unraid to distribute tasks to the cpu. Or something that says ok prefer to only boost these 4/6 cores. In windows theres alot of controlling the cpu happening but my vague understanding is unraid leaves to the CPU to decide. 

 

I can manually do it in bios under volting the CPU getting it to hover around 1.5ghz with boosting up to 2.5ghz. More cores also default to the lower 0.8ghz this way if not in use. This is on the performance governor. 

 

There should be 37 powerstates to use with turbo at 90% 'when needed'. 

 

cat /sys/devices/system/cpu/intel_pstate/num_pstates

37

cat /sys/devices/system/cpu/intel_pstate/max_perf_pct 

100

cat /sys/devices/system/cpu/intel_pstate/min_perf_pct 

18

cat /sys/devices/system/cpu/intel_pstate/no_turbo

0

cat /sys/devices/system/cpu/intel_pstate/status

active

cat /sys/devices/system/cpu/intel_pstate/turbo_pct

90

cat /sys/devices/system/cpu/intel_pstate/hwp_dynamic_boost

0

 

Powertop shows this: note the readout of CPU frequencies seem to differ between powertop and running: watch -n3 "cpufreq-info | grep 'current CPU'

 

821701811_Screenshot2023-01-11at13_17_47.thumb.png.78682548976bdf633ddb49c72b571d5b.png

 

My system seems to be using C1_ACPI - C3_ACPI
       

873268721_Screenshot2023-01-11at13_17_20.thumb.png.1cd3cbcafe639d125f63ae8af092a1a5.png

 

Edited by dopeytree
Link to comment

Hi,

I'm building a new server to downgrade my power hungry Intel Xeon D-1528...

I've already bought a Fujitsu D3417-B motherboard, I'm planning to use it with an Intel i5 6400 (that I already have) or an i3 7100 or something similar.

I've seen that the most power efficient 10Gb NICs are the Intel X550 or X710 based.  

I found this card below 100€: 94Y5200 IBM Intel X710 Quad-Ports SFP+ 10Gbps PCI Express 3.0x8 Gigabit Ethernet Network Adapter (servers4less.com) 

Do you think it'll be ok? I'll use only 1 SFP+ port, can I disable the other 3?

 

Regarding power supply I was thinking about an RM550X. How do I recognize a 2021 model from the older one?

Thank you very much!

Simone

Link to comment
16 hours ago, mgutt said:

No experience regarding this. 🤷

Thank you, I didn't see that the card I've found was an ML2 Lenovo card 🤦‍♂️ Do tou have any advice for which card to use?

 

P.S. Does anyone use a card based on this chip?

57810S-PB00.fm (cisco.com)

I've found this card which is pretty cheap:

10Gb SFP+ PCI-E Network Card NIC, with Broadcom BCM57810S Chip, Dual SFP+ Port, PCI Express X8, Support Windows Server /Linux/VMware : Amazon.it: Informatica

 

Thank you!

Edited by Simone Valmacco
Link to comment

Hi all, and especially Mgutt, nice topic! Thanks for the write up!

 

I've been doing my homework too and I'm trying to lower the power usage of my server.

 

First the hardware which was sort of chosen for decent low power a couple years back (Dutch tech site Tweakers.net has a nice 'low energy server' topic that helped me choose this build. Not perfect but not a bad basis I believe):

Processor: Intel Pentium Gold G5420

Motherboard: ASRock H370M-ITX/ac

Memory: Crucial Ballistix Sport LT BLS16G4D240FSB (1x 16GB)

SSD: Toshiba RC500 500GB

Harddrives: 3x 10TB. 2x WDC_WD100EMAZ-00WJTA0_1EGLJKWZ 1x WDC_WD101EDBZ-11B1DA0_VCJDR1NP (all three shucked from a WD Elements desktop unit)

Powersupply: Seasonic G-series 360 watt

USB stick: Samsung FIT Plus 32GB

Cooler: Scythe Big Shuriken 2 Rev. B

Case: Fractal Design Node 304

 

HDD's spin down every 30 min

Running completely headless unless I'm twaeking bios

 

Running Plex, SABNZB, Radarr, Sonarr, Nextcloud, Pihole, Mariadb, SWAG, DuckDNS, unifi controller as dockers. All seem very low in cpu usage nearly all are at 0.01% 

Home Assistant as VM.

 

I ran powertop --auto-tune and all tunables jump to good without any issue.

Server now draws 18-20W when all the drives are idle. Aprox 35-40W when active. Measured with a Shelly plug S.

 

C-state doesnt drop below C3, haven't fugured out why that is. Even with the array disabled, so not running anything at all for 30 min. C3 is all I get.

 

Tips and Tweaks Power Saving CPU govenor set to Power Save

 

All C-states enabled in the BIOS. 

Turbo Boost disabled.

 

Lot of pictures and screenshots here:

https://imgur.com/a/25ZBsGY

 

My main goal is to see if I can get an even lower idle consumption

Questions:

Does any of the above info tell me (or you in this case) why I'm not getting a higher C-state than C3? If so, can I fix that?

I'm guessing the SSD may be preventing it as I hardly have any other hardware. but how to prove that?

 

I never edited the go-file before. From what I understand from your first post in this topic I should be able to disable more hardware (usb ports, PCI-e slot etc) with the list of commands you posted in the first post.

How can I test them? Just in the console run: "for i in /sys/class/net/eth?; do ethtool -s  $(basename $i) wol d; done" for example? see if it gives any errors and do all of them one by one. if all is fine add the complete list to the go file (just copy paste?)?

 

I am also now doing the powertop --auto-tune by hand after every boot. that can go in the go-file too right?

 

Any tips or tweaks to my dockers to prevent them from spinning up disks? it doesnt happen very often but a couple times a day. I expect Nextcloud does that and Plex checks the library or something.

 

I hope all my info you need for some tips is there.... if not, let me know what else you need!

Thanks for any tips or comments!

Edited by Wabs
Link to comment
3 hours ago, Wabs said:

I'm guessing the SSD may be preventing it as I hardly have any other hardware. but how to prove that?

 

Disable Array Autostart, remove all drives and boot without them. Then add one after the other, unitl the C-State becomes bad.

 

3 hours ago, Wabs said:

Just in the console run

yes. You should leave the array stopped while testing and check your logs for new error entries.

 

3 hours ago, Wabs said:

should be able to disable more hardware

powertop --auto-tune executes the same commands. So no. The advantage of the separate commands is that you can remove problematic commands, which are maybe incompatible to your hardware. And you wouldn't need to install powertop at all.

 

3 hours ago, Wabs said:

Any tips or tweaks to my dockers to prevent them from spinning up disks? it doesnt happen very often but a couple times a day

Search in the forums. I'm sure its already discussed.

 

 

Link to comment

Thanks for the reply @mgutt!

 

I was trying to find info in the different PowerTOP tabs to find the reason why I wasn't going above C3. Makes sense to just disconnect hardware and see what happens. Will do so later this morning. Great tip to help me find the cause.

 

Also thanks for explaining what and how the commands work. I did notice that both ethernet controllers have wake on LAN enabled, even after I run --auto-tune. I switched them manually in powertop. I'll try your command for that one too then.

 

I'll do a search for lowering the power consumption for the dockers I'm running. If I find anything useful I'll link it here.

 

/edit1: update to SSD firmware and BIOS made no noticable changes. still max C3

/edit2: pulled the SSD out, no change either, still max C3

/edit3: disconnected the hard drives, no change, still C3

 

I'm out of hardware to disconnect... I've only got RAM, a processor and the USB stick with unraid on it left. Can't figure out why it wont go beyond C state 3.

Edited by Wabs
Link to comment
16 hours ago, Simone Valmacco said:

Do you have any recommended SSD to use as cache drive? I was looking at Intel P3605 or P3700 PCI SSD or standard Sata SSD...

Have a look at the list of hardware on Hardwareluxx: https://docs.google.com/spreadsheets/d/1LHvT2fRp7I6Hf18LcSzsNnjp10VI-odvwZpQZKv_NCI/edit#gid=0 you can see very efficient systems there and see what SSD's they have used. maybe that helps.

 

On 1/20/2023 at 6:54 PM, mgutt said:

Maybe Ubuntu is worth a try? To exclude a problem inside of unRAID.

No change in Ubuntu. I now think that it could be my PSU. Thats quite old. Bought it in 2014 and the model exisits since 2012. I also read a couple of posts online on other forums that cheap and old PSU's prevent lower c-states. I'll see if I can find an affordable PSU that has a low power usage at very low load. I know the Corsair 550 2021 version would be a nice bet.

  • Like 1
Link to comment
4 hours ago, Simone Valmacco said:

in every case Sata is usually more efficient than NvMe right?

Not relevant. I wasn't able to measure any difference with a C246M-WU2. But multiple NVMe can reduce the C-States while multiple SATA SSDs do not. I think this depends on which Lanes are used (Chipset, which are already active or CPU, which maybe activate additional lanes).

 

Link to comment
On 1/20/2023 at 7:53 AM, Wabs said:

I'm out of hardware to disconnect... I've only got RAM, a processor and the USB stick with unraid on it left. Can't figure out why it wont go beyond C state 3.

I have the same motherboard as you (Asrock H370) and can reach C7.

 

Have you enabled all the ASPM settings in the BIOS? I also disabled the sound card and a few other settings in the BIOS.

 

My Hardware

Motherboard : ASRock H370M-ITX/ac
Processor: i5-8500T

Memory: 32GB (2x 16GB DDR4)

Disks:

    3x 16TG Seagate Exos.

    1 x 8TG Iron wolf,

    2x 6TG Iron wolf,

    2x 3TB WD Greens (currently in the process of clearing so I can remove)

    2x 500GB crucial SSD's

    1x Asus BW-16D1HT Bluray drive

    NVME 1TB Crucial P5 Plus 1TB M.2 PCIe Gen4 (should have got a 2TB :-)) in the m.2 slot

Array is connected via an Broadcom LSI 9207 SAS PCIe Card

PSU: Evga 500 W1, 80+ White 500W, Power Supply 100-W1-0500-K3
UPS: APC Smart-UPS 1000

I did the install one by one method last weekend and with everything installed except the SAS card and I was able to reach C7 states and was drawing 34W under idle (No HD's attached except the SSDs).

 

With the SAS card I could only reach C3

I also noted a maximum of 150w load on boot.

 

With everything installed:
Under a full parity sync I'm getting drawing 107W.

Note the 107W also includes:

    16 port Unifi managed switch

    A Unifi Controller Key

    Unifi Nano HD Wifi (via POE) + RSP pihole

They are all plugged in to the same UPS I'm using to measure.

 

I'm in the process of removing the SAS card and putting the NVME drive in its place.
Then putting a 4 port SATA pcie card into the M.2 slot (mainly because I already have the card)

Along with that I'm also of installing some power monitoring plugs so I can record the server separate from my other gear.

Arriving today:)

Edited by Spin-Master
  • Thanks 1
Link to comment

When I run:

powertop --quiet --auto-tune

I get this error:

powertop: /lib64/libncursesw.so.6: no version information available (required by powertop)
powertop: /lib64/libtinfo.so.6: no version information available (required by powertop)
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only
Cannot load from file /var/cache/powertop/saved_parameters.powertop
File will be loaded after taking minimum number of measurement(s) with battery only

 

My specs:

Intel® Core™ i7-8700K CPU @ 3.70GHz

ROG STRIX Z370-E GAMING

NVIDIA GeForce GTX 1080

32GB DDR4 RAM

Corsair TX850 80+ Bronze

6 120mm fans

2 80mm fans

LSI 9211-8i 6G SAS HBA

10 HDDS

2 NVME SSDs

 

Side question, does 90W seem normal with all drives spun down and with CPU at 2% and GPU in idle state of 10W?

Link to comment
2 hours ago, Kronos_ said:

Corsair TX850 80+ Bronze

Extremely bad. Consider a better and smaller power supply. If you stay with your dGPU then a Seasonic Titanium would be the best option.

 

2 hours ago, Kronos_ said:

LSI 9211-8i 6G SAS HBA

Replace against an ASM1166 6 port card and use onboard SATA ports.

 

2 hours ago, Kronos_ said:

6 120mm fans

2 80mm fans

Check their power consumption and think about reducing the amount of fans.

 

2 hours ago, Kronos_ said:

NVIDIA GeForce GTX 1080

Used in an VM or in unRAID? If by a VM: Do you set the GPU to standby if the VM is shutdown? German thread:

https://forums.unraid.net/topic/118039-strom-sparen-ansätze/

 

I would say you are wasting at least 40W.

 

 

2 hours ago, Kronos_ said:

I get this error

Only warnings, can be ignored.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.