Ryzen c-states / idle power draw


Go to solution Solved by JorgeB,

Recommended Posts

Goodday everyone.

 

This is my first post, ive seen plenty of topics regarding the subject but none really answer my question.

 

Im currently running esxi for vm's and a separate file server.

Now its time to upgrade i want to use unraid to combine fileserver with vm's (pfsense + home assistant).

Generally id go for intel, however its newer products dont support ecc (s1700 only from 12500 onwards).

So the easyest option is ryzen, as they all support ecc, however there is/was the issue with ryzen c-states.

Seeing electricity prices are "ryzing" (phun intended) efficiency is an important factor for me.

ive seen ppl having issues with 3000 series, and advising to change Power Supply Idle Control to typical.

 

My question is, how does this affect the c-states on the cpu? Will the cpu still be able to enter high c-states?

If idle power on ryzen is too high id rather spend extra money on 12500 or s1200 xeon, rather than spend it on electricity.

 

Link to comment

On the latest Ryzen models, the idle power draw has reached an acceptable range, at least how I have defined it for for myself, considering the advantages the platform does have like the mentioned not-disabled ECC functionality.
For my system, Idle is around 21-23W, Team blue is estimated to be 7-8W lower, at maximum. Not mentioning the initial costs would have been higher or availability not given on time of procurement.

Room for improvement, sure. But tbh, im curious what yet coming kernel updates will achieve here

Edited by kdwg
Link to comment

I have a server with a ryzen 9 3900x, when I look at the c-states in powertop I get the following:

image.png.cfce64035468338650a3f55ddeedcc27.png

 

In the bios I have the following enabled:

  • Global C-state Control = Enabled
  • Power Supply Idle Control = Low Current Idle
  • CPPC = Enabled
  • CPPC Preferred Cores = Enabled

I was thinking I should also be seeing C6 as c-state. But it's not showing. Is there a way to get the machine in to the c6 state? Or could it be possible that a pcie device is preventing this?

Link to comment

I have the same processor and a similar config in the BIOS; the only possible way to reach C6 states it's using the AMD PState; however, in my testing, Unraid lags a lot using this config given that the driver doesn't schedule priority task really well, one example its Telegraf+Prometheus+InfluxDB2, some metrics get missed. So I ended up reverting to acpi_cpufreq. Additionally, power use was reduced between 15% and 20%.

  • Like 1
Link to comment
On 11/2/2022 at 4:42 PM, Iker said:

I have the same processor and a similar config in the BIOS; the only possible way to reach C6 states it's using the AMD PState; however, in my testing, Unraid lags a lot using this config given that the driver doesn't schedule priority task really well, one example its Telegraf+Prometheus+InfluxDB2, some metrics get missed. So I ended up reverting to acpi_cpufreq. Additionally, power use was reduced between 15% and 20%.

Can you explain how you did this? Or give me a guide about how to do this? 

 

If I look at my cpu with CoreFreq I get the following:

image.png.569f4d6422092de79f5108b5aab9fbb3.png

 

Which says idle limit is C2, is there a way to change this? 

My bios is a gigabyte auros master B550. Don't know if I need to enable more settings in the bios? 

Edited by de Techneut
Link to comment

For the options already configured in BIOS everything seems to be okay. First; make sure that you are on the latest version of unraid (6.11.1), then add the following options to syslinux config

 

modprobe.blacklist=acpi_cpufreq amd_pstate.shared_mem=1

 

After that and just to be sure add this to your go file

modprobe amd_pstate

 

Reboot, and that's it; but, as I have stated; probably it's not going to worth all the trouble, the power reduce doesn't compensate for the latency problems.

  • Like 1
Link to comment
  • 2 weeks later...

Currently I'm trying to bring the powerconsumption of my server down because of the rising prices in electricity. I can't bring my Ryzen 1700 to use more then C1 and C2. I've enabled Global C-state Control = Enabled and Power Supply Idle Control = Low Current Idle in the BIOS and used your tweaks including powertop --auto-tune.

 

My setup:

Asus B450 F Strix 

Ryzen 1700

7x WD 8TB HDD

3x WD 4TB HDD

4x SanDisk SSDs

1x Crucial m.2

1x nvidia GTX1050ti for transcoding

5x Noctua 120mm Fans running in "silent mode" according to the BIOS

1x Dell H310 SAS Controller + 40mm Noctua fan

 

The whole system idles at 56-61W but I would love it to bring this down even further.

 

I am glad for any advice!

 

Link to comment

My server has a similar config (see siggy below).  With 18 Dockers running, and 3 of my drives always spinning, it idles around 65W.

 

I doubt messing with C-States will find you much savings vs the time and performance it may cost.  Given the list you shared, I would expect that the GTX1050ti, H310 and 10 HDDs contribute the large majority of the 60W you are seeing.

 

Depending how comfortable you are experimenting with your server, you could test my theory by making a trial boot drive and temporarily strip your system down.  Remove the GPU, unplug drives, remove HBA.  With no Docker/VM running, see how much power you draw with these components out of the equation.

Link to comment
55 minutes ago, ConnerVT said:

Depending how comfortable you are experimenting with your server, you could test my theory by making a trial boot drive and temporarily strip your system down.  Remove the GPU, unplug drives, remove HBA.  With no Docker/VM running, see how much power you draw with these components out of the equation.

I've got 22 Dockers running and one virtual machine (Home assistant - should not pull so much). I think the H310 pulls about 8W, unfortunatly. The 56-61W are with all drives spun down.

 

So I think I am at the lowest I can get.

 

I'm trying to hold the downtimes as low as I can get, so tinkering much more is not an option. But I am curious what would be possible with a Intel platform and a low power CPU, because the extra power of the 1700 is not really needed. Only the IO of the ATX board is really necessary.

 

Has anybody experience with a Intel platform and a similar setup?

 

Link to comment
  • 2 months later...

Post might be a bit late… but here goes anyway…
 

This is going to be difficult to do a direct comparison so let’s do some basic calcs.

 

All your HDDs and SSDs are somewhere around 0.5-1W idle. 15 of them, so let’s call that 8-10W

 

A 1050Ti apparently uses 3-4W when idle, however we’d need to check if it is indeed idling so might be worth removing it to see what happens.

 

Fans are probably using 1W each. Again can unplug them temporarily to get a reading.

 

When I removed my SAS controller from my old server (consolidated 10 4TB drives into four 10TB drives) I dropped 10W, plus a further 6W from the drive consolidation (and 40!W when they were all spinning!) I’m going to guess the H310 won’t be far off that. 
 

So adding all that up.. 25-30W bringing you down to ~30W.

 

You don’t state the PSU you’re using unless I’ve missed it, if it’s not a Titanium/Platinum the efficiency below 10% load could be as bad as 70%.

 

That leaves the power draw of the Ryzen + Mobo at low 20s of watts.

 

Add to the fact the GPU and LBA card will require PCI-E lanes to be switched on I can see the 20W reading being about right, given that the first Gen Ryzens really sucked idle power wise.

 

My server (thanks to mgutts script) without the four port NIC idles at ~9-10W according to the UPS.

 

That’s four 14TB drives, H310 motherboard, 16GB DDR4, 8600T, 1TB NVME, 550W Platinum PSU. 
 

So if you really want to lower the consumption, then sadly it’s Intel (I really wanted to go AMD but the idle power draw of early ryzens is poor) plus storage consolidation (to get rid of the LBA card).

 

May half the draw then.

 

Edited by Interstellar
Link to comment

OT: Interstellar now i know that intel it's better for idle ( i knew it before too, but didn't think it would be so awful with amd ) but anyway now that i have bought an amd ryzen just for this build I, like every other ryzen user, would love to be able to use higher c-state or p-state or whatever.

They've been around since 2006-7 with the first core-duos, i find unacceptable that in 2023 i can't go higher than c2 with my almost-brand-new ryzen.

It's just wrong. 

Edited by lory995
Link to comment
  • 2 weeks later...
On 1/24/2023 at 4:00 PM, Interstellar said:

What’s the power usage in windows?

 

it may just be with certain things happening you just can’t go into C2.


what does corefreq-cli say for power?

Hi, sorry for the late response but i was, and still am, really caught up with work.
Anyway i can't launch corefreq-cli it gives me an error atm 

Daemon connection error code 2
corefreq-ro-shm: 'No such file or directory' @ line 21300

but i didn't configure it nor looking into it so i don't know.

I'm gonna look into windows power consumption when i'll have my VM back running again, hopefully soon, but right now in unraid with 4 VMS up and running (2cores x 4 = 8 cores used , avg 30% usage all the time really low demanding tasks) and a total average of 10-15% of all CPU power used i have like 65/70W of real power draw ( at the power outlet ). I got amd_pstate drivers installed.

 

I guess the power draw in windows would be lower thanks to better drivers, after some search it seems that amd is working a lot in reducing their power usage but linux until kernel 6.x it's really behind. 

On 1/24/2023 at 10:22 PM, CobraPL said:

Disable Global C States and CPPC. First one caused hangs/resets, second one degrades performance a lot (there was a thread on Reddit with benchmarks).

Actually i have both of them up and am not noticing hangs nor resets at all ( at least with pstate driver and schedutil governor online for 14 days ). Same for performance.

Still, i'd like to ask you why you suggest such a thing. FWIK C-states and CPPC only function it's optimize, that's to say use the least amount of power for the most of performance. Of course if you are gonna disable them you got more performance, but that's not the topic of this thread. Actually it's the opposite.

I've written of all the above just to say: are you implying that they don't work adeguately with unraid so we can get more power saving disabling them instead of keeping them enabled? Can you post the source of your information? i'd like to know more about it before going to mess in the bios.

Edited by lory995
Link to comment
On 2/5/2023 at 6:41 PM, lory995 said:

Hi, sorry for the late response but i was, and still am, really caught up with work.
Anyway i can't launch corefreq-cli it gives me an error atm 

Daemon connection error code 2
corefreq-ro-shm: 'No such file or directory' @ line 21300

but i didn't configure it nor looking into it so i don't know.

I'm gonna look into windows power consumption when i'll have my VM back running again, hopefully soon, but right now in unraid with 4 VMS up and running (2cores x 4 = 8 cores used , avg 30% usage all the time really low demanding tasks) and a total average of 10-15% of all CPU power used i have like 65/70W of real power draw ( at the power outlet ). I got amd_pstate drivers installed.

 

I guess the power draw in windows would be lower thanks to better drivers, after some search it seems that amd is working a lot in reducing their power usage but linux until kernel 6.x it's really behind. 

Actually i have both of them up and am not noticing hangs nor resets at all ( at least with pstate driver and schedutil governor online for 14 days ). Same for performance.

Still, i'd like to ask you why you suggest such a thing. FWIK C-states and CPPC only function it's optimize, that's to say use the least amount of power for the most of performance. Of course if you are gonna disable them you got more performance, but that's not the topic of this thread. Actually it's the opposite.

I've written of all the above just to say: are you implying that they don't work adeguately with unraid so we can get more power saving disabling them instead of keeping them enabled? Can you post the source of your information? i'd like to know more about it before going to mess in the bios.

1.


2.

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.