Erlaubt der neue Linux Kernel stromsparende AMD Server?

[Pillendreher]

OK, so I've installed the latest version (Danke @ich777!) and tried registering everything. This is as far as I got:

 

 

This is what happens when trying to register the Idle driver:

 

 

This is what the Kernel data looks like:

 

 

PS: Idle Limit C3? Any way to change that? Also, it looks like the CPU is never quite leaving C1:

 

 

Anyway, this is the core power draw in idle:

 

 

And now the CPPC "ranges" are set to auto:

 

[CyrIng]

@Pillendreher You have to unload or blacklist any drivers

## Boot command line (for a AMD Ryzen 3950X)
nmi_watchdog=0 modprobe.blacklist=k10temp,sp5100_tco,acpi_cpufreq amd_pstate.shared_mem=1 idle=halt nowatchdog

## You can also try to unload current ACPI driver
modprobe -r acpi-cpufreq

## You can pass Registration and Idle route as parameters
insmod corefreqk.ko Idle_Route=1 Register_ClockSource=1 Register_Governor=1 Register_CPU_Freq=1 Register_CPU_Idle=1

## Or from the UI settings, do all registrations and select again route to "IO"

 

Remarks:

• ClockSource registration is still a Compilation option; project has to be built as below:

  make DELAY_TSC=1

• UMC Memory Controller channels frequency can be built as below:

  make ARCH_PMC=UMC

• And options can be combined:

• make ARCH_PMC=UMC DELAY_TSC=1

 

[ich777]

4 minutes ago, CyrIng said:

And options can be combined:

Should I change my build script to compile the AMD package with these two options?

[CyrIng]

@Pillendreher

Once CPU-Idle registered, you can select other Idle limit.

Default limit is a function of Sub C-State provided by architecture

 

[CyrIng]

2 hours ago, ich777 said:

Should I change my build script to compile the AMD package with these two options?

 

DELAY_TSC=1 appears safe, you can include it in your compilation

A compilation message will show up to warn that feature is builtin

 

ARCH_PMC=UMC is stable with my 3950X and 5300U but I don't have other reports from other AMDs

[CyrIng]

Examples of UMC monitoring

 

DIMMs are set to 3600 and FCLK to 1800 in BIOS

 

Idle and Loaded cycles of Dual channels CHA0 and CHA1

[CyrIng]

If Clock Source is not mastered by CoreFreq, you can select it from Kernel window:

 

[ich777]

27 minutes ago, CyrIng said:

DELAY_TSC=1 appears safe, you can include it in your compilation

Then I will integrate only this option for now...

[Pillendreher]

4 hours ago, CyrIng said:

@Pillendreher You have to unload or blacklist any drivers

## Boot command line (for a AMD Ryzen 3950X)
nmi_watchdog=0 modprobe.blacklist=k10temp,sp5100_tco,acpi_cpufreq amd_pstate.shared_mem=1 idle=halt nowatchdog

## You can also try to unload current ACPI driver
modprobe -r acpi-cpufreq

## You can pass Registration and Idle route as parameters
insmod corefreqk.ko Idle_Route=1 Register_ClockSource=1 Register_Governor=1 Register_CPU_Freq=1 Register_CPU_Idle=1

## Or from the UI settings, do all registrations and select again route to "IO"

 

Remarks:

• ClockSource registration is still a Compilation option; project has to be built as below:

  make DELAY_TSC=1

• UMC Memory Controller channels frequency can be built as below:

  make ARCH_PMC=UMC

• And options can be combined:

• make ARCH_PMC=UMC DELAY_TSC=1

 

 

OK, so the first line (with a "modprobe amd-pstate" attached to it) lets me register the Idle-driver. Yet now I can't register the freq-driver:

 

[Pillendreher]

4 hours ago, CyrIng said:

@Pillendreher

Once CPU-Idle registered, you can select other Idle limit.

Default limit is a function of Sub C-State provided by architecture

 

 

I can go all the way down to C6 now actually:

 

 

And the frequency dashboard actually shows the cores reaching C6:

 

 

Powerdraw is basically identical though compared to the previous configuration:

 

[CyrIng]

3 hours ago, Pillendreher said:

...

Powerdraw is basically identical though compared to the previous configuration:

 

 

Unfortunately, no hardware counter registers specified with AMD Zen.

CoreFreq is thus making virtual counter VPMC per idle level entry.

 

Using 3950X, I have noticed a Package power consumed difference between I/O-WAIT and MWAIT. That's why I/O or HALT is default idle asm instruction suggested in [Settings][CPU-IDLE route]

 

In [Performance Monitoring] window PC6 and CC6 are making a Power difference.

 

 

In [Power, Current & Thermal] window, when [Settings][HSMP enablement] can be enabled, [Thermal Design Power] can be altered.

With 3950X, I can't programmed better than Minus/Plus 5 watts

 

 

You may also cap the Core frequency (press [!] for Absolute monitoring); enable CPPC and decrease the target [TGT] ratio.

With 3950X where CPPC is a firmware implementation, I'm observing a Core Power decreased

 

[Pillendreher]

 

9 hours ago, CyrIng said:

Unfortunately, no hardware counter registers specified with AMD Zen.

CoreFreq is thus making virtual counter VPMC per idle level entry.

 

Which means? Is something missing in my UEFI settings?

 

9 hours ago, CyrIng said:

Using 3950X, I have noticed a Package power consumed difference between I/O-WAIT and MWAIT. That's why I/O or HALT is default idle asm instruction suggested in [Settings][CPU-IDLE route]

 

I'll give the other options a try.

 

9 hours ago, CyrIng said:

In [Performance Monitoring] window PC6 and CC6 are making a Power difference.

 

 

Both are already enabled.

 

9 hours ago, CyrIng said:

In [Power, Current & Thermal] window, when [Settings][HSMP enablement] can be enabled, [Thermal Design Power] can be altered.

With 3950X, I can't programmed better than Minus/Plus 5 watts

 

 

Unfortunately, I can't enable "HSMP". It's currently set to "off":

 

 

 

Yet when I try to enable it, nothing happens:

 

 

 

And the CPU-Freq driver still can't be enabled either:

 

 

 

 

9 hours ago, CyrIng said:

 

You may also cap the Core frequency (press [!] for Absolute monitoring); enable CPPC and decrease the target [TGT] ratio.

With 3950X where CPPC is a firmware implementation, I'm observing a Core Power decreased

 

 

I've put it at "1" (99,81 MHz):

 

 

Compared to what I posted yesterday (7.4 W total), I'm now at 4,74 W total in this screenshot - nice improvement

 

PS: What is "absolute frequency" in this setting? It's much higher than the accumulated cores' frequencies?

[CyrIng]

Hardware counters: I mean registers like Intel MSR which counts number of cycles elapsed per C-states 

 

AMD does not provide information about such registers. NDA or no will. 

All Zen impacted. 

 

My workaround has been to hook kernel idle entries to estimate the cycles delay, from TSC, spent in idle level. 

 

[CyrIng]

Relative frequency is a usage, delta of cycles over a sample period : 1 second

 

Absolute frequency is an immediate reading of current frequency which is made once per period. 

 

Indeed absolute mode is the way to get true frequency, especially when system is loaded whereas relative can measure the tiny variations of GHz processors. 

 

Relative frequency is the Intel recommandation. 

Edited August 28, 2022 by CyrIng

[CyrIng]

48 minutes ago, Pillendreher said:

I've put it at "1" (99,81 MHz):

Very interesting! 

What did you do exactly to improve the Power consumed ? 

[CyrIng]

51 minutes ago, Pillendreher said:

And the CPU-Freq driver still can't be enabled either:

When you reach that issue, give a look to kernel window to get the current driver name. 

Then try to unload it or blacklist it from boot. 

 

When "Missing" is written, it means the room is empty and CoreFreq can register. 

 

It's the same for Idle driver. 

 

I'm sorry if I can't make easier but Linux kernel was made that eavy way in some part of its code. 

[Pillendreher]

6 hours ago, CyrIng said:

When you reach that issue, give a look to kernel window to get the current driver name. 

Then try to unload it or blacklist it from boot. 

 

When "Missing" is written, it means the room is empty and CoreFreq can register. 

 

It's the same for Idle driver. 

 

I'm sorry if I can't make easier but Linux kernel was made that eavy way in some part of its code. 

 

It says that the p-state driver is the current CPU-Freq driver:

 

 

I'll try not loading it at boot, maybe that will help.

 

EDIT:

 

 

So now that everything is missing, I can successfully register everything via corefreq (Still can't enable HSMP though):

 

 

Although now the Governor driver seems to be missing, which - I assume - isn't good.

 

 

 

Edited August 28, 2022 by Pillendreher

[Pillendreher]

6 hours ago, CyrIng said:

Very interesting! 

What did you do exactly to improve the Power consumed ? 

Not much. I just enabled C-States in my UEFI, enabled C6 in corefreq and switched the CPPC Target to "1". That's all!

[CyrIng]

3 hours ago, Pillendreher said:

So now that everything is missing, I can successfully register everything via corefreq (Still can't enable HSMP though):

 

I have found the HSMP protocol into the EPYC datasheet; and probably also the same specs for ThreadRippers.

 

Except some HEDT Deskop Ryzen 3950X, 5950X, I have also noticed that HSMP is not implemented into Ryzen APU, like my laptop 5300U

[kdwg]

Ein kleines Update zum Thema:

 

https://www.phoronix.com/news/AMD-P-State-EPP-Linux

[CyrIng]

6 hours ago, kdwg said:

Ein kleines Update zum Thema:

 

https://www.phoronix.com/news/AMD-P-State-EPP-Linux

That indeed will be nice when the EPP patch gets into mainstream, especially:

+EXPORT_SYMBOL_GPL(cppc_get_epp_caps);

+EXPORT_SYMBOL_GPL(cppc_set_epp_perf);

 

[CyrIng]

On 8/28/2022 at 7:01 PM, CyrIng said:

I have found the HSMP protocol into the EPYC datasheet; and probably also the same specs for ThreadRippers.

 

Except some HEDT Deskop Ryzen 3950X, 5950X, I have also noticed that HSMP is not implemented into Ryzen APU, like my laptop 5300U

Bad news. Received a laptop 6600H based where Rembrandt has no HSMP, most SMU registers I need for voltage Vcore and UMC memory controller are empty. 

 

Fortunately legacy p-state, CPUID, Thermal are still  monitoring OK. 

 

So far I've not found a registers datasheet of this Zen3+

... and I fear it may be worst with Zen4 (but no test received) 

[ich777]

@Pillendreher gehe ich recht in der Annahme das du deiner syslinux.config das hinzugefügt hast:

initcall_blacklist=acpi_cpufreq_init

 

...und dann noch die Datei:

/boot/config/modprobe.d/amd_pstate.conf

 

um den Treiber zu laden?

 

Geht mir nur um die Klarheit für andere User die den amd_pstate Treiber zu aktivieren versuchen.

[kdwg]

22 hours ago, ich777 said:

@Pillendreher gehe ich recht in der Annahme das du deiner syslinux.config das hinzugefügt hast:

initcall_blacklist=acpi_cpufreq_init

 

...und dann noch die Datei:

/boot/config/modprobe.d/amd_pstate.conf

 

um den Treiber zu laden?

 

Geht mir nur um die Klarheit für andere User die den amd_pstate Treiber zu aktivieren versuchen.

 

Ja, oder eben im go-file

[ich777]

25 minutes ago, kdwg said:

 

Ja, oder eben im go-file

Aber diese beiden dinge genügen oder?