Jump to content

Unraid Power Inefficiency on 14-series Intel Processors


Recommended Posts

The intel_idle driver in the current Linux kernel (6.1.64) used by Unraid does not support Intel 14-series Meteor Lake processors.

This makes it impossible to reach lower C-states than C3, as shown by powertop:

 

image.png.b84e47c6a015bee6782f1afdc8e9e2af.png

 

The lack of support is indicated by the `_ACPI` suffix on the sleep state labels. For reference, I am using the recently released Intel i5-14500 processor.

 

* intel_idle driver in Linux 6.1.64 showing missing support for Meteor Lake: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/idle/intel_idle.c?h=v6.1.64

* intel_idle driver in the latest stable kernel with Meteor Lake support: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/idle/intel_idle.c#n1532

Link to comment
55 minutes ago, Limeth said:

It looks like this commit is what added the functionality to the Linux kernel. It isn't currently in any released kernel versions. I wonder if there is a way to backport it to the current version of the kernel used by Unraid? I am not experienced in this matter.

Do you think that also 13th gen is not supported by the current kernel? Because from powertop I see that I have ACPI_ as well and another user with my same motherboard does not   get any pkg read at any time. See my attachment. In my case I can't get past C3Screenshot_20240121_230857_Termius.thumb.jpg.9f21b130444e8b952f8df79a3c13ced3.jpg

Edited by firstTimer
Link to comment
4 minutes ago, firstTimer said:

Do you think that also 13th gen is not supported by the current kernel? Because from powertop I see that I have ACPI_ as well and another user with my same motherboard does not   get any pkg read at any time. See my attachment. In my case I can't get past C3Screenshot_20240121_230857_Termius.thumb.jpg.9f21b130444e8b952f8df79a3c13ced3.jpg

 

Actually, yes! I think I was mistaken about the i5-14500 using the codename "Meteor Lake", it seems to apply to mobile 14-series CPU's only. The i5-14500 is listed on the wiki page for "Raptor Lake", which covers both 13-series and some 14-series CPU's.

Due to my misunderstanding I was only looking for resources related to this issue in 14-series CPU's, but I might have more luck finding results on the 13-series, as well.

Note that support for "Raptor Lake" is neither in the current Unraid Linux kernel, nor the latest "master" Linux kernel. Strange, considering how long the 13-series has been around.

Link to comment
20 minutes ago, Limeth said:

 

Actually, yes! I think I was mistaken about the i5-14500 using the codename "Meteor Lake", it seems to apply to mobile 14-series CPU's only. The i5-14500 is listed on the wiki page for "Raptor Lake", which covers both 13-series and some 14-series CPU's.

Due to my misunderstanding I was only looking for resources related to this issue in 14-series CPU's, but I might have more luck finding results on the 13-series, as well.

Note that support for "Raptor Lake" is neither in the current Unraid Linux kernel, nor the latest "master" Linux kernel. Strange, considering how long the 13-series has been around.

Thanks @Limeth Yes I think 14th is referred as raptor lake refresh, If I am not mistaken though, from 12th gen Intel CPU to 13th and 14th, there were no big changes so I think that the Linux kernel could be easily updated. Well actually it seems that the Linux kernel from 6.6.0 was updated to add support to Intel 14th gen. I don't what version of Linux Kernel we are on though on Unraid.

https://www.phoronix.com/review/intel-14600k-14900k-linux

 

I just looked at Unraid latest version and the kernel seems to be based on 6.1.4 which should mean that 13th gen is/should be supported but not the 14th.

Screenshot_20240121_234133_Opera.jpg

Edited by firstTimer
Link to comment
8 hours ago, Limeth said:

 

What are you basing this on? I can't find any info on 13th gen being supported on Linux 6.1.x.

Relevant issue on the powertop repository: https://github.com/fenrus75/powertop/issues/140

 

I opened that issue. I'm not very familiar with the Intel_idle code but I didn't see any handling for Raptor Lake in it. I was going to email the maintainer of Intel_idle (who I think is a different person to the maintainer of powertop) and ask them about it, but I haven't gotten around to that yet. 

 

If someone submits a patch to add Raptor Lake support, it's likely that it could be backported to the kernel version that Unraid uses. I'm a software developer but I'm not familiar enough with C (or the internals of Intel processors) to try attempt it myself. 

Edited by Daniel15
Link to comment
48 minutes ago, Daniel15 said:

 

I opened that issue. I'm not very familiar with the Intel_idle code but I didn't see any handling for Raptor Lake in it. I was going to email the maintainer of Intel_idle (who I think is a different person to the maintainer of powertop) and ask them about it, but I haven't gotten around to that yet. 

 

If someone submits a patch to add Raptor Lake support, it's likely that it could be backported to the kernel version that Unraid uses. I'm a software developer but I'm not familiar enough with C (or the internals of Intel processors) to try attempt it myself. 

Yes, I am not into low-level coding as well. Anyway it means that with kernel we are not ok (at least for 14th that was supported from 6.6). BUT we still don't know/understand if intel_idle actually "applies" any low C-state to our 13th and 14th gen CPUs and/or powertop show or c-states correctly. Am I right?

Link to comment
6 minutes ago, dopeytree said:

Mines a 11900t but shows up as Processor:Genuine Intel® 0000 @ 1.10GHz

 

This is what powertop gives me:

664715830_Screenshot2024-01-22at11_59_17.thumb.png.d03f7177cdd0db3d62d7aa7f2648c198.png

what version of powertop are you using? I am using the 2.15. I would say that maybe the "un-updated" linux kernel could be the reason why powertop is not behaving correctly. But I don't know, I can't understand well enough how each piece of software is related to the other. Maybe it would be interesting to know the tree of dependencies for powertop and know what version of each dependency is installed in Unraid

Edited by firstTimer
Link to comment
8 hours ago, dopeytree said:

same version here. I might do an experiment and see if another 11th gen cpu changes it.

  @Limeth@dopeytree @Daniel15

I made this experiment. I created an USB stick with an Ubuntu Live, then inside ubuntu, I installed powertop (with version 2.15) to see what happens there. I discovered that they behave exactly the same, in Ubuntu I don't see any C-state and I see the X_ACPI string. The linux kernel version inside ubuntu was 6.5.0.9 (which should support raptorlake fully now), and powertop is also 2.15 (see screenshot 1 and 2). So, to recap, looking at the intel_idle (master branch, screenshot 3) it seems that RAPTOR_LAKE is completely missing! but meteorlake is there! So as far as I understood, if 13th gen raptorlake is not in this list, it means that everyone running a 13th cpu under linux cannot take advantage of low c-states, or at least cannot reach and/or check their actual c_state.

The only thing I would suggest at this point is that someone with a 14th gen cpu:

  1. installs linux on a disk (because the update of the kernel cannot be applied on USB key AFAIK)
  2. install a version of the linux kernel >= 6.6.0
  3. Installs and runs powertop

These are the expectations:

  1. The X_ACPI string should go away
  2. In a good scenario, C-states should appear correctly and in the best possible scenario, C-states could go lower than C2/C3 if any component/pcie does not create any problem with ASPM. 

 

image_2024-01-22_193039428.png

Screenshot 2024-01-22 192432.png

image_2024-01-22_193205417.png

Edited by firstTimer
Link to comment
14 hours ago, firstTimer said:

Yes, I am not into low-level coding as well. Anyway it means that with kernel we are not ok (at least for 14th that was supported from 6.6). BUT we still don't know/understand if intel_idle actually "applies" any low C-state to our 13th and 14th gen CPUs and/or powertop show or c-states correctly. Am I right?

I just search the mailing list and found this thread: https://lore.kernel.org/linux-pm/[email protected]/

 

It looks like Intel wanted to measure the latency of transitions between C-states on Raptor Lake to determine if the previous table from Alder Lake could be reused, or if Raptor Lake should have its own configuration. They said "Hopefully we will have a patch in a couple of weeks" around a year ago, but a followup email from August 2023 says it's still a work in progress.

 

Having said that, it sounds like the ACPI states should be fine, at least from how I'm reading that thread. Not sure.

 

If you want to try it out, the first post in that thread (https://lore.kernel.org/linux-pm/[email protected]/) has a patch that enables intel_idle support for Raptor Lake by reusing the Alder Lake config. You could build an Unraid kernel with that patch if you want to try it out and see if it helps.

Edited by Daniel15
  • Like 1
Link to comment
22 minutes ago, Daniel15 said:

I just search the mailing list and found this thread: https://lore.kernel.org/linux-pm/[email protected]/

 

It looks like Intel wanted to measure the latency of transitions between C-states on Raptor Lake to determine if the previous table from Alder Lake could be reused, or if Raptor Lake should have its own configuration. They said "Hopefully we will have a patch in a couple of weeks" around a year ago, but a followup email from August 2023 says it's still a work in progress.

 

Having said that, it sounds like the ACPI states should be fine, at least from how I'm reading that thread. Not sure.

 

If you want to try it out, the first post in that thread (https://lore.kernel.org/linux-pm/[email protected]/) has a patch that enables intel_idle support for Raptor Lake by reusing the Alder Lake config. You could build an Unraid kernel with that patch if you want to try it out and see if it helps.

If I figure out how to patch the Unraid kernel, I might give this a try, as my newly built Unraid server is waiting for an HDD replacement to be usable.

Link to comment

I emailed Rui Zhang (one of the maintainers of Intel_idle) and Arjan van de Ven (maintainer of powertop) at Intel earlier today to ask about this, and Rui just sent this reply:

 

Quote

intel_idle uses ACPI idle states if there is no custom table. This already brings most of the value of cstates.

A custom table is not mandatory unless we see PnP benefit compared with ACPI cstates.

For RPL, there are some evaluation ongoing internally, but we're not convinced with the previous result so far. So there may be some updates later, but it is not guaranteed.

 

(RPL = Raptor Lake)

Edited by Daniel15
Link to comment
3 hours ago, Daniel15 said:

I emailed Rui Zhang (one of the maintainers of Intel_idle) and Arjan van de Ven (maintainer of powertop) at Intel earlier today to ask about this, and Rui just sent this reply:

 

 

(RPL = Raptor Lake)

Thanks for the emails.

Just 1 question is still opened then:

Does it mean that with ACPI cstates we will only see a maximum c-state of 3, or is it just a powertop error to not interpret ACPI c-states correctly?

Rant moment 🤣:

There was a big scandal after the release of meteorlake that was just a rebranding of raptorlake with a maximum of 5% improvements over the previous gen, so how in the world is meteorlake getting supported in intel_idle when raptorlake is not present? It means they developed or they should have developed a custom table for meteorlake which IMHO should be compatible with raptorlake as well

Edited by firstTimer
Link to comment

Please correct me if a am wrong but (i think) the support for the 14th gen cpus is already there.

 

I set up a new Unraid (6.12.6) system yesterday with the following hardware and reached C8 only with optimizing the BIOS and installing Powertop:

 

MB: Gigabyte B760i Aorus Pro DDR4

CPU: i3-14100

PSU: Be Quiet! Pure Power 12 M (550W)

RAM: Kingston FURY Beast (1 x 32GB)

SSD: Samsung 860 EVO (500GB) as Cache

M.2: Samsung 870 EVO (1TB) as Array

Unraid on Samsung BAR Plus 32GB USB Stick

 

Added the following commands to the go file in /boot/config to run powertop and disable wake-on-lan:

 

grafik.thumb.png.13f432421ce91f771103fa1b3f0ab3d2.png

 

I reused an older BeQuiet Black Rock Pro 4 Cooler without any fans and it is running happily at around 25°.

With no keyboard, display or fans attached, the system is using around 8.5 watts with a spike up to around 11 watts every now and then.

I suppose the energy consumption will rise a little bit with any further drives attached and VMs/Docker Container installed & running.

 

As requested from @SB_max here is my current BIOS configuration. I am running on version F9 and added another Evo 860 in the meantime as btrfs cache pool. Furthermore i changed the "Initial Display Output" to iGPU which is not showing up on the pictures.

Further optimization might be possible with changing the CPU/RAM voltage but i didn't change anything there - any tips are welcome :)

 

PS. I know there might be better components out there but the higher costs for the hardware will probably not equal the savings with the slightly lower power consumption

 

 

20240211_141015.thumb.jpg.b71dd2e50da6b3617fada89468b24d8c.jpg

 

20240211_140919.thumb.jpg.61060f7a769676feb2ed02b0ebaf42e3.jpg

 

20240211_140902.thumb.jpg.f2711ee1472aec13d5e76211ced96532.jpg

 

20240211_140850.thumb.jpg.23480b8be1d11515a744d311e768ddb0.jpg

 

20240211_140831.thumb.jpg.5d2a25a873550fddb797a8c1ba758576.jpg

 

20240211_140813.thumb.jpg.3566abbfd2c9b80c9011e5e23e5ded29.jpg

 

20240211_140715.thumb.jpg.c6dd480598b0facea4f220e3f90864d9.jpg

 

20240211_140654.thumb.jpg.124395661b7e04f150a08f015d6d9f64.jpg

 

20240211_140628.thumb.jpg.7d3d9164daa74638cc00052ae603bcc6.jpg

 

20240211_140613.thumb.jpg.a97debcb72d0bdad01f2e2a3f8ba9106.jpg

 

20240211_140548.thumb.jpg.ac96ad9c55c4954b76d4d542e7965aac.jpg

 

20240211_140517.thumb.jpg.43c6f6e1f214f206961a4fea4c9c883b.jpg

 

20240211_140450.thumb.jpg.1fbdc55258f078c633a8a85766f45d76.jpg

 

20240211_140438.thumb.jpg.15b0547ca46c20f15aecff78c169b161.jpg

 

Unbenannt.JPG

 

Unbenannt2.JPG

Edited by Jabberwocky
added bios details
  • Like 2
Link to comment
1 hour ago, Jabberwoocky said:

Please correct me if a am wrong but the support for the 14th gen cpus is already there.

 

I set up a new Unraid (6.12.6) system yesterday with the following hardware and reached C8 with optimizing the BIOS and installing Powertop:

 

MB: Gigabyte B760i Aorus Pro DDR4

CPU: i3-14100

PSU: Be Quiet! Pure Power 12 M (550W)

RAM: Kingston FURY Beast (1 x 32GB)

SSD: Samsung 860 EVO (500GB) as Cache

M.2: Samsung 870 EVO (1TB) as Array

Unraid on Samsung BAR Plus 32GB USB Stick

 

I reused an older BeQuiet Black Rock Pro 4 Cooler without any fans and it is running happily at around 25°.

With no keyboard, display or fans attached, the system is using around 8.5 watts with a spike up to around 11 watts every now and then.

I suppose the energy consumption will rise a little bit with any further drives attached and VMs/Docker Container installed.

 

 

 

PS. I know there might be better components out there but the higher costs for the hardware will not equal the costs for the slightly higher power consumption.

 

Unbenannt.JPG

Unbenannt2.JPG

image.thumb.png.6149b77dd3be2141bfd5611e17d70334.png

@Jabberwoocky I am quite confused then... looking at the git blame, it seems that the support to intel_idle was added for the 14th gen, because it is in master. So once Unraid updates the linux kernel to version 6.8.0 (as it seems in RC version) then powertop should remove the ACPI strings and show the correct values.

BUT as we can see, even if Unraid is not using the latest kernel (therefore it is not using intel_idle, hence the reason why powertop shows ACPI strings) it seems the case that 14th can actually reach lower C-states nonetheless. So I guess that people with 13th gen CPUs can also reach lower states even without intel_idle (because we all have ACPI)

Edited by firstTimer
Link to comment

So @Daniel15 i guess that in our case the cause for inefficiency is to look elsewhere, either the IPMI card, or the ethernet controller (which has the cheapest Intel I226-V). Other things of course can be the NVME or HDD or anything attached to the MB really 😅

For you @dopeytree the issue is probably to look elsewhere as me and @Daniel15 then (so anything attached to the MB can cause the issue). 

@Limeth for you, accordingly to what @Jabberwoocky shows, it seems that you also can reach low c-states and therefore I would say that the issue is again, the MB or anything attached to it.

  • Like 1
Link to comment
2 hours ago, Daniel15 said:

I asked about this and the guy at Intel said that ACPI C3 is equivalent to CPU C10. 

@Daniel15 Thanks for the confirmation :) So yeah, I guess that the IPMI card is the first thing to check on our MBs if we want to reach C10 or ACPI C3. 🤔

Edited by firstTimer
Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...