Hi,

As of lately as prices of electricity have been rising everywhere and as I was running an older (spare parts from 2016) Unraid server 24/7 which last I had measured (last i remember, been awhile) 60-100w idle consumption. Which for a small server was in my opinion to high of a consumption, main problem was cpu had no igpu and had to run a dgpu (extra 11W just for idling 95% of the time) and also PSU is older than 10 years and is showing signs that it will bite the dust soon.

Anyhow I decided after watching a ton of videos from Wolfgang's Channel on how to build a power efficient server and all the different aspects you need to be aware to efficiently build an actually low power idle server (PSU, MB, HBA, ASMP, HBA, L0sL1, powertop, even the damn nvme ssds needed to support ASPM...). Quite helpful videos to start the journey and would recommend. I decided to research (as best as I could given the time I had). I found the list from HardwareLuxx which had loads of builds that achieved amazing results but sadly no newer gen CPUs were included (13th, 14th Intel) and no full ATX that I could find as well as at least 6-8x Sata ports for HDDs and SSDs.

Good sources to get some ideas as to what you will need to look for in hardware and software to get higher C-states:
- HardwareLuxx - list of prebuilds that achieved low idle consumption (PSUs, MBs,..)
- Wolfgangs YT channel
- Matt Gadient - actual guide on a low idle build (good for specific details and why you might not be getting higher C states)

- Asrock guide for hidden BIOS settings - a must to achieve higher C states on Asrock boards (beware you are dealing with BIOS and you might brick the MB so thread carefully)
- Fellow Unraidian Mgutt (thank you for the tool) - Powertop - a must tool for getting those sweet C6+ states

Main goals:
- ASPM needs to be supported by each device (quite hard to find data for certain devices)
- MB is better to be SFF (ITX or mATX) for lowest idle cons., but if you need the ports or will need them ATX is the way to go, plan accordingly

- Intel xx500 for Intel UHD 770 if you need transcoding (12th, 13th and 14th gen are all the same igpu)
- nvme - mainly price and ASPM support (not all nvme have ASPM support)

- No HBAs - unless buying newer gen with ASPM support or ASM1166 SATA expander (that might have ASPM support)

Thus I pulled the plug and went for a 13th gen Intel build below.
 

Build:
- ASRock Z690 Extreme Intel Z690 Sun. 1700 Dual Channel DDR4 ATX - 8x Sata ports (3x nvme - but only 2 chipset linked can be used so CPU can reach C10, use Intel 1gb Lan - realtek has some issues and Consumption did not fall below 18w)
- Intel Core i5 13500 - stock cooler - not that biosflashback is a must when buying older MBs with newer CPU support
- Corsair Vengance 32gb (2x16gb) 3200
- 650 Watt Corsair RM Series RM650 - can confirm low quality capac. are actually heard when near the pc, actually ear next to the case, just fyi. (if that bothers you go with the RMx series, japanese cap)
- 1TB KIOXIA Exceria Plus G3
- 1TB KIOXIA Exceria Pro
- Fractal Design Define R5 - can fit at least 8 HDDs
+ some thin sata cables for easier managing 4x4 and 6x6 (will see how they fair, some reviews state that ppl are getting errors since the cables are low quality)

Price: 690-ish, 600-ish 12th gen you can save around 50-100 (igpu is the same, just no DDR5 support)

Good to know:
Build is super quiet, I can barely hear it. Even the stock case fans I cannot hear. Without HDDs ofc.

For 3x+ 18tb Exos Seagate and 4x 4tb WD Red and 1x SSD cache I decided it was best not to use HBAs or any sata cards for now as I wanted as less of added hardware to keep the power cons. to the minimum. Might have to go ASM1166 as it has ASPM support for better realibility.

Currently running C10 with 12w (from the wall) - without HDDS, only using 2x nvme ssds as I am still testing. No mouse, no monitor. Just eth cable is plugged in.

Power Consumption with spinned down HDDs hopefully will be in the low 30w range.


Consumption (from wall) - CPU, MB, 2x Nvme(chipset) and USB (unraid), 1x CPU fan and 2x case fan - hidden BIOS settings enabled per guide, ASPM all enabled, powertop,






As many who have bought asrock mbs might have noticed that its almost impossible to reach C10 states, yet here we are.

Happy with the new build, next ill try ITX or mATX build for even lower idle backup server, the only issue is I have an old 5820k cpu id like to use that AFAIK has not mITX/mATX mbs avaliable and no igpu. And all SSD, which I know Unraid and Parity dont go well together, if anyone has tips or running an all SSD server I am all ears if they are running them in pool (can I have more than 3 ssds in a pool with protection?).

What I hope to give here is i guess if someone else will be deciding to go this route that there are options and low-ish idle can be achieved, even with an ATX build, even on Asrock boards and a normal low wattage PSU.

On 12/11/2024 at 5:46 PM, Matthews said:

Currently running C10 with 12w (from the wall) - without HDDS, only using 2x nvme ssds as I am still testing. No mouse, no monitor. Just eth cable is plugged in.

Thanks for sharing. It's good to know there's another person beside the one from reddit, who reached C10 on the ASRock board. Personally, I'm stuck at C3 on my ASRock Z690m-ITX with 13900k. Despite all possible ASPM/Cstate/undervolt tunings in the BIOS, I get minimum 20W in idle from the wall with the headless setup identical to yours (just two NVMEs without mouse/kb/monitor/HDDs) on Ubuntu Server.

 

I have to try this hidden "S0 low power" BIOS setting. I really hope to achieve 5-6W like Reddit's poster, since even 12W, in my opinion, is still a lot. My other machine, a Lenovo m720q + i7-9700, runs at a whopping 4W in idle in a similar headless setup (nvme only + eth cable). This is my best baseline so far.

 

Perhaps a month later you got some progress or improvements on the matter?

Edited January 21, 20251 yr by white-orb
correct C0 to S0

After a couple of days into the topic, I learned that the current Linux kernel doesn't have proper idle optimizations for Intel 13 Gen and later CPUs. At the moment, it relies on ACPI C-states through the BIOS. This should be enough, but it seems that dependency on BIOS might not be as efficient as a native support when BIOS has suboptimal hidden settings.

 

Discussions on Unraid, on LevelOneTechs, in the Linux kernel mailing list (started exactly 2 years ago).

Edited January 18, 20251 yr by white-orb

Just for the record. After enabling the hidden "Lower Power S0 Idle Capability" option an idle power consumption dropped from 20W to 14W (a headless system with no display/usb/pcie attachments or onboard devices). Yet the system didn't go beyond C6 package states and C10 package state was never reached. Considering other reports about 6-7W in idle, mine is far from perfect.

 

Then, I decided to try whether proposed RaptorLake patch from kernel mailing list gives any advantage over a BIOS ACPI that is used as a fallback for unsupported platforms. I downloaded and patched the Linux 6.12.10 kernel on my Ubuntu Server 24.04 but I didn't get any observable advantage neither in package C-states nor wattage. Basically, this proved Intel's engineer statement from the mailing list:

Quote

We do have finished a series of evaluation on RPL [note: Raptor Lake], but we didn't find obvious PnP benefit by introducing a custom table [note: for the intel_idle driver]. That is why we stopped shipping one for RPL.

This means RaptorLake CPUs won't get direct c-state control from Linux intel_idle driver and ACPI fallback will be always used.

 

Now I learned enough from this and other forums about ASRock motherboards and how they are a bad choice for low power setups. Time to get rid of it.

On 1/21/2025 at 2:51 PM, white-orb said:

Basically, this proved Intel's engineer statement from the mailing list:

This means RaptorLake CPUs won't get direct c-state control from Linux intel_idle driver and ACPI fallback will be always used.

 

@white-orb

 

Is this exclusive to Raptor Lake? I know Alder Lake is very similar to it, do you know if affects them too?

13 hours ago, wavrunnr said:

@white-orb

Is this exclusive to Raptor Lake? I know Alder Lake is very similar to it, do you know if affects them too?

@wavrunnr yep, it's exclusive to Raptor Lake CPUs. It doesn't affect Adler Lake, and it is supported by intel_idle. Regarding similarities, yes, they are very similar, but not identical. Here is a direct quote of Intel's engineer from the mailing list "the latency of each c-state are still different on different platforms". Thus, Raptor Lake requires its own c-state table of latencies in the intel_idle driver to correctly support them. From what I saw in a driver's code, all other Intel CPUs are supported. They also didn't share why they concluded to not support Raptor Lake. Yeah, it makes no difference in efficiency compared to ACPI, but besides that, these ACPI C-states reported by powertop are confusing as they do hide real CPU states.

Edited March 1, 20251 yr by white-orb

Hi, could someone please tell me how can i do Asrock guide for hidden BIOS settings on Unraid??
The post said that it needed to be done in Window, i tried to run it in Window VM but it didn't work.

It won't work in any sort of a VM as changing hidden settings requires direct access to BIOS. VMs are isolated from real hardware. This only can be done from a system that runs on a bare metal. To do what you want using Unraid, you need to follow steps like these:

1. In BIOS you shall enable password protection and disable Secure Boot.
2. Login to Unraid using local terminal or SSH.
3. Download SCELNX_64 app (can be found on GitHub).
4. Read all BIOS settings into a file:

   sudo ./SCELNX_64 /o /s nvram.txt /sd nvram_dups.txt

5. Edit nvram.txt
6. Write settings back:

   sudo ./SCELNX_64 /i /s nvram.txt /cpwd <YOUR_BIOS_PASSWORD>

7. Reboot

I don't use Unraid, so I can't advise on detailed steps.

Edited April 8, 20251 yr by white-orb

On 4/8/2025 at 11:23 AM, white-orb said:

Read all BIOS settings into a file:

sudo ./SCELNX_64 /o /s nvram.txt /sd nvram_dups.txt

I just tried it on a z370 board from asrock and get:

Quote

./SCELNX_64 /o /s nvram.txt /sd nvram_dups.txt

------------------------------------------------------------------------------
|                       AMISCE Utility. Ver 5.04.1147                        |
|               Copyright (c) 2021 AMI. All rights reserved.                 |
------------------------------------------------------------------------------
f - Error: Unable to make kernel driver.
 

But i am not able to get it working...

Edited April 15, 20251 yr by elgatobavaria

Hi @elgatobavaria,

according to the AMISCE documentation you need to have some additional tools installed like a compiler:

 

 

In my case it seems they were already there, so I didn't notice they are required.

If you want more details, just google for "amisce user guide pdf".