Haswell CPU - Help getting better power efficiency

[Unfrosted8575]

Hi,

I have just built my first Unraid server and I am trying to make it as power efficient as I can given my current hardware.

 

This was basically a 0 cost build as I already had everything, I just bought a PSU so I can't pretend much, but I am sure it can consume less looking at stats.

 

My build it is as follow:

• CPU: Intel® Core™ i7-4770K CPU
• MOBO: ASUSTeK COMPUTER INC. MAXIMUS VI EXTREME , Version Rev 1.xx
• RAM: 16(4x4) GiB DDR3
• PSU: SF Series™ SF600 — 600 Watt 80 PLUS® Platinum
• Storage:
  • 4 x WDC_WD3000FYYZ
  • 2 x WDC_WD20EZRX
  • (soon I will add 2 SATA SSD in RAID1 for cache)
• USB: SanDisk Cruzer_Force

 

For the time being, I have not absolutely nothing connected as I wanted to minimized the search area for stuff that draw power, so:

• Nothing plugged on to the USB (except the boot stick, obviously)
• No PCI cards
• Using only iGPU
• All ports on the case are not connected
• Only 2 FANs in use

 

Now, getting on my report, the power consumption I have observed:

• While it is in use/under load: Around 100W
• While in idle, with all the Disk spinned down and nothing on the server running (like VM, Docks, etc..): 60W

While sure, they are not astronomically high numbers, I know it can do better, at least while in Idle.

 

For now, I have tried adjusting all the settings I thought could do something in the BIOS. I then installed "powertop" on the server to watch for C states. But if you look at the pictures I have attached, the server only reach the C2 state.

 

Following this guide, I have taken a look at the Tunables, and so a bunch of "Bad" stuff it can be optimized (take a look at the attachment), but I have no clue on what it is what and how something can affect stuff.

 

The guide that I have linked showed that there is an auto-tune command, but can have drawbacks, especially for ASM Sata controller, which I am pretty sure I have. I saw there are lot of steps and manual things to do, but I would like some more guide on how to do stuff as I am pretty scared of fuck things up.

 

Any help it is welcome, thanks.

 

EDIT 1: Looking at CPU Frequency, while in idle, many threads still go for 3.5GHz and only few stay at 900MHz. It seems like there are some processes going on.

 

EDIT 2: I have tried to follow the guide as it seems my ASM1062 controller should support power management. For the time being, I just given the command on console and not saved to file.

The situation it is improved quite a lot. Now with disks spinned up (but idle) the server draw around 60-70W, while on Idle with all disks spinned down, it went as low as 35W. Good.

 

But the C state it keeps remaining on C2 and now I really don't get it (look at the picture).

All the Tunables are now in the "Good" state, except the "VM writeback timeout" (Picture uploaded).

The CPU freq now correctly stay as low as 800 MHz (Picture uploaded).

While checking this, the server as no video output connected and I was using SSH connection from my computer with no Web GUI open and all disks spinned down.

 

What am I missing?

Edited October 8, 2023 by Unfrosted8575
Updated informations

[Lolight]

paging @mgutt

[Unfrosted8575]

...Really no one that have any idea?

[Lolight]

On 10/8/2023 at 9:27 AM, Unfrosted8575 said:

For now, I have tried adjusting all the settings I thought could do something in the BIOS.

Probably need to look more closely into BIOS settings also try to disable every unneeded circuity in BIOS.

[Lolight]

On 10/8/2023 at 9:27 AM, Unfrosted8575 said:

But the C state it keeps remaining on C2 and now I really don't get it (look at the picture).

 

Make sure to set Maximum Energy Savings in the power plan.

Your mobo comes with a Wi-FI chip - disable it in the BIOS. Then remove by unscrewing a couple of screws located underneath and then pull it out of the socket.

Make sure to activate ASPM and enable ErP mode.

DMI Link Speed --> Auto

Some of you SATA ports are connected to a separate SATA controller (ASMedia) - the bottom 4 SATA ports.

Is there an option to deactivate that controller in the BIOS as a test? You will still get 6 SATA ports remaining, that are connected directly to the chipset.

Make sure that all of your 6 HDDs are connected to the chipset's SATA ports and then test.

Edited October 12, 2023 by Lolight

[Unfrosted8575]

Quote

Make sure to set Maximum Energy Savings in the power plan.

Where? Software wise on Unraid or on the BIOS?

 

2 hours ago, Lolight said:

Your mobo comes with a Wi-FI chip - disable it in the BIOS. Then remove by unscrewing a couple of screws located underneath and then pull it out of the socket.

Already done.

 

2 hours ago, Lolight said:

Make sure to activate ASPM and enable ErP mode.

DMI Link Speed --> Auto

Need to check those out, can't really remember.

Under which section should I find those?

 

2 hours ago, Lolight said:

Some of you SATA ports are connected to a separate SATA controller (ASMedia) - the bottom 4 SATA ports.

Is there an option to deactivate that controller in the BIOS as a test?

Not sure. Need to check it.

But in any case, I would need more than 6 ports. I am going to add 2 SSD so the total number of port needed will be 8.

If that's may be the problem, can it be solved with some external PCI-e card with SATA ports? If so, do you have any suggestion of some good efficient ones?

 

Thanks.

[Lolight]

12 hours ago, Unfrosted8575 said:

Where? Software wise on Unraid or on the BIOS?

Yeah, you need to go through every available power regulation setting in the BIOS, especially related to CPU power management.

Have you tried to update the BIOS?

I'm assuming you've dowloaded the mobos manual?

Also make sure that your USB Unraid key is connected to a USB 2.0 port

Find out if you can disable ASMedia USB 3.0 HUB controller in the BIOS.

Try to disable all PCIe slots via the onboard PCIe x16 Lane switch.

 

You can try a SATA card.

 

 

[Unfrosted8575]

On 10/13/2023 at 2:31 AM, Lolight said:

Yeah, you need to go through every available power regulation setting in the BIOS, especially related to CPU power management.

I have tried again, I honestly don't what other parameter to change in the BIOS...

 

On 10/13/2023 at 2:31 AM, Lolight said:

Have you tried to update the BIOS?

 

Already update to the latest.

 

On 10/13/2023 at 2:31 AM, Lolight said:

I'm assuming you've dowloaded the mobos manual?

I even have the physical book.

 

On 10/13/2023 at 2:31 AM, Lolight said:

Also make sure that your USB Unraid key is connected to a USB 2.0 port

Just checked and yes, I have a USB2 stick inserted in a USB2 port.

 

On 10/12/2023 at 1:40 PM, Unfrosted8575 said:

Make sure to activate ASPM and enable ErP mode.

DMI Link Speed --> Auto

Checked it again, and I should have set everything.

 

 

On 10/13/2023 at 2:31 AM, Lolight said:

Try to disable all PCIe slots via the onboard PCIe x16 Lane switch.

I have not seen an option in the BIOS to really "disable" PCIe slots, but something that ask if the speed I want is "auto" or "Gen1" or "Gen2", for now it is on auto. Not sure if this matter.

 

 

The only thing I haven't done is to disable one SATA controller as I need the ports, so I am not really sure how would I do. I can maybe buy the SATA card you mentioned and if nothing change, just return it from Amazon.

 

@Lolight Sorry for the late answer, but from the phone, for some reason, I was not able to post an answer. Now that's weekend I set here and did all the test you suggested. If you have any other idea, it would be welcome. I am also willing to take a picture of every page of the BIOS to post if this could help  I really cannot get that damn C2 state.

 

[Unfrosted8575]

On 10/12/2023 at 10:47 AM, Lolight said:

Is there an option to deactivate that controller in the BIOS as a test? You will still get 6 SATA ports remaining, that are connected directly to the chipset.

Make sure that all of your 6 HDDs are connected to the chipset's SATA ports and then test.

Done. I disabled the secondary controller and have all my 6 disks connected to the Intel Z87 SATA controller. No joy.

 

Still stuck on C2 State.

 

Waiting for any update. Tell me if I need to provide any sort of information, screenshot or anything. I'll do. 

[Unfrosted8575]

I also tried booting from a fresh Fedora LIVE USB, installing Powertop, performing auto-tune but still stuck at C2.

 

I think by this point it as obviously something at really low level, like BIOS, or some hardware. But the problem being, I have only HDD inside, so...is it possible that some HDD, even pinned down, can prevent more efficient C state? It sound strange to me.

 

Waiting for answers.

[Lolight]

17 hours ago, Unfrosted8575 said:

I have not seen an option in the BIOS to really "disable" PCIe slots, but something that ask if the speed I want is "auto" or "Gen1" or "Gen2", for now it is on auto. Not sure if this matter

Use the DIP switch.

 

Edited October 22, 2023 by Lolight

[Unfrosted8575]

6 hours ago, Lolight said:

Use the DIP switch.

Oh damn, thanks. I did not understand.

I did it, but no change.

 

I'll leave my powertop output, maybe there is something useful I am not seeing.

 

Let me know if you may need anything else.