Intel Socket 1151 Motherboards with IPMI AND Support for iGPU


Recommended Posts

9 hours ago, kaiguy said:

Thanks for the heads up, @Hoopster. I've never ran into that before, but I have had an intermittent issue where trying to login to the HTML5 IPMI gui would just keep going back to the login screen. The only way to correct it was to shut down and pull the power from the server, but I might try reflashing the BMC next time I encounter it.

If this happens, you can cold reset the BMC from the command line from your machine. I'm pretty sure it's 

ipmitool mc reset cold

but I could be wrong on that. I've had hte same issue you described with this board before with the BMC login loop and the BMC just needed a reset. Never lost any uptime on the machine itself.

  • Thanks 1
Link to comment

Yeah, I have a similar issue where the IPMI would just keep going back to the login screen as well.  I thought it was something odd in my network setup, but since nothing else seemed to be effected it wasn't high on my priority list to track down.

 

Sounds like this might be the same as what you both listyed above.  I will have to try reflashing next time I take the server down.  Thanks for the heads up!

Link to comment
8 hours ago, ionred said:

If this happens, you can cold reset the BMC from the command line from your machine. I'm pretty sure it's 

ipmitool mc reset cold

but I could be wrong on that. I've had hte same issue you described with this board before with the BMC login loop and the BMC just needed a reset. Never lost any uptime on the machine itself.

@ionred would you do this from the Unraid terminal session?  Also, you mention this can be done while the server is running with no downtime?

Link to comment
5 hours ago, Burizado said:

@ionred would you do this from the Unraid terminal session?  Also, you mention this can be done while the server is running with no downtime?

ipmitool is not installed by default so if you need ipmitool, install it through NerdTools.

 

I did a few things with the BMC with ipmitool but for some reason it was not working directly from a terminal session on the target  machine.  I had to do it remotely with a terminal session from my other Unraid server. This requires the following syntax:

 

ipmitool -I lan -H <ip address of target BMC> -U admin -P <admin pw> <ipmitool commands>

Edited by Hoopster
  • Thanks 1
Link to comment
1 hour ago, Hoopster said:

ipmitool is not installed by default so if you need ipmitool, install it through NerdTools.

 

I did a few things with the BMC with ipmitool but for some reason it was not working directly from a terminal session on the target  machine.  I had to do it remotely with a terminal session from my other Unraid server. This requires the following syntax:

 

ipmitool -I lan -H <ip address of target BMC> -U admin -P <admin pw> <ipmitool commands>

@Hoopster Thanks for the info!  I don't have a test Unraid server, but I could load up one of my RasPis and probably do it from there, if it doesn't work directly on my server.  I'll give it a try once I have my server in a quiet time, in case something goes horribly wrong. 😅

Edited by Burizado
Link to comment
13 hours ago, Burizado said:

@ionred would you do this from the Unraid terminal session?  Also, you mention this can be done while the server is running with no downtime?

Yes this can be accomplished from the unraid terminal, either via web or via ssh to unraid. 
 

Fortunately I did not have to do it from a remote machine like Hoopster, it just “worked”. The BMC takes a couple minutes to fully start back up. By rough memory I’d say it was another 4-5 minutes before I could get to the login screen, but it let me login immediately afterwards without any ugly loops. The unraid system (and ssh session) stayed up the entire time. 

  • Thanks 1
Link to comment

@Hoopster or anyone here do you happen to use bonding mode 802.3ad? I recently upgraded all my equipment and have a unifi 8 port switch with LAG and have set it up based on this guide but I'm always greeted with this message in the logs:

 

bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond

 

I'm having a hell of a time setting up VLANs as well with dockers but slowly getting them to work. Only one that's really giving me trouble is CUPs but I think that's a docker image issue as it only works on the local host (bridge) and refuses to function correctly on vlans.

Link to comment
12 minutes ago, JimmyGerms said:

@Hoopster or anyone here do you happen to use bonding mode 802.3ad? I recently upgraded all my equipment and have a unifi 8 port switch with LAG and have set it up based on this guide but I'm always greeted with this message in the logs:

 

I have that configuration. I have the US-24, but LAG is the same across "almost" their entire switch line.

 

image.png.de764ad468225daabc61fa0a256d8d93.png

 

image.png.8690ec882c039e37a8c8ab5db570b6be.png

 

My config is on VLAN 10

The only message I get is from "Fix Common Errors" because of a complex bonding which I ignore. 

 

It's been flawless since it's been set up.

 

12 minutes ago, JimmyGerms said:
bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond

 

I'm having a hell of a time setting up VLANs as well with dockers but slowly getting them to work. Only one that's really giving me trouble is CUPs but I think that's a docker image issue as it only works on the local host (bridge) and refuses to function correctly on vlans.

 

Link to comment

That bonding makes it so the 2nd adapter is a backup and only becomes active if the primary fails, so unRaid only sees one adapter. There are a number of bonding situations that do not involve a switch. Only 802.3ad will cause issues if you don't have a switch that supports it.

 

So, the short answer is NO, this will NOT cause issues.

Edited by Gonzo-the-great
Link to comment
  • 2 weeks later...

Just a heads up, I had to disable the bonding in IPMI due to my unifi switch getting confused. My local IPMI address would become unavailable due to the link aggregation and bond. Seems to be ok once disabled...and I turned off DNS just to see what would happen. Everything seems fine after a week!

Link to comment
7 minutes ago, Shane01638 said:

I wasn't going to upgrade until the turbo issue was fixed.

I doubt the Turbo Boost issue will ever be fixed. Socket 1151 boards are fairly long in the tooth now (in computer terms). I am not sure ASRock even recognizes it as an issue.

 

I am on BIOS L2.35 and it still exists there.

Link to comment

 

20 hours ago, Hoopster said:

I doubt the Turbo Boost issue will ever be fixed. Socket 1151 boards are fairly long in the tooth now (in computer terms). I am not sure ASRock even recognizes it as an issue.

 

I am on BIOS L2.35 and it still exists there.

 

I am on BIOS L2.40 and it still exists there.

Link to comment
  • 4 weeks later...

PSA for those who have a case that limits CPU cooler height.  I recently discovered the Noctua NH-D12L CPU cooler which has a 145mm height and fits many 4U cases.  Up to this point, the best I could do was the Noctua NH-U9S with a 125mm height and 92mm fans. 

 

My Silverstone CS380 case has a cooler height limit of 146mm.

 

I installed the NH-D12L on my E3C246D4U/E-2288G system and saw immediate results. Idle temps are 3-5C lower and under heavy load, CPU temps are up to 15C lower than with the NH-U9S cooler.  That 120mm fan makes a difference. 😀

 

At US$90, it is an expensive CPU cooler but the results are worth it for me.

Edited by Hoopster
  • Thanks 1
Link to comment

Here is some information from Noctua that helps explain the difference in temps noted in the prior post:

 

The NH-D12L is rated best for overclocking/turbo with the E-2288G (arrow to the far right) as seen below

image.thumb.png.5843ff0962220897a2fe7a02adc26ee0.png

 

The NH-U9S is rated for moderate overclocking/turbo with the E-2288G (arrow in the middle) as seen below

image.thumb.png.01c49f443797c94c7eccf1d0e658d542.png

 

In the NSPR scale Noctua uses to rate cooler performance (rather than a TDP rating), the NH-D12L scores a very respectable 148 and the NH-U9S is rated much lower at 93.

image.thumb.png.621dfdb9b795b142c0dbce5d2e98489c.png

Link to comment
  • 1 month later...

Rehashing the IPMI bonding question earlier on this page... Although I have a network cable plugged into both the IPMI LAN and LAN1 (LAN2 is unused), I notice that my new Unifi switch is associating both MAC addresses with a single port. So I figured I'd just go and break the bonding, but I don't seem to have the "Network Bond Configuration" option in the GUI. I also didn't see anything in the BIOS.

 

848907059_Screenshot2023-06-20at20-37-33ASRockRackIPMI.thumb.png.ee87bc2b47f8aa32274dd92fd4fceb2a.png

 

Any thoughts?

Link to comment
On 6/20/2023 at 9:39 PM, kaiguy said:

Rehashing the IPMI bonding question earlier on this page... Although I have a network cable plugged into both the IPMI LAN and LAN1 (LAN2 is unused), I notice that my new Unifi switch is associating both MAC addresses with a single port. So I figured I'd just go and break the bonding, but I don't seem to have the "Network Bond Configuration" option in the GUI. I also didn't see anything in the BIOS.

 

848907059_Screenshot2023-06-20at20-37-33ASRockRackIPMI.thumb.png.ee87bc2b47f8aa32274dd92fd4fceb2a.png

 

Any thoughts?

IPMI on my E3C246D4U looks just like yours; no network bond configuration under Network Settings.

 

On my E3C246D2I it looks like this:
image.thumb.png.c8310de59191ea64032db3a36486ad73.png

 

However, I am fairly certain I had to run something on the command line to get that to show up.  I'll dredge the depths of my memory (or Google) to see if I can come up with it.  😀

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.