Jump to content
demonmaestro

My SuperMicro Build

120 posts in this topic Last Reply

Recommended Posts

You noted this earlier, but I missed it ...

23 minutes ago, demonmaestro said:

Fan 3 = Front of case 3 - 120mm noctua fans

Fan 4 = Middle of case 3 - 120mm noctua fans

Fan 5 = Back 2 - 80mm noctua fans.

 

The specifications for the motherboard indicate it supports "up to 6 fans" using 6 headers => yet you have 11 fans connected.

 

The manual doesn't indicate the wattage limits of these headers, but I wonder if powering 3 fans from a single header may be a bit more power draw than they're designed for.   You have 3 of the headers with multiple fans connected (3 each to Fan 3 & 4 and 2 to Fan 5).    Hard to say if this is contributing to the slightly elevated temperatures; but when you get your Corsair fan controller I'd move the fans from Fan 3 & 4 to the controller and perhaps split the 2 fans currently on Fan 5 to separate headers.

 

Share this post


Link to post

I fully understand your concern for that.

However each noctua burns at max 0.05A. so 0.15A with all 3 per the 2 headers and the 2 - 80mm is at 0.16A together. 

Now the cpu fans burn 0.40A each.

 

http://noctua.at/en/products/fan/nf-f12-pwm/specification

http://noctua.at/en/products/fan/nf-a8-pwm/specification

 

 

**Woot I am at 300 posts*** O.o

Edited by demonmaestro

Share this post


Link to post

With those fan specs, I agree it shouldn't be a problem.   There are plenty of fans that draw 3 times as much current as the Noctua's, so 3 of them aren't any worse than one of the higher-current fans.    I knew the Noctua's were superbly quiet (I have several) ... but had forgotten how low their power consumption was.

 

Share this post


Link to post
15 hours ago, demonmaestro said:

But if that would make the system utilize a fan curve then wouldn't the Fan 1 and Fan 2 ramp up to 100% when the CPUs load up? 

Now the motherboard does not have labeled CPU header fans. It just has Fan headers #1 - #6.

 

I had read about a program "speedfan" but I did try to launch Core Temp to look at actual CPU temps in the windows VM and it had showed 0...

So I don't think speedfan would work unless there is another way?

 

 

@garycase Do you know anything about this?

Also would you think I should put the cpu fans on the controller as well?

Share this post


Link to post

Speedfan, CoreTemp, etc. all require direct hardware access to get the temperature info and/or control the fan speeds via the chipset ... so they aren't going to work in VM's.  (as you found out).

 

An external controller doesn't really work any differently than the built-in ones -- they simply sense temperature (via the thermistors you place at strategic locations in the case of the external one); and control fan voltages based on those temperatures.    There will be some differences in the temperature vs. speed curves, but I don't know any way to determine which units would be "better" at this except to simply try it.   As for the CPU fans ... if you attach those to the external controller you may have to change some BIOS setting so the system doesn't simply shut down due to no detected CPU fan (It's not clear whether your board will do this; but the manual does imply that Fan1 and Fan2 are for CPU fans ... so if it's expecting Fan1 and Fan2 to be CPU fans, it may balk if there aren't any fans detected on those headers).

 

Share this post


Link to post

I don't think it will throw a fit due to the motherboard utilizes a passive heatsink in the initial setup.

 

The controller says that it comes with 4 thermal probes. 

I am thinking about putting them in these locations.

  • Front of case to catch inlet temperature
  • Middle of case to catch temp after HDDs (But before the middle row of 120mm fans.)
  • Back of case before the 80mm fans to see outgoing temperatures 
  • somehow on the CPU heatsink.
Edited by demonmaestro

Share this post


Link to post

Don't see any reason for a probe at the front (inlet temps = ambient room temp, which I doubt will ever be high enough to trigger a fan)

 

Temp after HDD's is a good idea -- you could let it control the fans for the HDDs

 

Unless you've using the controller for the CPU fan, you probably don't need to put a sensor on the CPU heatsink (could be problematic to mount it anyway) ... the built-in CPU temp probe that the motherboard/BIOS uses to control that fan is almost certainly more reliable.

 

Share this post


Link to post

I believe you mentioned you have a fan blowing on the RAID cards -- might want a temp sensor on/by those cards to control that fan.

 

Share this post


Link to post
10 minutes ago, garycase said:

Unless you've using the controller for the CPU fan, you probably don't need to put a sensor on the CPU heatsink (could be problematic to mount it anyway) ... the built-in CPU temp probe that the motherboard/BIOS uses to control that fan is almost certainly more reliable.

 

That is the problem though. The motherboard doesn't ramp up/down the cpu fans with the cpu load/temps.

Share this post


Link to post
Just now, demonmaestro said:

That is the problem though. The motherboard doesn't ramp up/down the cpu fans with the cpu load/temps

Even with the Performance or Balanced setting?

 

Hmmm ... look carefully through the BIOS and see if you have to "tell" it where the CPU fans are attached.    If the default was a passive cooler (as you noted); then it may not be associating CPU temp with a fan (or fans).    I'll have another look at the manual for the board, but I don't believe it shows all of the BIOS details, so you should look around in it and see if there's a setting so it "knows" you have a PWM fan on the CPU.

 

Share this post


Link to post
3 minutes ago, garycase said:

Even with the Performance or Balanced setting?

 

Hmmm ... look carefully through the BIOS and see if you have to "tell" it where the CPU fans are attached.    If the default was a passive cooler (as you noted); then it may not be associating CPU temp with a fan (or fans).    I'll have another look at the manual for the board, but I don't believe it shows all of the BIOS details, so you should look around in it and see if there's a setting so it "knows" you have a PWM fan on the CPU.

 

Yes with both those settings.

I will dig around in the BIOS and get back with you on that one. (Unless you find something in the manual.)

Share this post


Link to post

I looked through the manual and all I could find was this (on Pg 4-19), which clearly implies the speed SHOULD be correlated with the temps on anything except "full speed":

 

Fan Speed Control Modes

This feature allows the user to decide how the system controls the speeds of the onboard fans. The CPU temperature and the fan speed are correlated. When the CPU on-die temperature increases, the fan speed will also increase for effective system cooling. Select "Full Speed/FS" to allow the onboard fans to run at full speed for maximum cooling. The FS setting is recommended for special system configuration or debugging. Select "Performance/PF" for better system cooling. The PF setting is recommended for high-power-consuming and high-density systems. Select "Balanced/BL" for the onboard fans to run at a speed that will balance the needs between system cooling and power saving. The BL setting is recommended for regular systems with normal hardware configurations. Select "Energy Saving/ES" for best power efficiency and maximum quietness. The Options are: Full Speed/FS, Performance/PF, Balanced/BL, and Energy Saving/ES.

 

Share this post


Link to post

Also found this in the manual:

The fan speeds are controlled by Thermal Management via Hardware Monitoring in the Advanced Setting in the BIOS. (The Default setting is Disabled.)

 

I presume it's not disabled anymore; but if so, that would explain why your fans aren't being controlled.    The manual also indicates (in the same section) that Fan 1 and Fan 2 are the headers for the CPU fans.

Share this post


Link to post
26 minutes ago, garycase said:

Also found this in the manual:

The fan speeds are controlled by Thermal Management via Hardware Monitoring in the Advanced Setting in the BIOS. (The Default setting is Disabled.)

 

I presume it's not disabled anymore; but if so, that would explain why your fans aren't being controlled.    The manual also indicates (in the same section) that Fan 1 and Fan 2 are the headers for the CPU fans.

I think this is probably is the issue... I will find out later on when I get near the box and will let you know what I find.

 

I will also look into my other SuperMicro board and see if it has something similar as well. Even though it has a passive heatsink on it. I had put a 60mm Noctua on top of it as well.

Share this post


Link to post

@garycase Looking through the manual and on page 4-4. I don't see anything reguarding to Hardware Monitoring or Thermal Management.

When I go into Hardware Health Configuration (4-17) is where I can set the "Full Speed/FS, Performance/PF, Balanced/BL, and Energy Saving/ES" But I don't see where it talks about thermal management..

Edited by demonmaestro

Share this post


Link to post

The quote I noted above is from Page 2-22 in the section titled "Fan Headers"

 

However, there is NO indication in the chapter on the BIOS about how this is enabled/disabled.   That's why I suggested a careful "look around" in the BIOS to see if you can find it.

 

Might warrant a call/e-mail to SuperMicro if you can't find the option.

 

Interestingly, since these systems shipped with passive heatsinks, that explains why they would have disabled CPU fan speed control ... but I'd still think it'd be an option that can be changed in the BIOS.    I know from experience, however, that these can sometimes be VERY difficult to find (they're not always in what seems a "logical" place :D

 

 

Share this post


Link to post

A few misc thoughts ...

 

From the Table on Page 1-5 (where I noted that fans 1 & 2 are for the CPU fans):

FAN 1-6

System/CPU Fan Headers (Fans 1~2: CPU Fans)

 

From the BIOS hardware health section on page 4-19 (I quoted this earlier) ... the only options shown do NOT include a "Disabled" option:

The Options are: Full Speed/FS, Performance/PF, Balanced/BL, and Energy Saving/ES.

 

So this is apparently NOT the same setting referred to in the quote from page 2-22 that refers to the fan speed control defaulting to "Disabled"

[The fan speeds are controlled by Thermal Management via Hardware Monitoring in the Advanced Setting in the BIOS. (The Default setting is Disabled.) ]

 

I'm at a loss as to where you would change that.   Just for grins, are you running the latest BIOS for that board?  (B16)

 

 

Share this post


Link to post
1 hour ago, garycase said:

I'm at a loss as to where you would change that.   Just for grins, are you running the latest BIOS for that board?  (B16)

 Yes, I am running the latest. That was the first thing that I had checked when I got it. I was initially having issues with the board  with both CPUs in the motherboard.

Come to find out I had taken the motherboard out of the case and it had booted just fine.(although did have a bent pin on the socket). 

So it was somehow grounding it self out on the motherboard mounts. I had then gotten some insulating washers and mounted the motherboard.

Now this fan issue.

It has been one thing after another with this build. I cant complain too much... I got the motherboard, CPUs, and ram all for $400. 

 

The funny thing is the place where I got it, I had told them initially about the motherboard issue when unable to use both CPUs. So they sent me another motherboard. I had gotten the same issue with the "new" motherboard.

 

I have been considering on going with this motherboard/CPUs and getting a whole different system all together. However I am concerned that I will have the same issue...

https://www.newegg.com/Product/Product.aspx?Item=9SIA4S844G6424

https://www.newegg.com/Product/Product.aspx?Item=N82E16819117630

Edited by demonmaestro

Share this post


Link to post

Well ... that would certainly be a NICE upgrade (to newer technology as well as a 40% increase in "horsepower" (Jump in PassMark from 10027 to 14101)  with a CPU that uses 30% less power at max load [TDP 85w vs. a pair of 60w CPUs].   You could also get it for a few hundred less if you just bought a single CPU motherboard; unless you want to retain the ability to add a 2nd CPU in the future.   SuperMicro has some excellent single CPU boards for $300-500  [This is a very attractive one for less than half the price of the dual-CPU board:  https://www.newegg.com/Product/Product.aspx?Item=N82E16813182959 ]

 

... but you won't get it for $400 :D

 

I suspect all of your issues would go away ... but there's no guarantee of course => new tech doesn't always mean "painless tech".

 

Personally, it sounds like your system is running flawlessly except for the cooling issue => and that's not likely a showstopper.   The drive temps look excellent (unless I'm missing something); and the CPU temps are well controlled at higher fan speeds ... so it's just a matter of getting a controller that will suitably adjust the fan speeds as needed to keep the temps in the range you'd like to see.    I would, however, contact SuperMicro to see if there's something we're missing vis-à-vis getting the onboard controller to adjust the CPU fan speeds.    I looked very carefully through the manual to see if this was by some chance a jumper rather than a BIOS setting; but that does not appear to be the case.

 

It's certainly likely, however, that the new gear you listed would resolve this very nicely for an extra grand or so.    The newer boards are generally much more power-efficient than the older ones; so not only would you gain a higher PassMark at lower power; but the motherboard almost certainly draws less as well.    Especially if you switched to a single CPU board.    In addition, DDR4 is lower power RAM.     Assuming your case fans that cool the drives are quiet enough already, that would be a relatively "painless" way to resolve everything ... all it takes is $$$ :D

 

Share this post


Link to post
19 minutes ago, garycase said:

Well ... that would certainly be a NICE upgrade (to newer technology as well as a 40% increase in "horsepower" (Jump in PassMark from 10027 to 14101)  with a CPU that uses 30% less power at max load [TDP 85w vs. a pair of 60w CPUs].   You could also get it for a few hundred less if you just bought a single CPU motherboard; unless you want to retain the ability to add a 2nd CPU in the future.   SuperMicro has some excellent single CPU boards for $300-500  [This is a very attractive one for less than half the price of the dual-CPU board:  https://www.newegg.com/Product/Product.aspx?Item=N82E16813182959 ]

 

... but you won't get it for $400 :D

 

I suspect all of your issues would go away ... but there's no guarantee of course => new tech doesn't always mean "painless tech".

 

I wanna keep the 2 CPUs but also that dual CPU board has quad 10G NICs.O.o

I will contact SuperMicro and see what they say.

 

I guess then the question would be IF I decided to get the new board is what to do with the "old" setup?

 

 

Share this post


Link to post
1 hour ago, demonmaestro said:

I guess then the question would be IF I decided to get the new board is what to do with the "old" setup?

 

It'd make a very nice backup server (if you don't already have one) :D

 

I have 4 servers -- 2 that I use for various purposes, a test box, and a backup server that backs up the other 3.   [I tend to build-up rather than migrating to larger drives ... i.e. my oldest server still has 2TB drives (and a few 1 & 1.5) => but when I needed more space I simply built a new server with 4TB drives].    The backup server has 8TB Seagate archives (and it's only single parity -- at least for now).

 

 

Share this post


Link to post
1 hour ago, demonmaestro said:

I wanna keep the 2 CPUs but also that dual CPU board has quad 10G NICs.O.o

 

Ahh yes, 10Gb !!    A bit of extra icing on the cake :D

 

... and with a pair of those new CPU's, that would REALLY be a "HOT" setup.     It seems a bit pricey until I put it in historical perspective.   I once (~1977) paid $1300 for an 8K (yes, that's a "K") memory card; and a couple years later paid $4500 for a 26GB (not a typo) hard disk ... both for systems that had cost a couple grand.     In today's dollars that's a flipping fortune --  and at the time I was making under $30k, so they were BIG expenses relative to my income.    And of course if those systems were rated with PassMark they'd probably score something like 10 :D

 

Share this post


Link to post
2 hours ago, garycase said:

 

Ahh yes, 10Gb !!    A bit of extra icing on the cake :D

 

According to specs of Supermicro it is 4x 1Gb, but hey that is nice too!

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.