Dual Xeon E5-2670 build


17 posts in this topic Last Reply

Recommended Posts

So I've been asking a lot of questions lately in the Good Deals section and thought it might be better to start a new build thread since I know others are in a similar situation.

 

With my upgrade to UnRAID v6, I realized my current server is underpowered for my current (and future) needs.  I'm currently using Sonarr, SabNZBd, Emby Media Server and Deluge for dockers.  I quickly discovered that 2GB of RAM is barely enough to run all the dockers at the same time and forget about transcoding any of my content to other devices.  I also want to create a Win10 VM and eliminate my desktop PC.

 

Anyway, I jumped on the recent E5-2670 bandwagon and purchased a motherboard/cpu/ram combo from Natex.

 

Here's my equipment list thus far:

 

Case:  Lian Li PC-P80 (confirmed SSI-EEB MB fits my removable MB mounting plate with some small tweaks), 3 140mm intake fans and 2 exhaust (1 rear and 1 top)

Drive Cages: 4x 3in4 Lian LI EX-H34

PSU:  EVGA 850W Supernova P2 Platinum

MB/CPU/RAM:  Intel S2600CP LGA 1100 with dual Xeon E5-2670's and 128GB RAM

Controllers:  Dual Intel RS2WC080 flashed to IT

Drives:  14 total now, 2TB-4TB combination of Seagate, Toshiba and HGST

 

The motherboard combo from Natex comes with two passive copper heatsinks.  Given the relatively low investment for the MB/CPU/RAM, I thought it might be prudent to upgrade these to active cooling.  I am considering 2x Noctua NH U9DX I4 CPU Heatsink and Cooling Fans.

 

Being new to IPMI, I think I'll be able to monitor temps actively with my new configuration to see if additional CPU cooling is necessary (guessing it will be but thought as long as I don't get too hot it would be worth checking before I invest in the Noctua's).  I'm also wondering about the orientation of the CPU fans (i.e. intake from bottom and exhaust upwards or intake from front of case and exhaust to rear)?  The latter scenario has me wondering if I'll just be pulling hot air from CPU1 into CPU2 given their close proximity?

 

Here's a layout of my case configuration:

 

http://www.lian-li.com/en/dt_portfolio/armorsuit-pc-p80/

 

Any other suggestions on plugins/dockers that might help me monitor the status of the server would be appreciated.  Currently, I haven't found a way to get CPU temp info on my Asus P5Q Premium board so I'm hoping this will be resolved with the new board/bios/IPMI configuration?

Link to post

128MB RAM?  "640KB ought to be enough for anyone."

 

Of course I realize you mean 128GB.

 

On a normal Linux machine, the 'sensors' command would work for reporting CPU temperatures.  I tried this on unRAID and apparently it relies on Perl, which isn't installed.

 

I know it's roundabout but you could install some other Linux onto your machine (that has a working 'sensors' command), then run some torture tests, monitor CPU temperatures, adjust cooling, etc. . . . *then* install unRAID with whatever cooling solution the above approach yielded, with confidence that it will suffice.

 

No doubt someone else knows a better way to do it, that's just one way I can see.

Link to post

After some searching I found this page:

 

http://lime-technology.com/wiki/index.php/Setting_up_CPU_and_board_temperature_sensing

 

After following those directions, I can see my CPU temperatures now (among others):

 

root@Luna:/mnt# sensors

coretemp-isa-0000

Adapter: ISA adapter

Core 0:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 1:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 2:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

Core 3:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

 

jc42-i2c-0-18

Adapter: SMBus I801 adapter at e000

temp1:        +33.1°C  (low  =  +0.0°C)                  ALARM (HIGH, CRIT)

                      (high =  +0.0°C, hyst =  +0.0°C)

                      (crit =  +0.0°C, hyst =  +0.0°C)

 

jc42-i2c-0-1a

Adapter: SMBus I801 adapter at e000

temp1:        +35.8°C  (low  =  +0.0°C)                  ALARM (HIGH, CRIT)

                      (high =  +0.0°C, hyst =  +0.0°C)

                      (crit =  +0.0°C, hyst =  +0.0°C)

 

nct6776-isa-0290

Adapter: ISA adapter

Vcore:          +0.74 V  (min =  +0.00 V, max =  +1.74 V)

in1:            +0.17 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

AVCC:          +3.34 V  (min =  +2.98 V, max =  +3.63 V)

+3.3V:          +3.33 V  (min =  +2.98 V, max =  +3.63 V)

in4:            +0.56 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

in5:            +1.86 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

in6:            +1.68 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

3VSB:          +3.46 V  (min =  +2.98 V, max =  +3.63 V)

Vbat:          +3.31 V  (min =  +2.70 V, max =  +3.63 V)

fan1:            0 RPM  (min =    0 RPM)

fan2:            0 RPM  (min =    0 RPM)

SYSTIN:        +45.0°C  (high =  +0.0°C, hyst =  +0.0°C)  ALARM  sensor = thermistor

CPUTIN:        +40.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor

AUXTIN:        -12.5°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor

PCH_CHIP_TEMP:  +0.0°C

PCH_CPU_TEMP:    +0.0°C

PCH_MCH_TEMP:    +0.0°C

cpu0_vid:      +0.000 V

intrusion0:    OK

intrusion1:    OK

beep_enable:  disabled

 

The big piece that was missing was Perl, which the directions show you how to install.

 

All I have left to do is augment my 'go' script so it still works after a reboot.

 

Link to post

Being new to IPMI, I think I'll be able to monitor temps actively with my new configuration to see if additional CPU cooling is necessary (guessing it will be but thought as long as I don't get too hot it would be worth checking before I invest in the Noctua's).  I'm also wondering about the orientation of the CPU fans (i.e. intake from bottom and exhaust upwards or intake from front of case and exhaust to rear)?  The latter scenario has me wondering if I'll just be pulling hot air from CPU1 into CPU2 given their close proximity?

From my testing, I've found CPU Cooler orientation makes no discernible variation on CPU0 + CPU1 temps, (As long as you have good case airflow.)

Link to post

I'm certainly no cooling expert but "if it were me" I'd try to make it work with the passive heatsinks, and just increase airflow through the case (if necessary).

 

Those are some nice server boards over there at Natex, value-wise.  I'm tempted to pick one up, torn between that and an LGA 2011 v3 build.

 

 

Link to post

Yeah, I'm working on getting the CPU temps setup.

 

Very pleased with the Natex combo.  I still have a lot to figure out.  My transfer speeds are still capping around 40-50 MB/sec (previously 30-35).  Not sure what else I need to tweak though?!

Link to post
My transfer speeds are still capping around 40-50 MB/sec (previously 30-35).

 

I suspect that's just about the drives.  For 4TB drives those aren't bad speeds.  If your parity drive is 7200 RPM and you have a matching data drive (same make/model), that's the write speed to test that will yield best results.  Otherwise you're limited by the slower of the two drives involved (parity/data).  Also note that writing to the beginning of a drive (outer cylinders) is as much as twice as fast as writing to the end.

 

 

Link to post

Yeah, I'm working on getting the CPU temps setup.

 

Very pleased with the Natex combo.  I still have a lot to figure out.  My transfer speeds are still capping around 40-50 MB/sec (previously 30-35).  Not sure what else I need to tweak though?!

 

What are you running for CPU Coolers?  I've seen reference to the Noctua I4 having issues with fan clearance over the RAM sticks. 

Link to post

After some searching I found this page:

 

http://lime-technology.com/wiki/index.php/Setting_up_CPU_and_board_temperature_sensing

 

After following those directions, I can see my CPU temperatures now (among others):

 

root@Luna:/mnt# sensors

coretemp-isa-0000

Adapter: ISA adapter

Core 0:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 1:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 2:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

Core 3:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

 

jc42-i2c-0-18

Adapter: SMBus I801 adapter at e000

temp1:        +33.1°C  (low  =  +0.0°C)                  ALARM (HIGH, CRIT)

                      (high =  +0.0°C, hyst =  +0.0°C)

                      (crit =  +0.0°C, hyst =  +0.0°C)

 

jc42-i2c-0-1a

Adapter: SMBus I801 adapter at e000

temp1:        +35.8°C  (low  =  +0.0°C)                  ALARM (HIGH, CRIT)

                      (high =  +0.0°C, hyst =  +0.0°C)

                      (crit =  +0.0°C, hyst =  +0.0°C)

 

nct6776-isa-0290

Adapter: ISA adapter

Vcore:          +0.74 V  (min =  +0.00 V, max =  +1.74 V)

in1:            +0.17 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

AVCC:          +3.34 V  (min =  +2.98 V, max =  +3.63 V)

+3.3V:          +3.33 V  (min =  +2.98 V, max =  +3.63 V)

in4:            +0.56 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

in5:            +1.86 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

in6:            +1.68 V  (min =  +0.00 V, max =  +0.00 V)  ALARM

3VSB:          +3.46 V  (min =  +2.98 V, max =  +3.63 V)

Vbat:          +3.31 V  (min =  +2.70 V, max =  +3.63 V)

fan1:            0 RPM  (min =    0 RPM)

fan2:            0 RPM  (min =    0 RPM)

SYSTIN:        +45.0°C  (high =  +0.0°C, hyst =  +0.0°C)  ALARM  sensor = thermistor

CPUTIN:        +40.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor

AUXTIN:        -12.5°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor

PCH_CHIP_TEMP:  +0.0°C

PCH_CPU_TEMP:    +0.0°C

PCH_MCH_TEMP:    +0.0°C

cpu0_vid:      +0.000 V

intrusion0:    OK

intrusion1:    OK

beep_enable:  disabled

 

The big piece that was missing was Perl, which the directions show you how to install.

 

All I have left to do is augment my 'go' script so it still works after a reboot.

 

So when I run the sensors cmd, all I get is this:

 

Tower login: root
Linux 4.1.18-unRAID.
root@Tower:~# sensors
i350bb-pci-0700
Adapter: PCI adapter
loc1:         +55.0°C  (high = +120.0°C, crit = +110.0°C)

root@Tower:~#

 

Link to post
  • 2 weeks later...

After some searching I found this page:

 

http://lime-technology.com/wiki/index.php/Setting_up_CPU_and_board_temperature_sensing

 

After following those directions, I can see my CPU temperatures now (among others):

 

root@Luna:/mnt# sensors

coretemp-isa-0000

Adapter: ISA adapter

Core 0:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 1:      +33.0°C  (high = +98.0°C, crit = +98.0°C)

Core 2:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

Core 3:      +32.0°C  (high = +98.0°C, crit = +98.0°C)

 

I'm still struggling with getting temps setup on my Intel S2600CP board. I can't download the latest version of sensors-detect because lm-sensors.org is down. Any other suggestions?

Link to post

I just used the sensors-detect that's already there (in /usr/sbin/).

 

All I was missing was Perl . . . there are directions for installing Perl on that page I linked to.

 

I notice now that the Perl installation didn't stick . . . it's now not installed, presumable since rebooting the server.  It's okay though because you only need it long enough to run sensors-detect.

 

 

Link to post

I just used the sensors-detect that's already there (in /usr/sbin/).

 

All I was missing was Perl . . . there are directions for installing Perl on that page I linked to.

 

I notice now that the Perl installation didn't stick . . . it's now not installed, presumable since rebooting the server.  It's okay though because you only need it long enough to run sensors-detect.

 

Ok, I think it finally get it.  I installed the Dynamix System Temp plugin but not Perl.  I didn't realize the "detect" on the plugin required Perl as I thought it was only required for sensors-detect.  Thanks!

Link to post

I just used the sensors-detect that's already there (in /usr/sbin/).

 

All I was missing was Perl . . . there are directions for installing Perl on that page I linked to.

 

I notice now that the Perl installation didn't stick . . . it's now not installed, presumable since rebooting the server.  It's okay though because you only need it long enough to run sensors-detect.

 

Well, I installed Perl and ran sensors-detect.  This is the output:

 

# sensors-detect revision 6170 (2013-05-20 21:25:22 +0200)
# System: Rackable Systems Inc. CYPRESS11 [..................]
# Board: Intel Corporation S2600CP

This program will help you determine which kernel modules you need
to load to use lm_sensors most effectively. It is generally safe
and recommended to accept the default answers to all questions,
unless you know what you're doing.

Some south bridges, CPUs or memory controllers contain embedded sensors.
Do you want to scan for them? This is totally safe. (YES/no): YES
Silicon Integrated Systems SIS5595...                       No
VIA VT82C686 Integrated Sensors...                          No
VIA VT8231 Integrated Sensors...                            No
AMD K8 thermal sensors...                                   No
AMD Family 10h thermal sensors...                           No
AMD Family 11h thermal sensors...                           No
AMD Family 12h and 14h thermal sensors...                   No
AMD Family 15h thermal sensors...                           No
AMD Family 15h power sensors...                             No
AMD Family 16h power sensors...                             No
Intel digital thermal sensor...                             Success!
    (driver `coretemp')
Intel AMB FB-DIMM thermal sensor...                         No
VIA C7 thermal sensor...                                    No
VIA Nano thermal sensor...                                  No

Some Super I/O chips contain embedded sensors. We have to write to
standard I/O ports to probe them. This is usually safe.
Do you want to scan for Super I/O sensors? (YES/no): YES
Probing for Super-I/O at 0x2e/0x2f
Trying family `National Semiconductor/ITE'...               No
Trying family `SMSC'...                                     No
Trying family `VIA/Winbond/Nuvoton/Fintek'...               No
Trying family `ITE'...                                      No
Probing for Super-I/O at 0x4e/0x4f
Trying family `National Semiconductor/ITE'...               No
Trying family `SMSC'...                                     No
Trying family `VIA/Winbond/Nuvoton/Fintek'...               No
Trying family `ITE'...                                      No

Some systems (mainly servers) implement IPMI, a set of common interfaces
through which system health data may be retrieved, amongst other things.
We first try to get the information from SMBIOS. If we don't find it
there, we have to read from arbitrary I/O ports to probe for such
interfaces. This is normally safe. Do you want to scan for IPMI
interfaces? (YES/no): YES
Found `IPMI BMC KCS' at 0xca2...                            Success!
    (confidence 8, driver `to-be-written')

Some hardware monitoring chips are accessible through the ISA I/O ports.
We have to write to arbitrary I/O ports to probe them. This is usually
safe though. Yes, you do have ISA I/O ports even if you do not have any
ISA slots! Do you want to scan the ISA I/O ports? (YES/no): YES
Probing for `National Semiconductor LM78' at 0x290...       No
Probing for `National Semiconductor LM79' at 0x290...       No
Probing for `Winbond W83781D' at 0x290...                   No
Probing for `Winbond W83782D' at 0x290...                   No

Lastly, we can probe the I2C/SMBus adapters for connected hardware
monitoring devices. This is the most risky part, and while it works
reasonably well on most systems, it has been reported to cause trouble
on some systems.
Do you want to probe the I2C/SMBus adapters now? (YES/no): no
Now follows a summary of the probes I have just done.
Just press ENTER to continue:

Driver `to-be-written':
  * ISA bus, address 0xca2
    Chip `IPMI BMC KCS' (confidence: 

Driver `coretemp':
  * Chip `Intel digital thermal sensor' (confidence: 9)

Note: there is no driver for IPMI BMC KCS yet.
Check http://www.lm-sensors.org/wiki/Devices for updates.

 

I still can't get the Dynamix GUI to auto detect coretemp so I can load it.  I noticed in the terminal output the following when I hit the detect button:

 

grep: /etc/sensors.d/sensors.conf: No such file or directory
yes: standard output: Broken pipe
yes: write error
grep: /etc/sensors.d/sensors.conf: No such file or directory

 

Did you use the Dynamix plugin to load coretemp?

 

EDIT:  After a few minutes I tried the Detect button again and now I have the following options:

 

Sensors.JPG?raw=1

Link to post
  • 2 weeks later...
  • 4 years later...

Resurrecting this thread for any S2600cp owners out there. Just wondering what bios everyone is running? My server is currently on 86B.02.06.0002 but I’ve seen updates to 02.06.0006, .0006rev2 and .0007. Not positive if these apply to my board or not?  Also interested in updating the SELViewer. My system event logging is showing off in IPMI so I guess I need to dig into the BIOS. It seems my event log was full and stopped recording events (unbeknownst to me).

 

Edit: upon further investigation, it looks like these new BIOS revisions are for 60X chipsets.

 

Finally, has anyone upgraded their system from the s2600cp? If so, to what?

Edited by betaman
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.