$90 Xeon E5-2670 2.6Ghz (8cores / 16threads)


Recommended Posts

so what GPU are you all using?

Quadros or medium GPUS with 1 slot?

 

i don't wanna play on the server, but a cheap GPU for 50€ wouldn't be that bad, or am i wrong?

Like the GTX 730?

 

Anything new in the 50€ pricerange is useless.

 

Linustechtips for the long explanation,

 

TL;DR

Graphics power wise its overpriced. Assuming Western-Europe and a decent used market in your country you should be able to get a HD6870 for the same money.

Link to comment

I am looking at pulling the trigger on two E5-2670's but i need to find an ATX motherboard in the UK any idea's. Also because this will going in a Fractal Design R5 (No space for huge 4U case not yet anyway) will i be ok cooling wise with two Noctua i4

 

You should be able to import this one from Germany,

http://www.pcking.de/_hardware/asus-z9pa-d8-motherboard-atx-lga2011-socket-c602-a-usb-30.html

Link to comment

I had bought the cpu a couple of months ago, but was waiting for good deals (and wife approval) before I ordered the mobo and the rest.

 

Finally pulled the trigger.

 

$270 Supermicro X9SRL-F-O: http://www.amazon.com/Supermicro-LGA2011-Server-Motherboard-X9SRL-F-O/dp/B007J5SPTM

$40  Supermicro 4U cooler: http://www.amazon.com/Supermicro-Heatsink-Cooling-Systems-SNK-P0050AP4/dp/B007UXPIX4

$87  Norco RPC-470 case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219020

$110 Samsung 64GB (8x8GB) ECC Reg ram: http://www.ebay.com/itm/152033507959

 

Everything except for ram will be delivered today. I'll have a fun weekend :D

Have you tested that the memory works with the X9SRL-F board? I have the same board and need some memory and saw that the memory you bought is not on the supermicro tested memory list.

Link to comment

I had bought the cpu a couple of months ago, but was waiting for good deals (and wife approval) before I ordered the mobo and the rest.

 

Finally pulled the trigger.

 

$270 Supermicro X9SRL-F-O: http://www.amazon.com/Supermicro-LGA2011-Server-Motherboard-X9SRL-F-O/dp/B007J5SPTM

$40  Supermicro 4U cooler: http://www.amazon.com/Supermicro-Heatsink-Cooling-Systems-SNK-P0050AP4/dp/B007UXPIX4

$87  Norco RPC-470 case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219020

$110 Samsung 64GB (8x8GB) ECC Reg ram: http://www.ebay.com/itm/152033507959

 

Everything except for ram will be delivered today. I'll have a fun weekend :D

Have you tested that the memory works with the X9SRL-F board? I have the same board and need some memory and saw that the memory you bought is not on the supermicro tested memory list.

 

I'm using non-ECC DDR3 (currently) it's not on the approved list and it's working fine. It's some old "ECO" RAM that runs at 1.35V baseline.

Link to comment

I had bought the cpu a couple of months ago, but was waiting for good deals (and wife approval) before I ordered the mobo and the rest.

 

Finally pulled the trigger.

 

$270 Supermicro X9SRL-F-O: http://www.amazon.com/Supermicro-LGA2011-Server-Motherboard-X9SRL-F-O/dp/B007J5SPTM

$40  Supermicro 4U cooler: http://www.amazon.com/Supermicro-Heatsink-Cooling-Systems-SNK-P0050AP4/dp/B007UXPIX4

$87  Norco RPC-470 case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219020

$110 Samsung 64GB (8x8GB) ECC Reg ram: http://www.ebay.com/itm/152033507959

 

Everything except for ram will be delivered today. I'll have a fun weekend :D

Have you tested that the memory works with the X9SRL-F board? I have the same board and need some memory and saw that the memory you bought is not on the supermicro tested memory list.

 

I'm using non-ECC DDR3 (currently) it's not on the approved list and it's working fine. It's some old "ECO" RAM that runs at 1.35V baseline.

I have non-ECC memory that is not on the list, but I want to buy some ECC memory :-)

Link to comment

I had bought the cpu a couple of months ago, but was waiting for good deals (and wife approval) before I ordered the mobo and the rest.

 

Finally pulled the trigger.

 

$270 Supermicro X9SRL-F-O: http://www.amazon.com/Supermicro-LGA2011-Server-Motherboard-X9SRL-F-O/dp/B007J5SPTM

$40  Supermicro 4U cooler: http://www.amazon.com/Supermicro-Heatsink-Cooling-Systems-SNK-P0050AP4/dp/B007UXPIX4

$87  Norco RPC-470 case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219020

$110 Samsung 64GB (8x8GB) ECC Reg ram: http://www.ebay.com/itm/152033507959

 

Everything except for ram will be delivered today. I'll have a fun weekend :D

Have you tested that the memory works with the X9SRL-F board? I have the same board and need some memory and saw that the memory you bought is not on the supermicro tested memory list.

Yep. It was picked up right away. Voltage and the timings were recognized correctly. I ran a memtest overnight (only 1.5 passes lol) no issues there

 

The server has been chugging along fine for the last few days (since Sunday)

 

Link to comment

I had bought the cpu a couple of months ago, but was waiting for good deals (and wife approval) before I ordered the mobo and the rest.

 

Finally pulled the trigger.

 

$270 Supermicro X9SRL-F-O: http://www.amazon.com/Supermicro-LGA2011-Server-Motherboard-X9SRL-F-O/dp/B007J5SPTM

$40  Supermicro 4U cooler: http://www.amazon.com/Supermicro-Heatsink-Cooling-Systems-SNK-P0050AP4/dp/B007UXPIX4

$87  Norco RPC-470 case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219020

$110 Samsung 64GB (8x8GB) ECC Reg ram: http://www.ebay.com/itm/152033507959

 

Everything except for ram will be delivered today. I'll have a fun weekend :D

Have you tested that the memory works with the X9SRL-F board? I have the same board and need some memory and saw that the memory you bought is not on the supermicro tested memory list.

Yep. It was picked up right away. Voltage and the timings were recognized correctly. I ran a memtest overnight (only 1.5 passes lol) no issues there

 

The server has been chugging along fine for the last few days (since Sunday)

Thanks for the info  :)

Link to comment

Anybody using a Natex combo with the S2600CP have a link to a guide on bios settings?  What FW version is everyone using?  Looks like there's an update for EFI bios from Feb?

 

So bumping this since there seems to be quite a few posters with the S2600CP motherboard.  Also, if I'm running this board in a non Intel chassis, is there any way to turn off sys fan error logging?  I'm not running a 5th sys fan so I keep getting an error in the BMC.

Link to comment

Anybody using a Natex combo with the S2600CP have a link to a guide on bios settings?  What FW version is everyone using?  Looks like there's an update for EFI bios from Feb?

 

So bumping this since there seems to be quite a few posters with the S2600CP motherboard.  Also, if I'm running this board in a non Intel chassis, is there any way to turn off sys fan error logging?  I'm not running a 5th sys fan so I keep getting an error in the BMC.

 

Did you  run through the BIOS, SDR, and FRU updates?  During that process it'll attempt to automatically detect an Intel Chassis (with specific PS or cooling fan configs), or you can select Other.  If you have all PWM fans, then it should detect those fans that are connected and ignore the ones that aren't.  Keep in mind that I don't have any practical experience with this board in non-Intel chassis, so I may be completely off-base...

 

If that doesn't work, you could try hacking up the SDR file with your specific requirements then load it via the SDRUpdate tool, but I'd suspect that you'll get zero support here as the SDR file values are in unsigned two's compliment.  If you don't know what that is, then you most likely want to avoid monkeying with it.

Link to comment

Anybody using a Natex combo with the S2600CP have a link to a guide on bios settings?  What FW version is everyone using?  Looks like there's an update for EFI bios from Feb?

 

So bumping this since there seems to be quite a few posters with the S2600CP motherboard.  Also, if I'm running this board in a non Intel chassis, is there any way to turn off sys fan error logging?  I'm not running a 5th sys fan so I keep getting an error in the BMC.

 

Did you  run through the BIOS, SDR, and FRU updates?  During that process it'll attempt to automatically detect an Intel Chassis (with specific PS or cooling fan configs), or you can select Other.  If you have all PWM fans, then it should detect those fans that are connected and ignore the ones that aren't.  Keep in mind that I don't have any practical experience with this board in non-Intel chassis, so I may be completely off-base...

 

If that doesn't work, you could try hacking up the SDR file with your specific requirements then load it via the SDRUpdate tool, but I'd suspect that you'll get zero support here as the SDR file values are in unsigned two's compliment.  If you don't know what that is, then you most likely want to avoid monkeying with it.

 

I got my server together and tested it briefly before I had to go on a business trip.  I apologize if I missed some posts on these updates you mention but that's exactly what I'm asking about.  I noticed several different bios versions on Intel site as well as some other updates (like the SDR and FRU you mentioned) but honestly I need a little hand holding.  Is there something like the instructions for updating the Intel SAS cards posted for updating the S2600CP?

 

Edit:  So I missed a few recent posts where you describe updating the board with UEFI FW using a USB stick.  Since the Feb 2016 UEFI FW update includes the BMC, ME and FRUSDR updates, I'll start there and report back any issues.  Thanks again for all the info you've contributed on these Intel components.

Link to comment

Ok, so I completed the update of the BMC, Bios, FRUSDR and ME.  I went through the fan configuration and I'm no longer gettng an amber alert for sysfan5 and sysfan6.  Here's a temp sensor log from the BMC:

 

S2600CP.JPG?raw=1

 

I used a Thermapen to check the temp of the heatsink.  CPU1 is around 94 degrees F and CPU2 is closer to 100 F.  Based on my case configuration, I can see why CPU2 is running hotter.  I think I'm going to need to invest in the CPU coolers.

 

For some reason I can no longer get access to the BMC though.  This seemed to happen after the first reboot once I loaded defaults.  If I don't have the RMM4 module then do I use the same IP address as my UnRAID tower for the BMC IP?  On the old firmware, this is how I had it configured and I was required to login to the BMC before I could get access to the UnRAID GUI.  Is there a way around this?  I tried specifying a port for the BMC but this wasn't allowed in the BMC configuration.

Link to comment

If I recall correctly, both the basebaord NICS are disabled in the BIOS settings by default; if you load the defaults, should verify and/or re-enable (that is, if you wanna use um).

 

In an intel chassis, the BMC uses a closed loop system of monitoring various sensors such as front panel temp, misc physical and virtual temp sensors, airflow sensors, etc to properly maintain system cooling.  ...When the P# Therm Mgn (Processor # thermal margin) falls below 20% (roughly 80c), the BMC will start ramping up the cooling fan(s) RPM serving the CPU domain to bring the margin back above 20%.

 

As you aren't using an Intel chassis, you most likely don't have the front panel temp sensor that the BMC relies upon, so you'll want to investigate a document called the "Fan Control Whitepaper" (that Intel suggests in the P4000/s2600 tech specs doc), you may also want to consider using the PWM Fan offset (value of 15 or 20?) to raise idle speed of your fans (things will get a bit louder as the fans will now be idling higher), but will provide more airflow out of the box.

 

Also what CPU heat sinks are you using currently?  If they are the short ones that Natex supplies, those are entirely unsuitable for most 3rd party chassis applications.  Those are intended to be tightly airducted etc.

 

Edit: regarding the BMC IP address; in the BIOS settings in the System Managment pages, you'll configure which NICS that you want BMC to listen on, and by what IP address.  You'll choose whether to statically assign, or have DHCP allocate.  If you set it statically, make sure it's a unique (unused) IP.  This BMC IP will be different from your Unraid IP even when both are using the baseboard NIC 1 (or 2).

Link to comment

If I recall correctly, both the basebaord NICS are disabled in the BIOS settings by default; if you load the defaults, should verify and/or re-enable (that is, if you wanna use um).

 

In an intel chassis, the BMC uses a closed loop system of monitoring various sensors such as front panel temp, misc physical and virtual temp sensors, airflow sensors, etc to properly maintain system cooling.  ...When the P# Therm Mgn (Processor # thermal margin) falls below 20% (roughly 80c), the BMC will start ramping up the cooling fan(s) RPM serving the CPU domain to bring the margin back above 20%.

 

As you aren't using an Intel chassis, you most likely don't have the front panel temp sensor that the BMC relies upon, so you'll want to investigate a document called the "Fan Control Whitepaper" (that Intel suggests in the P4000/s2600 tech specs doc), you may also want to consider using the PWM Fan offset (value of 15 or 20?) to raise idle speed of your fans (things will get a bit louder as the fans will now be idling higher), but will provide more airflow out of the box.

 

Also what CPU heat sinks are you using currently?  If they are the short ones that Natex supplies, those are entirely unsuitable for most 3rd party chassis applications.  Those are intended to be tightly airducted etc.

 

I went to the NIC management section of the BIOS and it says both Nics are enabled.  Before I lost the web console, I couldn't get a dropdown for the Baseboard Mgmt.  I just confirmed my UnRAID server is online as I can access my shares through my Windows PC.  Any other thoughts on the BMC and the IP settings in the bios?  I'm going to try restarting everything as well.  UnRAID GUI is working now on the server IP address but still no BMC.

 

I think you already noted that the 1U copper heatsinks weren't suitable but then I got some conflicting reports.  I don't know if you noticed but I edited my posts to show the temps were in Fahrenheit.  Anyway, seems foolish at this point not to add more cooling to these CPUs.  Fan noise isn't an issue as my server is located in a separate room so I've got the case fans turned all the way up with the built-in case controller.  Thanks for the info about the fan control whitepaper.  I will check it out.

 

EDIT:  I'm getting quite a few timeouts and waiting for available socket messages using Chrome on PC to try and access the server.  I still cannot get back into the BMC...very frustrating.

 

Link to comment

Skip the cooling hassle, get a P4000 Intel chassis

 

I started with a 120mm fan just laid carelessly over top of both passive coolers from natex. If you have horizontal config like a rackmount it will just lay in place. A vertical tower won't work this way.  But it can work horizontal.

 

But @jwegman introduced me to the P4000 Intel chassis specially made for this Intel 2600cp2 motherboard. For $150 NIB you get an industrial design that can work as a tower or a 4u rackmount and uses the almost free passive coolers effectively.  Once the bios and BMC is updated, it takes these monsters and makes them almost silent, (when compared to servers)  Comes with dual platinum redundant 750w PSU and 10 x 3.5 hotswap drives. You can't mess around with cooling, PSU and case and get anything of this class with so little effort.

 

Highly recommended.  I have 3. 

 

Link to comment

Edit: regarding the BMC IP address; in the BIOS settings in the System Managment pages, you'll configure which NICS that you want BMC to listen on, and by what IP address.  You'll choose whether to statically assign, or have DHCP allocate.  If you set it statically, make sure it's a unique (unused) IP.  This BMC IP will be different from your Unraid IP even when both are using the baseboard NIC 1 (or 2).

 

D*mn it! Somehow I missed this edit!  Makes sense but on the older FW I would get the BMC login after boot and once logged in the UnRAID web gui would appear.

 

Anyway, Im trying to get some kind of output on my CPU temps thru the sensors on the board to report in UnRAID.  I have a thread started here if you have any suggestions?

 

http://lime-technology.com/forum/index.php?topic=47933.msg463061#msg463061

 

I was expecting to see more output from the sensors command?

Link to comment

@jwegman ...

 

Couple of questions to satisfy my curiousity..

 

With a S2600CP2J and a Intel P4308 case and a Intel RS2WC080..

 

1. Should I see the HDD lights on the 3.5" hot-swap cages light up when they are being accessed (mine don't..).  The 8x3.5" backplane is connected to the RS2WC080 (flashed to run in IT mode, no BIOS support).  The SGPIO cable is connected to the motherboard. I'm using the proper intel AXXCBL650MSMS (ie mini-SAS to mini-SAS) cables between the RS2 and the HSBP..

 

2. Using the nice IPMI plugin - if I run ipmi-fru I get

 

FRU Inventory Device: Baseboard (ID 00h)

  FRU Chassis Type: Main Server Chassis
  FRU Chassis Part Number: TR2104
  FRU Chassis Serial Number: FZUP2200381
  FRU Chassis Custom Info: Rackable Systems Inc.

  FRU Board Manufacturing Date/Time: 08/29/13 - 10:36:00
  FRU Board Manufacturer: Intel Corporation
  FRU Board Product Name: S2600CP
  FRU Board Serial Number: QSCP33202015
  FRU Board Part Number: G56797-510
  FRU FRU File ID: FRU Ver 1.00

  FRU Product Manufacturer Name: Intel Corporation
  FRU Product Name: S2600CP
  FRU Product Part/Model Number: AMZ-CYPRESS-CR2-0004
  FRU Product Version: ....................
  FRU Product Serial Number: A00A432A
  FRU Product Asset Tag: R200001

FRU Inventory Device: front panel (ID 04h)

  FRU Board Manufacturing Date/Time: 02/27/12 - 13:23:00
  FRU Board Manufacturer: Intel Corporation
  FRU Board Product Name: FFPANEL
  FRU Board Serial Number: QSBT20802163
  FRU Board Part Number: G10279-401
  FRU FRU File ID: FRU Ver 0.01

FRU Inventory Device: Pwr Supply 1 FRU (ID 02h)

  FRU Product Manufacturer Name: DELTA
  FRU Product Name: DPS-750XB A
  FRU Product Part/Model Number: E98791-006
  FRU Product Version: 01  
  FRU Product Serial Number: E98791D1148018137

  FRU Power Supply Overall Capacity: 750 Watts
  FRU Power Supply Peak VA: 1744 VA
  FRU Power Supply Max Inrush Current: 55 Amps
  FRU Power Supply Inrush Interval: 5 ms
  FRU Power Supply Low End Input Voltage 1: 90000 mV
  FRU Power Supply High End Input Voltage 1: 140000 mV
  FRU Power Supply Low End Input Voltage 2: 180000 mV
  FRU Power Supply High End Input Voltage 2: 264000 mV
  FRU Power Supply Low End Acceptable Frequency: 47 Hz
  FRU Power Supply High End Acceptable Frequency: 63 Hz
  FRU Power Supply A/C Dropout Tolerance: 20 ms
  FRU Power Supply Predictive Fail Support: Yes
  FRU Power Supply Predictive Fail: Tach output, two pulses per rotation
  FRU Power Supply Power Factor Correction Supported: Yes
  FRU Power Supply AutoSwitch Supprt: Yes
  FRU Power Supply Hot Swap Support: Yes
  FRU Power Supply Peak Capacity: 850 Watts
  FRU Power Supply Hold Up Time: 12 s
  FRU Power Supply Voltage 1: 12V
  FRU Power Supply Voltage 2: 12V
  FRU Power Supply Total Combined Wattage: 0 Watts

  FRU DC Output Output Number: 1
  FRU DC Output Output on Standy: No
  FRU DC Output Nominal Voltage: 12000 mV
  FRU DC Output Maximum Negative Voltage Deviation: 11400 mV
  FRU DC Output Maximum Positive Voltage Deviation: 12600 mV
  FRU DC Output Ripple and Noise pk-pk: 120 mV
  FRU DC Output Minimum Current Draw: 0 mA
  FRU DC Output Maximum Current Draw: 62000 mA

  FRU DC Output Output Number: 2
  FRU DC Output Output on Standy: Yes
  FRU DC Output Nominal Voltage: 12000 mV
  FRU DC Output Maximum Negative Voltage Deviation: 11400 mV
  FRU DC Output Maximum Positive Voltage Deviation: 12600 mV
  FRU DC Output Ripple and Noise pk-pk: 120 mV
  FRU DC Output Minimum Current Draw: 0 mA
  FRU DC Output Maximum Current Draw: 1500 mA

FRU Inventory Device: HS Backplane 1 (ID 05h)

  FRU Error: board info area checksum invalid

 

Should I be concerned about the Backplane 1 error?  I'm guessing that the backplane for the drives?

 

I've applied all the latest firmware/bios/fru updates as per your excellent instruction (so FRU is at v 1.11).

 

I've found a c200_c600 enclosure management cabling guide and that seems to imply that I don't need the sgpio cable (as the mini-sas connectors contain that cabling) - is that the cause of the issue?

 

EDIT: Hmm..

 

If I run ipmi-fru -skipchecks I do get

 

FRU Inventory Device: HS Backplane 1 (ID 05h)

  FRU Board Manufacturing Date/Time: 04/25/12 - 16:50:00
  FRU Board Manufacturer: Intel Corporation
  FRU Board Product Name: FUP8X35HSBP
  FRU Board Serial Number: QSCP21603368
  FRU Board Part Number: G15584-301
  FRU FRU File ID: FRU Ver 0.01

 

Wonder if it's possible to fix the checksum error.. (that the skipchecks ignores)

 

Thanks

Chris

Link to comment

@jwegman ...

 

Couple of questions to satisfy my curiousity..

 

With a S2600CP2J and a Intel P4308 case and a Intel RS2WC080..

 

1. Should I see the HDD lights on the 3.5" hot-swap cages light up when they are being accessed (mine don't..).  The 8x3.5" backplane is connected to the RS2WC080 (flashed to run in IT mode, no BIOS support).  The SGPIO cable is connected to the motherboard. I'm using the proper intel AXXCBL650MSMS (ie mini-SAS to mini-SAS) cables between the RS2 and the HSBP..

...

Should I be concerned about the Backplane 1 error?  I'm guessing that the backplane for the drives?

 

Yes, you should see the drive activity LEDs light up when they are accessed.

 

When you state that you have the spgio cable from the HSBP to the mobo, do you actually mean you have the i2c cable connected?  The i2c cable allows the HSBP firmware updates during the bios/me/fru/sdr update.  Did you have this connected when running through the BIOS/ME/SDR/FRU updates so that the HSBP fw could be updated?

 

...I had one OCZ SSD that wouldn't trigger the enclosure's activity LED until I upgraded that SSD's FW...

 

FYI, I get the same checksum error when querying

FRU Inventory Device: HS Backplane 1 (ID 05h)

  FRU Error: board info area checksum invalid

 

...when issuing the -skipcheck param:

FRU Inventory Device: HS Backplane 1 (ID 05h)

  FRU Board Manufacturing Date/Time: 04/22/12 - 15:58:00
  FRU Board Manufacturer: Intel Corporation
  FRU Board Product Name: FXX8X25HSBP
  FRU Board Serial Number: QSRU21504111
  FRU Board Part Number: G15232-451
  FRU FRU File ID: FRU Ver 0.02

FRU Inventory Device: HS Backplane 2 (ID 06h)

  FRU Board Manufacturing Date/Time: 09/18/14 - 03:48:00
  FRU Board Manufacturer: Intel Corporation
  FRU Board Product Name: FUP8X35HSBP
  FRU Board Serial Number: QSCP43801644
  FRU Board Part Number: G15584-301
  FRU FRU File ID: FRU Ver 0.01

Link to comment

Glad you are seeing the same ipmi output as me - I'll quit worrying about that one :)

 

As for the drive lights - yes i probably did mean the i2c cable - the 3 wire one that connects from the lower right (looking from the back) corner of the backplane to the motherboard (over by the PCIe slot..). It's been connected since day 1 - 7 3.5" drives attached (4 different models) and not a single one shows a light..

 

I tried updating the FRU/BMC etc again today and no change.

 

Does anyone else with this combo get the HDD led activity indicators (or not as the case maybe..)?

 

NB: The HD indicator in the top 'status' panel lights up - just not the ones on the HS drive bays..

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.