Advice for Motherboard - Asus Z97 vs ASRock C226WS+

[limefrog]

Hi everyone,

 

I'm a noob to servers and building computers. I've done serious homework in the last week but I still need the help of experienced unRAIDers to choose a motherboard. My #1 priority when building my unRAID server is future-proofing. My short-term use will be Plex, but I want the option to do all the other fancy stuff like virtualization, etc.

 

CPU Choice: Intel Xeon E3-1241 v3 @ 3.50GHz because it has a high value for a Xeon chip > 10,000 Passmark

Case: Norco RPC-4224 because I can expand to 24 drives in the future.

 

Motherboard: I don't know what makes the ASRock C226WS+ a "server" motherboard and why the Asus Z97 is not considered a "server" motherboard (based on NewEgg). On paper, the Asus Z97 seems to be the winner (because of the additional 3 PCI slots and the 2 SATA express ports) and is only $10 more expensive than the ASRock. ASRock wins because it has native capacity for 4 additional SATA drives than the Asus Z97. The SATA express ports seem appealing to me because I'd be able to hook them up to SSDs, since SSDs will eventually be limited to the speeds of SATA3.

 

Asus Z97 ($279.99)

-4 PCI 3.0 (x8/x8/x8/x8)

-3 PCI 2.0 (x4/x1/x1)

-4 SATA 3

-2 SATAe

-M.2 Socket 3

 

ASRock C226WS+ ($269.99)

-4 PCI 3.0 (x8/x8/x8/x8)

-6 SATA 3

-4 SATA 3 (through Marvell SE9230)

 

If any of my logic is off-mark, please kindly educate me. Thank you for your help folks.

[c3]

The Asus Z97 is a desktop motherboard because it has features like overclocking, and does not support ECC memory.

 

ASRock C226WS+ is a server board because it supported Xeon processors and ECC.

 

Desktop motherboards often have extra shiney things, bells and whistles. While servers are often lacking such features. Servers have only what is needed to run and run and run, extra bits are just more things to consume power and fail. Server motherboards are designed to do a job well. For example, between the two boards you listed, both have -4 PCI 3.0 (x8/x8/x8/x8), but the server board is laid out so all of them can be used by double wide cards.

[garycase]

The Asus Z97-WS is a nice board, but it's based on the consumer-grade Z97 chipset, which does not support ECC memory, so it's not considered a server-grade board like the AsRock C226-based board.

 

The C226WS+ board is a very nice board with more onboard SATA ports than the Asus board and ECC support.  I think it's a far better choice for UnRAID than a consumer class board like the Asus.  I can't imagine that you need more than 4 PCIe x16 slots; and while the SATA express and M.2 slots on the Asus can indeed provide PCIe bandwidth to SATAe or M.2 devices with PCIe capability, you can get this same capability by simply buying a PCIe SSD.    In an UnRAID environment it's VERY unlikely that the 600MB/s bandwidth of a SATA=3 SSD is likely to ever be a performance bottleneck !!    [These aren't going to be array drives; and that's plenty of bandwidth for a cache or application drive.]    ECC support is a far nicer thing to have than the M.2/SATAe features.

 

By the way, you may want to consider using the E3-1246v3 instead of the 1241 => nearly identical performance PassMark 9896 vs 10041) but also has integrated video, so you won't need to add a video card to support a console.

 

 

 

[ohlwiler]

Both boards are called "workstation boards". Both would make nice foundations for a server. Neither would make a great foundation. Which way you should go comes down to VMs and what you want to do with them. If you want multiple GPUs then that is beyond my expertise, but I do know that you will run out of high speed PCIe slots. If you want to run a few Dockers and some less extravagant VMs then I think you will save money prodding the path most choose and that is to go with a Supermicro board.

 

Having 10 SATA ports is only useful if they are not choked by a DMI 2.0 path, which is the situation with the Asrock. I didn't verify, but I am almost positive the Asus PCIe 2.0 slots are also throttled by a DMI 2.0 path, making them useless in a high bandwidth situation. In my mind ECC memory is required for this level of a server, so that eliminates the Asus (which is nothing more than a gussied up Z97-A, which I own and paid less than a $100 for)

 

In order to support 24 drives at high speed, the requirements are fairly simple. You need 2-3 PCIe 8x 2.0 slots. 2-3 HBAs (IBM M1015, Dell PERC H310 etc.), 16-32GB of ECC memory and as much Xeon as you feel you need. If your needs aren't too outlandish, you can get away with something like a Supermicro X10SLL-F-O, Xeon E3-12xx, 32 G. ECC and (3) IBM M1015. This is the direction most people go today that are aiming for 24 drives in a Norco. It is the direction I would choose if starting from scratch.

 

How much processor do you want? What do you need your VMs to do? You must answer those questions before we can recommend further.

[limefrog]

Folks, thanks for the informative education and help!

 

The Asus Z97 is a desktop motherboard because it has features like overclocking, and does not support ECC memory.

 

ASRock C226WS+ is a server board because it supported Xeon processors and ECC.

 

I overlooked this obvious and very important fact, thank you!

 

In an UnRAID environment it's VERY unlikely that the 600MB/s bandwidth of a SATA=3 SSD is likely to ever be a performance bottleneck !!    [These aren't going to be array drives; and that's plenty of bandwidth for a cache or application drive.]

 

This is exactly the type of logic I needed to hear! I'll stick with the ASRock C226WS+.

 

By the way, you may want to consider using the E3-1246v3 instead of the 1241 => nearly identical performance PassMark 9896 vs 10041) but also has integrated video, so you won't need to add a video card to support a console.

 

Thank you! You saved me the hassle of buying a cheap video card.

 

Neither would make a great foundation. Which way you should go comes down to VMs and what you want to do with them ... I do know that you will run out of high speed PCIe slots. If you want to run a few Dockers and some less extravagant VMs then I think you will save money prodding the path most choose and that is to go with a Supermicro board.

 

Thank you for the info, do you know of any specific Supermicro boards that would make a great foundation? The top of the line Supermicro board for the LGA 1150 is the X10SAE, and the specs on it don't look as good as the ASRock C226WS+ (but I may be looking at it wrong).

 

Having 10 SATA ports is only useful if they are not choked by a DMI 2.0 path, which is the situation with the Asrock.

 

I have some more homework to do, I have no idea what a DMI 2.0 is.

 

How much processor do you want? What do you need your VMs to do? You must answer those questions before we can recommend further.

 

I want to be able to use my unRAID for Plex, Sickbeard, Couchpotato, torrent server, and in the future, perhaps use it for SteamOS. I'm not too sure how much processor I want or need, so I guess that's my first problem.

[garycase]

I think you'd be quite happy with the AsRock and an E3-1246v3.

 

The DMI v2.0 "restriction" is that it limits communications between the CPU & the peripherals to 2GB/s (i.e. 2000MB/s).    This is not going to bottleneck 10 traditional SATA drives ... indeed I can't imagine many situations in which you'd even be aware of this limit.

 

As for "how much processor you want" => any CPU with a PassMark in the 10,000 range is going to have PLENTY of "horsepower" for what you've outlined.

 

If you want higher I/O bandwidth, you'd really need to step up to a socket 2011v3 board with an E5 series Xeon.  The "sweet spot" in terms of performance gains vs. cost in this series is probably the E5-1650v3  (about $600 for a CPU with 6 cores and a PassMark of 13445).    The 2011-based motherboards will have more than twice the PCIe bandwidth ... the 1150 boards support a total of 16 lanes;  a 2011v3 board supports 28-40 lanes (Depending on the CPU - the E5-1650v3 supports 40).

 

An E5-based system will have several advantages ... higher I/O bandwidth;  buffered memory w/ECC;  higher performance; etc. => but it will add several hundred $$ to the cost of the system.

 

 

 

 

[limefrog]

The DMI v2.0 "restriction" is that it limits communications between the CPU & the peripherals to 2GB/s (i.e. 2000MB/s).    This is not going to bottleneck 10 traditional SATA drives ... indeed I can't imagine many situations in which you'd even be aware of this limit.

 

Roger. Thanks for the education.

 

If you want higher I/O bandwidth, you'd really need to step up to a socket 2011v3 board with an E5 series Xeon.  The "sweet spot" in terms of performance gains vs. cost in this series is probably the E5-1650v3  (about $600 for a CPU with 6 cores and a PassMark of 13445).

 

Indeed you are correct! I took a quick look at https://www.cpubenchmark.net/CPU_mega_page.html and found the E5-1650v3 to have the best CPU value > 13,000 passmark.

 

The 2011-based motherboards will have more than twice the PCIe bandwidth ... the 1150 boards support a total of 16 lanes;  a 2011v3 board supports 28-40 lanes (Depending on the CPU - the E5-1650v3 supports 40).

 

An E5-based system will have several advantages ... higher I/O bandwidth;  buffered memory w/ECC;  higher performance; etc. => but it will add several hundred $$ to the cost of the system.

 

Thank you for educating me about the limitations of the 1150 and the advantages of the socket 2011v3 board, it's not a factor I even considered.

 

You guys/gals have all been awesome in educating me, thank you so much! I've got some more homework to do and I'm going to weigh the benefits and increased costs of going to a 2011v3 socket E5-based system vs the 1150v3 socket E3-based system. My current needs don't justify an E5 based system ...

[garycase]

... My current needs don't justify an E5 based system ...

 

That's true for just about all of us 

 

However ... my "need" for a car to commute in back in the days when I was working didn't justify a Porsche either -- but guess what I bought 

 

I suspect my next system will be a Socket 2011v3 with an E5-1650v3 ...

 

[limefrog]

Good points garycase

 

So I think I'm going to spend the extra dough and go with a 2011v3 socket motherboard with E5-1650v3. I think the initial extra costs will be worth the extra future-proofing. But I need some expertise.

 

I don't know if I should go for the SUPERMICRO MBD-X10DRI-T-O (Intel X540 Dual port 10GBase-T; $537.99) or the SUPERMICRO MBD-X10DRI (Intel i350 Dual port GbE LAN; $429.99). Both are essentially the same specifications except one has dual 10Gb lan ports and the other only has dual 1Gb lan ports. I just don't know enough if dual 1Gb lan ports will ever be a bottleneck in my future applications.

 

Any thoughts?

[garycase]

I'd go with the Gb ports => I really can't imagine needing 10Gb ports in a home environment.

 

[tdallen]

I'd go with the Gb ports => I really can't imagine needing 10Gb ports in a home environment.

Not to mention that you'd need to upgrade the rest of your home network to take advantage of 10Gb ports.  My routers and switches are all 1Gb and my cables are cat5e so I'll be staying with 1Gb for the foreseeable future.

[limefrog]

I'd go with the Gb ports => I really can't imagine needing 10Gb ports in a home environment.

Not to mention that you'd need to upgrade the rest of your home network to take advantage of 10Gb ports.  My routers and switches are all 1Gb and my cables are cat5e so I'll be staying with 1Gb for the foreseeable future.

 

Thank you gentlemen, I'm glad I have your common sense to rely on!

[garycase]

Whoa!!    Just looked again at the specific motherboards you listed => those are dual-socket LGA 2011 boards.

 

Certainly nice boards ... but they require E5-26xx series Xeons and won't work with the 1650v3 I suggested. 

 

You CAN simply buy an E5-2630v3 and use that board (a slight reduction in Passmark ... 12849 vs. 13452, but 8 cores instead of 6 -- so depending on what you're doing that may be an advantage).

 

Or you could use a single socket board like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

The only advantage of a dual socket board is that you could, if you later wanted to, add a second CPU.    Note that while this doubles the # of cores, it doesn't double the overall Passmark.    For example, with 2 E5-2630v3's the Passmark goes up to 18838.

 

Personally, I plan to buy a single socket board and a 1650v3.

 

[limefrog]

Certainly nice boards ... but they require E5-26xx series Xeons and won't work with the 1650v3 I suggested. 

 

Yikes! I didn't realize that, thank you!

 

The only advantage of a dual socket board is that you could, if you later wanted to, add a second CPU.    Note that while this doubles the # of cores, it doesn't double the overall Passmark.    For example, with 2 E5-2630v3's the Passmark goes up to 18838.

 

I did not realize this either, thanks for the education.

 

You CAN simply buy an E5-2630v3 and use that board (a slight reduction in Passmark ... 12849 vs. 13452, but 8 cores instead of 6 -- so depending on what you're doing that may be an advantage).

 

I don't think I'll be doing anything that will take advantage of the extra cores, so I'll stick with your intuition and get a single CPU MOBO.

 

Personally, I plan to buy a single socket board and a 1650v3.

 

Thanks again for the input, I'll be doing the same. Glad I have your expertise to benefit from.

 

Or you could use a single socket board like this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

What do you think of these other single socket boards by Supermicro? The only difference I see is that they offer less PCIe 3.0 lanes than the one you suggested (SUPERMICRO MBD-X10SRA-F-O), but they are cheaper and offer onboard graphics, which I will need since the E5-1650v3 CPU does not have integrated graphics (I may get a GPU in the future when the need arises but won't be getting one at the moment).

 

SUPERMICRO MBD-X10SRL-F (not available at Newegg but is elsewhere)

SUPERMICRO MBD-X10SRI-F

[garycase]

Either of those look fine.    As long as the available PCIe slots match what you anticipate needing, it doesn't really matter.   

 

Note that ALL of these boards have built-in video, including the one I had suggested.    Not sure why Newegg's specs don't show it, but here's the SuperMicro page for it:  http://www.supermicro.com/products/motherboard/xeon/c600/x10sra-f.cfm

 

All three of the boards support IPMI -- a really neat feature if you're not familiar with it.    It essentially completely eliminates the need for a keyboard/mouse/monitor on your server, as you can control it completely from another PC ... including the BIOS.

 

Any of those boards would work well for your new system.

 

 

[garycase]

The only reason you'd ever want to add a GPU is if you want one to pass-through to a virtual machine.  In fact, you may want more than one ... which may be a good reason to choose a board with at least a couple x16 (physical) slots.    [The MBD-X10SRL-F board you asked about has 2 x16 slots plus additional x8 and x4 slots, so it would work well with a couple graphics cards plus add-in SATA controllers as needed]

 

[limefrog]

Note that ALL of these boards have built-in video, including the one I had suggested.    Not sure why Newegg's specs don't show it, but here's the SuperMicro page for it:  http://www.supermicro.com/products/motherboard/xeon/c600/x10sra-f.cfm

 

All three of the boards support IPMI -- a really neat feature if you're not familiar with it.    It essentially completely eliminates the need for a keyboard/mouse/monitor on your server, as you can control it completely from another PC ... including the BIOS.

 

Thanks for explaining IPMI to me in layman's terms. And I didn't realize that the MOBO you recommended (SUPERMICRO MBD-X10SRA-F-O) had built-in video, so I will be going with that.

 

The only reason you'd ever want to add a GPU is if you want one to pass-through to a virtual machine.  In fact, you may want more than one ... which may be a good reason to choose a board with at least a couple x16 (physical) slots.    [The MBD-X10SRL-F board you asked about has 2 x16 slots plus additional x8 and x4 slots, so it would work well with a couple graphics cards plus add-in SATA controllers as needed]

 

Good point indeed. And the MOBO you recommended has the most number of lanes so more reason to go with the MBD-X10SRA-F-O.

 

Gary, thanks so much for all of your time and input, you have helped so much! I wish there was a way I could give you rep points or something.

[limefrog]

Gary, do you know if the MOBO you suggested (SUPERMICRO MBD-X10SRA-F-O) supports quad or dual channel RAM architecture? I couldn't find any specific mention on the spec sheet or at Newegg.

 

http://www.supermicro.com/products/motherboard/Xeon/C600/X10SRA-F.cfm

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

This wikipedia article (https://en.wikipedia.org/wiki/Multi-channel_memory_architecture#Quadruple-channel_architecture) makes it sound like the E5-1650v3 CPU will support it, just not sure if the MOBO will.

 

And I assume this MOBO supports non-registered (unbuffered) RAM since there is no mention of it.

[garycase]

The board has four memory channels, so if you've installed 4 modules it will operate in quad-channel mode.

 

You do NOT want to buy unbuffered modules ... buffered modules present 1/16th the electrical load to the address and data buses and are consequently FAR more reliable.    You want registered ECC modules !

 

(Yes, they cost more)

 

 

 

[limefrog]

The board has four memory channels, so if you've installed 4 modules it will operate in quad-channel mode.

 

You do NOT want to buy unbuffered modules ... buffered modules present 1/16th the electrical load to the address and data buses and are consequently FAR more reliable.    You want registered ECC modules !

 

Thanks Gary. I didn't realize that if a MOBO takes LRDIMM, that automatically means it is registered (buffered) RAM.

 

Reading this helped for others that are confused like me.

 

http://www.anandtech.com/show/6068/lrdimms-rdimms-supermicros-latest-twin/2

https://en.wikipedia.org/wiki/Registered_memory

 

Buffered = Registered

UDIMM = unregistered DIMM

RDIMM = registered DIMM

LRDIMM = load reduced DIMM (similar to RDIMM but offer larger mem capacity and reduce power consumption)

 

[limefrog]

I think you'd be quite happy with the AsRock and an E3-1246v3.

 

Sorry to revive this semi-old thread, but I had another question about the ASRock C226WS+ motherboard and a Xeon E3 1200 v3 CPU.

 

1) The ASRock supports 32 PCIe lanes (PCIE1,PCIE3,PCIE5,PCIE7: x16/x0/x16/x0 or x8/x8/x8/x8 mode), but the E3 1200 v3 chips only support 16 lanes. Am I missing something? What is the point of getting a motherboard that supports more PCIe lanes than the CPU can support?

 

2) And what is meant by "PCIE1,PCIE3,PCIE5,PCIE7?" I don't think the numbers are referring to lanes.

[ohlwiler]

I looked up the manual for the C226WS+ board and found something called a PEX 8747 between the 32 lanes of the PCIe bus and the 16 lanes of the processor. More info here.

 

http://www.avagotech.com/products/pcie-switches-bridges/pcie-switches/pex8747

 

The numbering scheme refers to the physical position of the different PCIe slots. It is part of the ATX standard. Slot 1 is closest to the processor and they increment by one. The C226WS+ uses double spacing between PCIe slots (meant for video cards) and thus the numbers increment by two.

[garycase]

The 1150-based systems only support 16 active PCIe lanes, as you've noted.    The AsRock board has 4 PCIe slots that use x16 sockets, but run at x8.    That does NOT mean you can run 32 lanes ... there can't be more than 16 active lanes due to the chipset/CPU limitation of 16 lanes.    However, these are PCIe v3 lanes, so there is PLENTY of bandwidth (1000MB/s per lane).    As I noted earlier, if you want more bandwidth, you need to use a Xeon E5 series and Socket 2011 board ... then you'll have support for 28-40 lanes (depending on the CPU).

 

The PCIE1, PCIE3, PCIE5, PCIE7 designations are referring to the physical location of the sockets on the board ... note that they're spaced apart so you can use double-width boards in the sockets.

 

 

 

[limefrog]

The numbering scheme refers to the physical position of the different PCIe slots. It is part of the ATX standard. Slot 1 is closest to the processor and they increment by one. The C226WS+ uses double spacing between PCIe slots (meant for video cards) and thus the numbers increment by two.

The PCIE1, PCIE3, PCIE5, PCIE7 designations are referring to the physical location of the sockets on the board ... note that they're spaced apart so you can use double-width boards in the sockets.

 

Thanks guys, this community never fails to impress and educate.

 

The 1150-based systems only support 16 active PCIe lanes, as you've noted.    The AsRock board has 4 PCIe slots that use x16 sockets, but run at x8.    That does NOT mean you can run 32 lanes ... there can't be more than 16 active lanes due to the chipset/CPU limitation of 16 lanes.    However, these are PCIe v3 lanes, so there is PLENTY of bandwidth (1000MB/s per lane).    As I noted earlier, if you want more bandwidth, you need to use a Xeon E5 series and Socket 2011 board ... then you'll have support for 28-40 lanes (depending on the CPU).

 

Got it, thanks garycase, I assumed that since the specifications noted "x16/x0/x16/x0" for the 4 PCIe slots, that the motherboard could support 32 lanes, but that that was a moot point since the CPU it supports can only do 16 lanes.