Socket 1155 Motherboard and cpu

[cpny]

Hello , things changed a bit and felt like i would rather not take my case apart later after i do cable management to change the board , so decided to go with the right one from the start.

 

So i'm going with a 22 drive project or well 30ish when unraid allows it.

 

I have some rather big projects that i have to compile on a regular basis , some lasting over six hours on a core 2 quad , so i would like a powerful system to compie it in , if not possible to compile it with just the toolchain available for unraid i would probably use virtualization.

 

I assume the 1155 is the socket to pick right now ? thought about a bulldozer but intel has had the edge for the last years.

 

What motherboard would be the best pick ? going to need 3-4 BR10i's to so i would need matching pci express speeds

 

What would be the best xeon model for this use excluding the ones over 500$

 

Thank you very much

[c3]

Sounds like you want a Norco 4224 build with a Supermicro X9SC?-F using a Xeon E3-1240 or even E3-1270. See more CPU info

 

The Supermicro X9SCM is very popular because it has 4 PCIe slots, but the X9SCA can give you PCI slots if you need those.

If you really want to jump on more than 16GB of RAM right away, the X8SI? family might be an alternative due to the limited (high price) availability of 8GB ECC memory sticks.

 

You can reach your drive count without using 3-4BR10is by using a SAS Expander like Intel's RES2SV240.

 

If you really want to go past 24 drives, Supermicro has a case for 36 drives.

 

Much of these details, including virtualization are available on the Atlas thread.

[cpny]

Hello thanks for the info , i already have a case it's a lian li pc-343b wich holds 30 hot swap drives.

 

Mostly wanted to know wich board would be the best to pick and cpu so thanks for the help , probably going with 32gb at first or so

 

Will look into the cpu and sas expander as well , thanks for the information.

 

Also overlooked that topic somehow , alot of information stuff indeed

 

And if the intel expander works fine in unraid it will save me alot of ehadaches trying to squeeze so may sata cards.

[Johnm]

per our PM's, it sounds like you are on the right path.

 

As far as AMD options. I know they are trying to catch Intel again. Unfortunately I have not been keeping up with them. So I do not know what their high end offering can offer.

 

C3 read my mind.. what he said..

 

the only concern I might have is how disk IO intensive is this compile? perhaps you might need to change something there.

 

Also be aware that the BR10i can not use drives greater in size then 2.2 TB.

if you already own them, great. if you are considering buying them, it might be worth looking at other cards.

 

[c3]

A couple of SSDs and the 32G will make a big difference on compile. The X9SCM has (2) 6GB SATA ports, the X8SIL does not.

 

RES2SV240.

 

I would be concerned about the power cable length required in the lian li pc-343b. The motherboard and power supply are in different compartments. It's $350 and then add (6) 5x3 @$100 plus a big power supply, The Supermicro 36 bay is only a small premium with 1400W redundant power.

[cpny]

What do you think i need to change Johnm ? how disk instensive hmm , one is a custom linux distro all compiled from source such as linux from scratch , the other is equally large , but yeah it's mostly compiling somewhat of big projects .

 

Also besides that i will use it with sickbeard etc so i would like speed since it will be extracting stuff on a regular basis etc.

 

Also i was planning on going with 2tb drives for now , the case i already bought , i needed something that could blend in the house , so yeah as price goes i know i'll end up at a loss of around 100ish euros in europe with the caddies due to higher prices of the norco here.

 

Does this sound good then for what i need ?

 

Motherboard : SUPERMICRO MBD-X9SCM-F-O

 

Cpu : Still going to decide but one of the xeons

 

Memory : Probably starting with 16gb ecc

 

Case : Lian li pc-343b

 

Caddies : Norco SS-500x6

 

Adapters : IBM BR10I and sas extender Intel RES2SV240

 

Cooler : People tend to use the default intel one or ? was thinking of a silver arrow or a coolmaster one but not sure if worth it.

 

Thanks for your time

 

 

[Johnm]

Well,

 

Lets look at this.

 

You did not mention if you already own the BR10i's.

If you don't lets look at other cards, these can be a bottleneck later on in your build.

both the 2.2 TB limit and they are a bit slow on an expander.

if you have them, we'll work around it.

 

either way, while they can be expanded, they are so cheap and you have spare ports, I would not.

 

ram:

see if you can find the Kingston 8GB sticks.

Single: KVR1333D3E9S/8G    8GB 1333MHz DDR3 ECC CL9 DIMM

2 Pack: KVR1333D3E9SK2/16G    16GB 1333MHz DDR3 ECC CL9 DIMM (Kit of 2)

 

It might not be an option for you. yet..

 

 

OS.

Are you going to run ESXi on this or just straight unraid?

 

 

[cpny]

Sorry , i don't have the BR10i's yet .

 

But i found the br10i for like 30$ and in europe i can find the supermicro aoc 8 port for 100 something euros.

 

But yeah i don't want it to cap the system.

 

So is the sas out of the equation or still good ? so the super micro has 6 sata ports + 8 from some card (what can we find at a decent price that ships to europe? br10i refurb were like 30$ but i don't mind going up to 100 if it's worth it) with an objective for 30 drive plus maybe 3 or 4 on the back.

 

Not sure 100% still if going to go the esxi way or just unraid , above all want to be flexible.

 

Thanks alot for the help , want to make sure this is money well spent for the objectives i have.

 

Thanks for the ram info , going to pick what best suits me from those as well.

 

Damn the 16g double pack is expensive hehe , yeah that one will have to be a later pick

[c3]

I think you need to decide if this is going to be unRAID on bare metal or virtualized.

 

The difference is the motherboard ports, which are the fastest.

[cpny]

Hard to tell for now since i need to test both , anyway my idea would be to have the 30 drives all together working as raid media on the server with a drive to do the compile stuff.

 

That assuming unraid gets more than 22 drive support sometime , that's why i want flexibility , if i end up needing 30 drives before unraid supports it i will need to have it working on a different system until unraid supports it.

 

Anyway i think the key here is that i would like at least 20 drives all working together with decent speeds to serve my media players at home , also to unpack stuff at a decent speed , doesnt need to be bleeding edge 300 something mb/s even tho i wouldn't mind it , if possible 30 with a spare for cache and parity.

 

Cheers

 

So bottom line is i want 30 drives working sometime down the line , so it would be good if my solution supported unraid freebsd and the esxi option.

 

Will the sas expander and one other 8 port card do that ? or would i need 4 sata cards for that ? shrug i always have to complicate things

[Johnm]

But i found the br10i for like 30$ and in europe i can find the supermicro aoc 8 port for 100 something euros.

 

But yeah i don't want it to cap the system.

at $30 vs $100, it might be worth buying a few BR10i's and sell them or throw them away when you go to 3 or 4TB drives.it would be a bit of a waste of cash, this option might take a bit of the bite off upfront if you're spending $2000+ upfront for the rest of the server.

See if you can get an ebay M1050 cheap?

 

 

So is the sas out of the equation or still good ? so the super micro has 6 sata ports + 8 from some card (what can we find at a decent price that ships to europe? br10i refurb were like 30$ but i don't mind going up to 100 if it's worth it) with an objective for 30 drive plus maybe 3 or 4 on the back

.

 

for unRAID the SAS is really only a feature of cable management (and port expansion).

SAS HBA's are also slightly more common and usually a better quality product and support.

 

Not sure 100% still if going to go the esxi way or just unraid , above all want to be flexible.

 

here is what I am thinking and others might agree.

Keep ESXi as an option..

 

Build your unraid system as just an unraid system. but as a guest.

 

Then do your compiling and downloading on another guest(s).

this would keep the production unRAID install clean.

It would also keep your other applications from impacting your unraid server.

 

keep in mind you would loose the onboard sata ports for unraid (not actually true, you can RDM them. I just don't think it is a good idea for you plans)

so if you base unraid build is 8 drives you need one HBA. for 16 drives, 2 HBA's etc...

 

I would then RDM the onboard ports or use virtual drives to your other guests.

 

Damn the 16g double pack is expensive hehe , yeah that one will have to be a later pick.

Yeah , it is still hard to find item. it is a bit overpriced still.

In 6 months i bet it is about 1/2-2/3 that price.

 

[Johnm]

Hard to tell for now since i need to test both , anyway my idea would be to have the 30 drives all together working as raid media on the server with a drive to do the compile stuff.

 

That assuming unraid gets more than 22 drive support sometime , that's why i want flexibility , if i end up needing 30 drives before unraid supports it i will need to have it working on a different system until unraid supports it.

 

Anyway i think the key here is that i would like at least 20 drives all working together with decent speeds , if possible 30 with a spare for cache and parity.

 

Cheers

 

 

 

If you go ESXi, you could put 2 unRAID boxes into your 30 drive box.. 20 drives on one and 10 drives on the other.. or some combo.

 

You do not have to make the decision now. but you should plan that you will go ESXi if you're interested in it. that way you buy ESXi compatible hardware...

 

So bottom line is i want 30 drives working sometime down the line , so it would be good if my solution supported unraid freebsd and the esxi option.

Will the sas expander and one other 8 port card do that ? or would i need 4 sata cards for that ? shrug i always have to complicate things

It depends on the  HBA and the expander combo you buy and the performance you need.

 

As i said, i would not bother expanding a BR10i if you have free PCIe slots.

 

For the freebsd, i would just RMD a motherboard port. possibly use a BR10i later when you upgrade.

 

 

 

[cpny]

Yes i understand john and i am keeping that in mind.

 

But one question remains , if i need to fill 30 drives + some other for guests or so i would need 4+ sas hba's , and on the case of the br10i they require a 8x pci express 8x port i believe.

 

So it's impossible to do that with any of the supermicros as far as i know.

 

Is the sas expander the thing to do ? otherwise i guess i would need 4x supermicro aoc wich would be quite costly compared to the sas , but one question remains , would the sas be too much of a bottleneck? really inexperienced with the expanders so a bit clueless.

 

So if i added the intel expander + a br10i or supermicro + 6 mb ports or well 5 due to the rmb , i would have well 7 too many or would more ports be shafted?

 

Makes me scared how you treat the expander as just cable management and more ports , then again i would assume 24 drives on a x4 pci express would cap rather easily unless i'm missing something , but then again can't see a viable option for the 4 hba's.

 

Love the 20 and 10 drive unraid idea ! but i'm a bit stumped with the technical side of it right now

 

Shrug

 

Thanks a bunch

 

Actually on some more reading , if i use a sas expander with 24 ports all those drives will only have the transfer bandwidth of one sata port on the motherboard ? then that would be quite bad heh

 

What do you think would be the best idea to host 30 drives + 2 or 3 and be compatible with esxi Johnm ? if i go that route i think i need to use the supermicro 8 port even tho i can't seem to find it easily so probably have to buy in the us + custom taxes , going to order one a time if that's the case to not be hit so hard , i say the supermicro since it uses x4 and the br10i x8 , even then need to see if the supermicro board that i want can power 4 hba cards at x4 speed.

 

[Johnm]

you are correct.. the bandwidth on the pcie 8x vs pcie 4x is an issue.. you will only feel that during parity and rebuilds. in normal use each port runs full speed.

 

The X9SCM has 2x PCIe 8x and 4 PCIe 4x (in an 8x slot).

that is one reason I am steering you away for the BR10i and an expander. While it is an x8 card it only about 4x in speed.

It is only 2.5GB/s per port. once you start expanding those, the performance degrades fast during parity.

the BR10i should work fine in a supermicro 4x slot.

 

If you are going to use an Expander (or 2) I would go with SAS2 cards.

 

unless you already have the hard drives now, you can buy these parts over time..

 

this thread might shed some light http://lime-technology.com/forum/index.php?topic=16769.0

 

 

 

[cpny]

Nah i will buy as i go.

 

So as long as a motherboard can support it the best would be to go for 4 supermicro 8 port cards or the m1015 ones ? are those x4 as well or ? noticed you have some m1015's.

 

This with the X9SCM-F  motherboard , with 4 x4's i wouldn't have an impact with those correct ?

 

But yeah if going 4 supermicro or m1015 is the better option i will go with that , already sinking so much money that it may as well be perfect.

 

All of the things i picked are esxi compatible right ?

 

Cheers

[Johnm]

I would not worry about 30 drives until that is a reality. I have a strange feeling that if there is an increase in drive #'s. Logically an increase to 24 drives would make more sense to fit a norco. and that might never happen..

 

I would go with 1 M1015 and a Intel RES2SV240 for 16 hard drives.

configured as m1015 dual linked to the Intel RES2SV240 for 16 drives.

 

if you go past 16 drives..

I would keep using that combo with only one SAS connector expanded until I get to 22(24) drives.

 

If i find the performance is to slow, i would then get a second M1050 (or other SAS2 card).

I would configure that as 1 untouched M1015 and 1 m1015 to Intel RES2SV240 with dual linked 16 port configuration.

this would get you about full speed on all ports with green SATA drives... if unRAID go past 24 drives. get a second expander.

 

If you do not care how long parity takes.. for the 30 drive build you could use two RES2SV240 on a single M1015 with 15 drives per expander...

 

does this make sense?

 

Right now my configuration is 1 SASLP-MV8 and 2 M1015's.

that's a bit overkill and I plan to go to a single M1015 and a single Intel RES2SV240 in January.

I'll post performance changes when I do that.

 

I am also considering putting both my unraid pro servers onto the one ESXi server. the second one (or both) will be in DAS boxes.

 

 

the SASLP-MV8 is not natively supported in ESXi. there is a real easy hack to add it. There are forum members with 2 or 3 in ESxi just fine..

With the price of expanders, in most cases, a physical HBA is cheaper then an Expander. plus better performance.

[cpny]

Thanks yeah i will add as i go.

 

Actually the m1015 is more expensive for me so i guess i'll go with the supermicro.

 

Still going to think about the expander vs more supermicros , either way the 30 drive should be distant if we count hdd prices right now and i'm going to add as i need anyhow.

 

So 4 supermicro 8 ports would be overkill but the motherboard wouldn't be a bottleneck right ? specially since other than parity checks i would not max it probably but want to make sure , even tho 30 drives is in future it would suck to buy another motherboard .

 

The expander thing is still a but confusing to me but will spend some time reading more about it and it will be cleared then.

 

Thank you very much for your patience

 

By the way what read / write speeds can one expect with unraid with a cache drive ? will it be limited to the single disk you test normal performance i figure ? since the raid in unraid is different.

 

Cheers

 

[Johnm]

the saslp-mv8's are only 4X cards. in theory they get saturated after 6 drives with green drives.. also they are not officially expandable. they are not expander aware.. but people have posted they do work.  like the BR10i, it would be a dog on an expander I would not bother expanding one.

 

If you only have 8 drives that are 2TB now. I say get one BR10i now if they are so cheap. then just watch ebay for a used m1015?

you can always find a new use for that card later. not being in a rush often leads to good deals.

 

Getting one SM card is also an option i guess.. whats you upfront build size?

i know 30 drives down the road,, what is you first month or two target size?

[cpny]

Hello first 2 months will probably suffice with 8. at most 16 but very doubtful as don't want to go crazy while the hard drive prices are so crazy.

 

So my biggest doubt is , if i don't want to go the expander way i will still hit a bottleneck right ? in this case a motherboard one.

 

If i bought 3 or 4 m1015 and placed them on the Supermicro X9SCM-F motherboard 2 of them would still be capped at x4 speeds right ? and other 2 would be at 8x .

 

Just want to make sure i can go either way as i still need to do a lot of reading on the expander method so worst comes would like to still have a solution with 4 hba's or 3.

 

Cheers

 

 

[Johnm]

bingo!.. that's why I run 2 m1015's and 1 SASLP-MV8..

 

I used to have 3 M1015's and moved one to my other unraid server. replacing it with an MV8 i had in my parts bin.

 

I only use a single SAS port on the MV8 for now.. once I cross from 20 drives into 21/22 drives,  i'll have to put a fanout cable on my second port and mount my parity/cache internally.

 

with drives priced like they are, that will be a while.. unless i use smaller used drives..

[cpny]

Hehe cheers , now i know !

 

I guess if i go the full hba way i'll go with 2 m1015 and 2 super micros or so , or the expander , will wait for your data when you get and test yours , no rush

 

Thank you very for all the info , now to take care of everything , case is here along with 2 caddies need to order the board and cpu and ram now to start running the server.

 

Cheers

[cpny]

Found the m1015's for almost 40 euros cheaper than the supermicros , i can use 3-4 m1015's and still have same performance on the x4 slots as the supermicro ones right ? it will still function just at a lower speed ?

 

Sent you a message but guess this way others can know if they have doubts in the future as well.

 

Cheers

 

Also what is the max speed i can expect unraid to perform at with a cache drive ? such as

hdparm -t

or actual read and write speeds ?

 

I guess it will be limited to what a normal hard drive would perform since i believe there's no stripping ?

 

This assuming it's running without esxi and installed on the machine without anything else

 

Cheers

[Johnm]

Yes,

 

the cache drive will perform at the single drive speed. whether it is a spinner or ssd.

 

nice on the m1015's.

keep in mind they need to be reflashed. not all motherboards can flash them.

 

honestly, the speed difference wont be that great. when you use a fully loaded MV8, your parity check might be an average of 65MB/s instead of 75MB/s.

unraid is not a racehorse. it more like a reliable mule.

[cpny]

Gotcha , thanks

 

So it won't matter if i populate it with 4 m1015's as it will still at least work on the pci express 4 slot speeds just slower .

 

Going to order one for now i guess and the motherboard and cpu are on the way , went with the 1240 since it seemed like the best bang for the buck between speed and price

[Optimist]

Found the m1015's for almost 40 euros cheaper than the supermicros , i can use 3-4 m1015's and still have same performance on the x4 slots as the supermicro ones right ? it will still function just at a lower speed ?

 

Sent you a message but guess this way others can know if they have doubts in the future as well.

 

Cheers

 

Also what is the max speed i can expect unraid to perform at with a cache drive ? such as

hdparm -t

or actual read and write speeds ?

 

I guess it will be limited to what a normal hard drive would perform since i believe there's no stripping ?

 

This assuming it's running without esxi and installed on the machine without anything else

 

Cheers

 

Hi

 

Trying to source a M1015 in the Uk and cant seem to find a reasonably priced one. Would you mind letting me know where you found yours?

 

Thanks