Jump to content
Sign in to follow this  
tyrindor

Need a motherboard with 3x PCI-E x8

32 posts in this topic Last Reply

Recommended Posts

Thanks.

 

Thoughts on this?

http://www.newegg.com/Product/Product.aspx?Item=N82E16813131714

 

2 x PCIe 2.0 x16 slots (@ x16 or x8);

2 x PCIe 2.0 x 16 (@ x8)

 

More expensive than I wanted but it seems you need to go to about $250 to get what I want. This one also will give me an extra PCI-E slot that runs at x8 in the rare case I want to expand to 4 SAS cards. Any issues with P67 and unRAID?

Share this post


Link to post

Thanks.

 

Thoughts on this?

http://www.newegg.com/Product/Product.aspx?Item=N82E16813131714

 

2 x PCIe 2.0 x16 slots (@ x16 or x8);

2 x PCIe 2.0 x 16 (@ x8)

 

More expensive than I wanted but it seems you need to go to about $250 to get what I want. This one also will give me an extra PCI-E slot that runs at x8 in the rare case I want to expand to 4 SAS cards. Any issues with P67 and unRAID?

 

I would be more worried about those slots working with anything but video cards.  Some of the higher end gaming boards have that issue.

Share this post


Link to post

Thanks.

 

Thoughts on this?

http://www.newegg.com/Product/Product.aspx?Item=N82E16813131714

 

2 x PCIe 2.0 x16 slots (@ x16 or x8);

2 x PCIe 2.0 x 16 (@ x8)

 

More expensive than I wanted but it seems you need to go to about $250 to get what I want. This one also will give me an extra PCI-E slot that runs at x8 in the rare case I want to expand to 4 SAS cards. Any issues with P67 and unRAID?

 

I would be more worried about those slots working with anything but video cards.  Some of the higher end gaming boards have that issue.

 

Just can't win. :(

 

Is no one else using 3 of these cards? What are they using? Many people seem to have 24 drive cases, I assume they got all their connections wired up.

 

EDIT: According to this review you are right, no hard drives show up when SAS cards are installed.

http://www.newegg.com/Product/Product.aspx?Item=13-131-714&SortField=0&SummaryType=0&Pagesize=10&PurchaseMark=&SelectedRating=-1&VideoOnlyMark=False&VendorMark=&IsFeedbackTab=true&Keywords=SAS&Page=3#scrollFullInfo

Share this post


Link to post

Motherboard with 2xPCIe (x16+x8) and 8-10 SATA-ports.  =26xSATA  8)

 

My AsRoick Fatal1ty P67 Professional has 10x SATA and PCIe x16+x8+x4+x1+x1 in 1155 but not graphics.

(It's the desktop, not the server)

 

I have Graphics on one PCIe and revodrive on another (as a boot drive) so might work with two SAS-cards?

Share this post


Link to post

If you are not tied to socket 1155, I've used (or am using) 3x IBM BR10i as well as M1015 cards (all x8) on the following motherboards with success:

 

MSI 790FX-GD70 (socket AM3).  This is an AMD based board so you can run low power Semprons and other more powerful AM3 socket processors.  While this is an older board, the newer 890 and 990FX chipsets should be fine as well.  The x90FX chipsets have plenty of PCI-e bandwidth (lots of lanes).  I don't use this board other than for testing as AMD's idle state consumption is higher than Intel's.

 

Supermicro X8SIA-F (socket 1156).  This is an Intel based board.  It uses a PCI switch that allows three x8 slots to be used from the x16 normally available from the processor.  This is not an issue since the 16 lanes at PCI-e 2.0 have enough bandwidth to support about 32 hard drives (not SSD as they can easily saturate the bus).  Beginning parity checks with 20 data drives easily exceeds 100MB/s.

 

Socket 1155 only allows 16 lanes from the CPU, and DMI to the south bridge is already loaded so finding a board with 3 x8 close is going to be difficult.  Maybe Supermicro will make an X9SIA-F.  Otherwise the older X58 chipset (LGA 1366) or the new X79 (LGA 2011) offer enough PCI-e lanes.

 

Regards,  Peter

Share this post


Link to post

Forgot to mention that the Supermicro boards support mass storage controllers to be used in all the slots and have bios options to disable the roms on the cards as well.

 

Regards,  Peter

Share this post


Link to post

The other thing you can take into consideration that a single unRAID Pro license is currently limited to 22 drives total, there isn't really a need to run so many 8x cards in one unRAID system unless you are set on having the drives all run purely off controller cards (none onboard). The performance bottleneck is also more likely to be the drives themselves or the network, rather than the PCI-E bus.. at least for a home media server.

Share this post


Link to post

Forgot to mention that the Supermicro boards support mass storage controllers to be used in all the slots and have bios options to disable the roms on the cards as well.

 

Regards,  Peter

 

Thanks, out of the boards you linked.. I like the supermicro the most. I currently use one and it's been great. I'm looking at it and it looks like it has 3x PCI-E x8, but also has a PCI-E x16 (that runs at x16). I believe the controllers I linked in the first post won't take up more than 1 slot, so I could use 4 SAS cards if needed. However, with 3 I could set one in the x16 and 2 in the x8s to keep them more separated it seems (If they are anything like my SAT2s they put off a lot of heat). They would both still run at the full x8 then, correct?

 

 

What are your thoughts on this as my final build? It's about $900 and i'm aware the processor is overkill, but for $20 more, i'd rather get a true i3 even if unRAID doesn't need it.

 

Motherboard:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182235

 

Processor:

http://www.newegg.com/Product/Product.aspx?Item=N82E16819115065

 

RAM:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820178369

 

SAS Cards (x3):

http://www.provantage.com/supermicro-aoc-sas2lp-mv8~7SUP92PM.htm

 

SAS Cables (x6):

http://www.newegg.com/Product/Product.aspx?Item=N82E16816133034

 

The other thing you can take into consideration that a single unRAID Pro license is currently limited to 22 drives total, there isn't really a need to run so many 8x cards in one unRAID system unless you are set on having the drives all run purely off controller cards (none onboard). The performance bottleneck is also more likely to be the drives themselves or the network, rather than the PCI-E bus.. at least for a home media server.

 

I sent in a email and 22 data drives, parity, and cache does not seem that far off. :)

 

I have a SAS case, so I prefer to use SAS to SAS cables. The motherboard is only SATA. Call me OCD.. but I prefer to keep it nice and neat. :)

Share this post


Link to post

I bought my X8SIA-F board from Newegg as well though I waited for a refurb unit.  When they become available, the pricing is around $145.  The unit I received seemed new to me.  If you can wait, it might be worth it.

 

The processor is fine.  If you decide that you want ECC ram support, you'll need a Xeon.  I am toying with that now.

 

I think the ram is fine.  I use Kingston ECC ram as I might make the switch to a Xeon.

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139262

 

The SAS cables are too costly at Newegg.  You can get them cheaper directly from Norco:

 

http://www.ipcdirect.net/servlet/Detail?no=216

 

or buy similar cables even cheaper from my favorite cable place Monoprice:

 

http://www.monoprice.com/products/product.asp?c_id=102&cp_id=10254&cs_id=1025410&p_id=8189&seq=1&format=2

 

And don't be ashamed by OCD.  I like the neat layout and bandwidth capabilities of having all drives running of identical controllers on x8 slots.  Although I must admit I am looking at getting one of the following:

 

http://www.provantage.com/highpoint-technologies-rocketraid2760a~7HPTI03W.htm

 

Good luck!

 

Regards,  Peter

Share this post


Link to post

......... I'm looking at it and it looks like it has 3x PCI-E x8, but also has a PCI-E x16 (that runs at x16). I believe the controllers I linked in the first post won't take up more than 1 slot, so I could use 4 SAS cards if needed. However, with 3 I could set one in the x16 and 2 in the x8s to keep them more separated it seems (If they are anything like my SAT2s they put off a lot of heat). They would both still run at the full x8 then, correct?

 

It does NOT have an electrical x16 slot.  It is only a physical x16 slot and will only support x8 speeds.  You would use the "x16" and the next two x8 slots.  The last x8 slot is actually an x4 slot fed off of the PCH 3420 chip (along with everything else including Lan, video, ipmi, usb, etc., etc.).  You can see all this if you download the user manual.  It contains a very useful diagram showing the layout.

 

Regards,  Peter

Share this post


Link to post

I bought my X8SIA-F board from Newegg as well though I waited for a refurb unit.  When they become available, the pricing is around $145.  The unit I received seemed new to me.  If you can wait, it might be worth it.

 

The processor is fine.  If you decide that you want ECC ram support, you'll need a Xeon.  I am toying with that now.

 

I think the ram is fine.  I use Kingston ECC ram as I might make the switch to a Xeon.

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139262

 

The SAS cables are too costly at Newegg.  You can get them cheaper directly from Norco:

 

http://www.ipcdirect.net/servlet/Detail?no=216

 

or buy similar cables even cheaper from my favorite cable place Monoprice:

 

http://www.monoprice.com/products/product.asp?c_id=102&cp_id=10254&cs_id=1025410&p_id=8189&seq=1&format=2

 

And don't be ashamed by OCD.  I like the neat layout and bandwidth capabilities of having all drives running of identical controllers on x8 slots.  Although I must admit I am looking at getting one of the following:

 

http://www.provantage.com/highpoint-technologies-rocketraid2760a~7HPTI03W.htm

 

Good luck!

 

Regards,  Peter

 

Well I was just debating with my friend on if ECC was really worth it for just a media server. I really don't feel ECC is needed for a home media server, as my 20 drive server has had no issues with it's current non-ECC DDR2 800.

 

Looks like i'll grab these instead and keep the i3 (they are faster and $20 cheaper):

http://www.newegg.com/Product/Product.aspx?Item=N82E16820231440

 

......... I'm looking at it and it looks like it has 3x PCI-E x8, but also has a PCI-E x16 (that runs at x16). I believe the controllers I linked in the first post won't take up more than 1 slot, so I could use 4 SAS cards if needed. However, with 3 I could set one in the x16 and 2 in the x8s to keep them more separated it seems (If they are anything like my SAT2s they put off a lot of heat). They would both still run at the full x8 then, correct?

 

It does NOT have an electrical x16 slot.  It is only a physical x16 slot and will only support x8 speeds.  You would use the "x16" and the next two x8 slots.  The last x8 slot is actually an x4 slot fed off of the PCH 3420 chip (along with everything else including Lan, video, ipmi, usb, etc., etc.).  You can see all this if you download the user manual.  It contains a very useful diagram showing the layout.

 

Regards,  Peter

 

Ah, thanks.

 

Here's what I have, unless someone states this won't work or if I can get a better deal somewhere, i'll order this after I get my next paycheck.

 

Motherboard:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182235

 

Processor:

http://www.newegg.com/Product/Product.aspx?Item=N82E16819115065

 

RAM:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820231440

 

SAS Card (x3):

http://www.provantage.com/supermicro-aoc-sas2lp-mv8~7SUP92PM.htm

 

SAS Cable (x6):

http://www.ipcdirect.net/servlet/Detail?no=216

Share this post


Link to post

If you are just running mechanical consumer drives in the SAS cards, you are most likely  not going to saturate a PCIe2 4x channel with 8 drives.

Many serverboards have 4x plugs in 8x 16x slots. 

 

If you were running SATAIII SSD's then you might.

Share this post


Link to post

If you are just running mechanical consumer drives in the SAS cards, you are most likely  not going to saturate a PCIe2 4x channel with 8 drives.

Many serverboards have 4x plugs in 8x 16x slots. 

 

If you were running SATAIII SSD's then you might.

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182235

 

I have people telling me that this board ONLY supports ECC regardless of what CPU you use. Can anyone confirm/deny? My old supermicro board said the same and it worked fine with non-ecc.

Share this post


Link to post

So i'm a little confused then, you need a Xeon processor to use ECC on it, but the motherboard clearly states it works with i3? How does that work if the motherboard requires ECC ram? Do I need ECC unbuffered for an i3 instead of ECC registered?

 

Can someone give me a link to 8GB DDR3 1333 with good timings that will work on this board with a i3? Preferably Newegg.

Share this post


Link to post

If I recall, to use the I3 with the supermicro boards. You need to use the UDIMMs.

Share this post


Link to post

There was no fire...  ;D but also not boot.  Yep, you need ECC ram as regular ram will not boot.  So it is:

 

i3 = ECC unbuffered

Xeon = ECC buffered or unbuffered

 

As a bonus, it seems that you get ECC functionality even with an i3 (at least according to reports and the boot screen identifies ECC ram).

 

Regards,  Peter

Share this post


Link to post

If you are just running mechanical consumer drives in the SAS cards, you are most likely  not going to saturate a PCIe2 4x channel with 8 drives.

Many serverboards have 4x plugs in 8x 16x slots. 

 

If you were running SATAIII SSD's then you might.

 

I agree with you on the bandwidth numbers.  Unfortunately, many people are using the AOC-SASLP-MV8 that, while an x4 card, is only PCI-E Gen 1 so it only has access to half the available bandwidth of a PCI-E Gen 2 slot.  The AOC-SAS2LP, an x8 Gen 2 card, has 4 times the bandwidth of the older card.

 

Regards,  Peter

Share this post


Link to post

There was no fire...  ;D but also not boot.  Yep, you need ECC ram as regular ram will not boot.  So it is:

 

i3 = ECC unbuffered

Xeon = ECC buffered or unbuffered

 

As a bonus, it seems that you get ECC functionality even with an i3 (at least according to reports and the boot screen identifies ECC ram).

 

Regards,  Peter

 

Thanks for confirming, i'll grab the ECC unbuffered RAM.

 

I did find this micro board though:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-_-RSSDailyDeals-_-na-_-na&AID=10521304&PID=4003003&SID=1wo9275oec94d'>http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-_-RSSDailyDeals-_-na-_-na&AID=10521304&PID=4003003&SID=1wo9275oec94d

 

Would allow me to put a more power efficient processor in it, the only downside is the fact that 2 of the slots run at x4. It's still PCI-E 2.0 though, which still makes it have double the bandwidth that the original SALSP card could? Which should be PLENTY for 8x green drives? I would hate to go this route and then find that my 3rd card running at x4 would slow down my entire system during parity syncs (even 5-10MB/s).

 

New build would end up like this (which is more power efficient, faster, and cheaper... assuming x4 PCI-E 2.0 won't bottleneck 8x green drives):

 

Motherboard

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253

 

Processor

http://www.newegg.com/Product/Product.aspx?Item=N82E16819115077

 

RAM

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139262

 

SAS Cards (x3)

http://www.provantage.com/supermicro-aoc-sas2lp-mv8~7SUP92PM.htm

 

SAS Cables (x6)

http://www.ipcdirect.net/servlet/Detail?no=216

 

------------------

 

Go with the above, or go with my original build (below) if PCI-E 2.0 x4 could bottleneck?

 

Motherboard:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182235

 

Processor:

http://www.newegg.com/Product/Product.aspx?Item=N82E16819115065

 

RAM:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139262

 

SAS Card (x3):

http://www.provantage.com/supermicro-aoc-sas2lp-mv8~7SUP92PM.htm

 

SAS Cable (x6):

http://www.ipcdirect.net/servlet/Detail?no=216

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this