Need a motherboard with 3x PCI-E x8


Recommended Posts

Hmm.. that X9 build looks very familiar. It will be rock solid... your speed will be fine unless your array is all SSD's

 

Haha, just noticed you choose pretty much the same parts. I have 21x WD Greens, and 1x WD Black for cache. I do notice your not using one of the x4 bandwidth slots, however if my math is correct your cards can only use PCI-E 1.0 x4 bandwidth, which would still be 2x less bandwidth than the newer cards running in PCI-E 2.0 x4. So I *should* be plenty fine if you aren't bottlenecking, and x8 would be ultra overkill.

 

Hopefully I see a noticeable improvement over my current 3x PCI-X 133mhz AOC-SAT2-MV8 cards. 65MB/s parity syncs with 3TB of parity is getting pretty old. Takes over half a day to complete every month, and date rebuild takes even longer. My impression is the PCI-E 2.0 x4 interface, is still over double the bandwidth of PCI-X 133mhz. The 2 cards that are running at x8 should be over four times more bandwidth.

 

Also, sort of sad to see the Norco 4224 on V3 now (noticed you have it). I'm on V2, no front USB. Not sure what else changed.

Link to comment

I'm not sure how Supermicro is doing it with this X9SCM board but the 1st x4 slot is running off of the processor just like the two x8 slots.  According to Intel, the layout should either be one x16 or two x8 only, not two x8 plus one x4 as there are only 16 lanes available.  The X8SIA board gets around this using a PCI-E switch.

 

You need to verify that the AOC-SAS2LP-MV8 will function correctly in an x4 slot.  There is no documentation on the Supermicro website for this card so I cannot tell.  Assuming that it does indeed work then an x4 Gen 2 PCI-E has plenty of bandwidth for 8 green drives.

 

Also be mindful of the unraid version you use.  v5b11 is the fastest release of unraid.  My parity checks run in excess of 110MB/s with all green drives (21 drives total) on the X8SIA board with 3 LSI controllers.  Of course as the checking approaches the inner tracks of the drives, the speed will slow down.  I am testing rc3 and it seems fast but not quite as fast as b11 for me so far (prelim).

 

Regards,  Peter

Link to comment

I have run 4 m1015's on the X9SCM with no issue. it just runs at PCIe2 4x speed (this is why SM makes the slot 8x in size).

 

i am running a 4 drive raidz on one of the 4x slots at the same speed as the 8x slot with an M1015. that is why I bumped it down to the 4x. leaving the 8x slots for m1015's with expanders.

Link to comment

I'm not sure how Supermicro is doing it with this X9SCM board but the 1st x4 slot is running off of the processor just like the two x8 slots.  According to Intel, the layout should either be one x16 or two x8 only, not two x8 plus one x4 as there are only 16 lanes available.  The X8SIA board gets around this using a PCI-E switch.

 

You need to verify that the AOC-SAS2LP-MV8 will function correctly in an x4 slot.  There is no documentation on the Supermicro website for this card so I cannot tell.  Assuming that it does indeed work then an x4 Gen 2 PCI-E has plenty of bandwidth for 8 green drives.

 

Also be mindful of the unraid version you use.  v5b11 is the fastest release of unraid.  My parity checks run in excess of 110MB/s with all green drives (21 drives total) on the X8SIA board with 3 LSI controllers.  Of course as the checking approaches the inner tracks of the drives, the speed will slow down.  I am testing rc3 and it seems fast but not quite as fast as b11 for me so far (prelim).

 

Regards,  Peter

 

I'm under the impression the all PCI-E x8 devices would work in a x8 slot with only x4 bandwidth. Same thing as putting a x16 graphics card into a x16 slot with only x8 bandwidth, which i've never seen not work.

 

I have a hard time believing Supermicro would make a board with 4 8x slots and then have it so you can only use 2 at a time. That would be false advertisement as far as i'm concerned. 16 lanes would still allow you to do 4 slots at x4 bandwidth, which is still more than you need for 8 HDDs. I'm sure the motherboard has an option to force the slots to x4, if it's not automatically done. Otherwise, you couldn't use all 4 slots at the same time (so why put them there?).

 

I'm going to give them a email just to make sure...

Link to comment

Got a response from Supermicro for anyone that may consider these parts in the future. All seems to be good.

 

My email:

I'm planning on hooking up 4x Supermicro AOC-SAS2LP-MV8 SAS cards onto a Supermicro MBD-X9SCM motherboard. Here are my concerns before I purchase these parts...

 

1) The SAS cards are PCI-E 2.0 x8 interface, however they should still work in x8 slots that provide x4 bandwidth, correct?

2) Will PCI-E 2.0 x4 provide enough bandwidth to supply 8x green drives (105MB/s reads each) without bottlenecking the performance?

3) Socket 1155 processors can only provide 16 lanes of PCI bandwidth. The MBD-X9SCM has 2x slots that run at x8 speed, and 2x slots that run at x4 speed. That's 24 lanes of bandwidth. Is there a way to make all 4 slots run at x4 (total of 16 lanes), or is this motherboard capable of doing 24 lanes?

 

Their response:

1.      Yes

 

2.      Should be enough

 

3.      It should provide 24 lanes

 

Thanks

 

FP

Link to comment

I'm not sure how Supermicro is doing it with this X9SCM board but the 1st x4 slot is running off of the processor just like the two x8 slots.  According to Intel, the layout should either be one x16 or two x8 only, not two x8 plus one x4 as there are only 16 lanes available.  The X8SIA board gets around this using a PCI-E switch.

 

You need to verify that the AOC-SAS2LP-MV8 will function correctly in an x4 slot.  There is no documentation on the Supermicro website for this card so I cannot tell.  Assuming that it does indeed work then an x4 Gen 2 PCI-E has plenty of bandwidth for 8 green drives.

 

Also be mindful of the unraid version you use.  v5b11 is the fastest release of unraid.  My parity checks run in excess of 110MB/s with all green drives (21 drives total) on the X8SIA board with 3 LSI controllers.  Of course as the checking approaches the inner tracks of the drives, the speed will slow down.  I am testing rc3 and it seems fast but not quite as fast as b11 for me so far (prelim).

 

Regards,  Peter

 

C204 (and C206) chipsets allow 20lanes of PCIe from the CPU and includes 8 lanes from the chipset itself.

 

So according to Supermicro's diagram, 20 of the CPU's lanes are occupied by 3 slots (8/8/4) and the remaining x4 comes from the chipset. We could have had a third x8 were it not for the 2 gigabit lan port taking up x1 each.

 

The thing is, you have to use Xeon E3 processors to get the 20 lanes as normal desktop processors only have 16 lanes.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.