Jump to content
Vitalsignser

Help with IPMI MB & processor choice

7 posts in this topic Last Reply

Recommended Posts

So I've done everything a bit backwards. I've gotten a bunch of parts purchased except my MB, processor, RAM and NVMe. Unraid server with plex and makemkv docker. Eventual windows 10 pro VM

Case: Supermicro CSE-836BE16-R920B with BPN-SAS2-836EL1

HBA: Dell h220 flashed in IT mode (Aka LSI 9207-8i) - will connect 2 ports to the backplane and I believe an HP or intel expander is not needed

10G SFP+ card: Mellanox CX312A ConnectX-3EN Dual Port 10GB SFP+ MCX312A-XCBT

GPU: quadro p2000 (transcoding)

Drives: data 10TB x 10 WD elements, parity 12TB x2 WD easystore

4kUHD slim drive (For MakeMKV): LG BU40N

Cache: Want dual 2TB NVMe - need MB to accomodate or card with bifurcation of x8 port

 

So by my math I need a single PCI-e x16 (dual width on MB to accommodate size of GPU??), Three PCI-e x8 ports for the HBA, 10G SFP+ and NVMe bifurcation port

I'd like to stay somewhat power-saving so I don't need a threadripper but I'm open to AMD or SP/DP Xenon. I've looked at the supermicro boards x9 and x10. IPMI would be nice as I plan on this being headless. 

 

Would love to hear suggestions or if someone has a board with processors to sell....

Share this post


Link to post

So as I understand from reading your post, these are your requirements: IPMI, lots of PCIe slots, cores for VM's, and somewhat power consumption conscious...correct?

 

If that's the case, then your really have two paths that aren't going to cost an arm and a leg (so no Xeon Scalable and AMD Epyc) and give you the PCIE slots you want (so no Xeon E and AMD Ryzen). The two paths you are left with are LGA2011-3 Xeon (older but has USB 3.0, DDR4 ECC, and is becoming cheaper very quickly) and AMD Threadripper. Here are prospective choices with those paths.

 

LGA2011-3 Xeon

- CPU: If you want VM's or lots of Docker containers running, then you need to go with the E5-26XX v3 series and probably not the E5-16XX v3 series (they have less cores but higher frequency and single thread rating). I'd stick with a single CPU since you're wanting to go lower in power consumption and it will save you money in both motherboard and CPU cost. The E5-2640 v3 is a really nice lower power CPU (it will idle much lower than its 90W TDP) to use for you. It has 8 cores/16 threads, 13,864 Passmark, and can be found Used on eBay for around $75. You could also go up to the E5-2680 v3 (18,426 Passmark, 120W TDP, 12 cores/24 threads, $150) and the E5-2690 v3 (19,240 Passmark, 135W TDP, 12 cores/24 threads, $200) if you want more power. Not to mention that any motherboard that can have a v3 processor can also take a v4 one. The E5-26XX v4 is still really expensive but these will decrease over time and give you a decent upgrade path. **Side Note: Don't buy QS/ES samples or ones from China. There are plenty of good, regular, used ones around so don't be drawn in by slightly lower costs.**

 

- Motherboard: The X10SRL-F is pretty much the standard. It has a ton of PCIE lanes, USB 3.0, IPMI, fantastic support, and plenty of used ones on eBay. This seller: https://www.ebay.com/itm/SuperMicro-X10SRL-F-ATX-Single-Socket-LGA2011-v3-DDR4-Motherboard-X10/123818059106?hash=item1cd421a562:g:n3sAAOSwwrlckRA- regularly takes around $60 lower for a one than what is listed. The I/O plate can be found on Supermicro's website for only a couple of dollars. If you want to buy it new, Provantage has them for around $260 which will also come with a warranty which is nice. Just beware that if you pick this board, you'll need to find a heatsink that fits the LGA2011-3 Narrow socket. Noctua has a 3U one that would be perfect for your Supermicro chassis (I actually have the exact chassis as you!).

 

- RAM: DDR4 ECC RDIMM's are dropping considerably in price. If you look on the Used market (eBay, ServeTheHome, or r/homelabsales), you can get some great deals. Hard to say what you'll pay because you didn't specify how much RAM you were planning on using. If you're going to do VM's and Docker containers, more is better. Also, take into account that this is a Quad Channel CPU so you'll want to use at least 4 sticks to really get your money's worth.

 

AMD Threadripper

- CPU: The 1900X is actually not a bad choice if you're planning on getting started with this setup but don't want to spend a bunch of money. It has 8 cores/16 threads, 16,108 Passmark, and can be found for about $150. You could also go with a 1920X ($250) if you want more cores. All threadripper CPU's will come with a higher idle power consumption though (starting at 180W TDP with an idle around 100W). You do, however, get higher Single Thread Ratings and a bigger upgrade path.

 

- Motherboard: If you want IPMI, this leaves you with exactly one choice: ASRock Rack X399D8A-2T. This one is actually pretty hard to find and is damn expensive. It costs about $550 (I know! Crazy right?!) and it can be found on eBay and a couple of other sites. NewEgg is out right now but it'll probably come back relatively soon. Although it costs a ton, this motherboard has everything. PCIE lanes, IPMI, 10Gig, you name it.

 

- RAM: DDR4 ECC UDIMM's are starting to go down in price and Provantage has some nice 2666mhz Kingston ram sticks for a decent price. You can also do non-ECC dimms and get better speeds if ECC isn't important to you. The only issue with Threadripper and RAM is that it really likes to have overclocked higher speeds to get the most out of the CPU. Since this is a NAS we're talking about, I don't condone overclocking ram but some people on here think it's worth it for the performance boost.

 

Conclusion

You can build a damn nice LGA2011-3 build for a lot lower cost than a Threadripper build while only suffering a little in high end performance.

 

**Personally, I really want to do a LGA2011-3 build but I just don't have the need for all those PCIE lanes so I'm probably going to go AMD Ryzen instead.**

Share this post


Link to post

Thanks!  That was an awesome reply. 

 

Edit...follow up

 

I see that supermicro MB has a physical x16 but it is electrically x8. THe quadro 2000 needs an electric x16 port. Also, I couldn't find if one of the x8 ports has bifurcation to support a card with 2 NVMe at x4 on it. Back to the drawing board

Edited by Vitalsignser

Share this post


Link to post
On 12/5/2019 at 1:00 AM, Vitalsignser said:

Thanks!  That was an awesome reply. 

 

Edit...follow up

 

I see that supermicro MB has a physical x16 but it is electrically x8. THe quadro 2000 needs an electric x16 port. Also, I couldn't find if one of the x8 ports has bifurcation to support a card with 2 NVMe at x4 on it. Back to the drawing board

You don’t need a full x16 electrical for a P2000. No GPU even really maximizes the potential PCIe 3.0 x16. If you’re just transcoding, you wouldn’t even saturate a x4 slot.

 

The X10SRL-F with newer BIOS updates does support bifurcation. It’s not a full x4x4x4x4 in a x16 slot but it does x4x4 in a x8 slot.

Share this post


Link to post

I ended up getting a great buy it now deal on the X10DRi-T. 

 

I'd like to bifurcate one of the x8 ports to x4x4 for 2 x NVMe 2TB cache drives. There are two different supermicro cards that do this. Is there a difference between the AOC-SLG3-2M2 and SLG3-2E4T ??

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.