Looking for feedback

[hansolo77]

I just got off the direct chat with a rep named Nate at "TheServerStore".  He was so nice.  I've been looking to build a new server using the SuperMicro 846 chassis.  I had an opportunity last week to order a barebone system from them but was hesitant and have been regretting it ever since.  They sold out of the barebone systems, and all they seem to have left is 5 of their prebuilt systems (with motherboard/processor/ram/HBA).  Nate was kind enough though to offer me a barebone system from one of their prebuilt systems.  Basically just gutting it out.  I don't recommend going this route though, because he mentioned they are trying to keep these prebuilt systems all together.  It was an extremely kind gesture on his part.  But, I did it.  He offered and I took the offer.  I've already paid for it now, so it's on it's way.  But, I need some real world feedback.

 

This chassis will be legit bare, except for 2 920W SQ power supplies and the backplane (also the PDB, caddies, etc).  I know there are eBay sellers offering parts, so I need to know what to get next.  I'm still figuring out my mobo/cpu/ram stuff.  But for the HBA/Controller, what should I get?  Also, the backplane he's shipping will be a BPN-SAS2-846EL1.  Is this good or should I look to upgrade that too?

 

Here's what I have going.  I'm using 24 "NAS" drives.  These aren't legit server-grade SAS drives.  Just consumer level SATA.  I'd like to have as much speed as I can get for a reasonable price.  I think I remember (although I could be wrong) seeing a YouTube video that said this backplane is actually capable of "SAS3" if the controller supports it.  Meaning I could get the faster throughput you get with that.  If that's not true, should I consider upgrading the backplane?

 

As for the controller, I'm totally lost here.  The prebuilt systems come with a LSI 9266-8i.  Since the barebone won't have that, I need to buy.  Is this a card I should get, or is there something better that will work with this backplane?  This eBay seller seems to have a lot of the goods, but I don't know what to get.  Opinions/Thoughts/Feedback is welcome!

 

Thanks again everybody for helping me with my server build!

 

[ramblinreck47]

You picked a really good server case. I have a Supermicro 835 and 836, and I love them. They're built like tanks and they run great.

 

To address a couple of your concerns:

 

- The barebones 846's are really hard to find right now so the fact that you were able to get one without having to purchase a complete system is a really good deal (unless you had to pay a lot). I put my name on the waiting list to buy one eventually but I'm not in any hurry to replace my 836 when I still have some drive bays open.

- That SAS2 backplane has an expander with it and is preferred. You can use only 1 HBA and use both ports to connect to the backplane to get full use out of that PCIE lane.

- Unless you have a really hefty budget, a bunch of really fast drives, and want the fastest possible setup, I'd stay away from buying a SAS3 backplane. They're very expensive and if you're not ready to spend serious money, it's not worth it.

- For HBA, you shouldn't go with the LSI 9266-8i that comes with. It's a SAS2208 chipset on an actual RAID card and it's harder to flash than a SAS2008 or SAS23008 HBA.

- You found a good store to buy a HBA from. I have a 9211-8i that I bought from Art of the Server and it's been great. For your system though, I'd recommend you get a card with a SAS2308 or SAS3008 chipset. You're going to need a PCIE 3.0 x8 bandwidth to get the most speed out of your drives especially when doing a parity check. You'll finish faster. This is a good in expensive one that should work with the two SAS cables that will come with your 846: https://www.ebay.com/itm/Lenovo-03X4446-9217-8i-6Gbps-SAS-PCI-E-3-0-HBA-P20-IT-Mode-ZFS-FreeNAS-unRAID/164248205524?hash=item263df4b8d4:g:J38AAOSw8Sde6XeW

- If you're going to go with a non-Supermicro motherboard, you'll need a special cable so you can use the front panel connector: https://store.supermicro.com/supermicro-15cm-16-pin-front-control-panel-split-extension-cable-cbl-0084l.html

- If you ever want to quiet the chassis down, there is a non-destructive way of doing it that'll cost a little over $100. Replace the 3 fans on the fan wall with FAN-0074L4's (trim the webbing on the side of the fan with some pliers...it pops right off) and replace the back 2 fans with FAN-0104L4's. They'll push lots of air without sounding like a jet engine.

[hansolo77]

Thanks so much for your input!  We spoke previously in another thread when I was starting off.  Nice to see you're still willing to help!

 

Yeah "TheServerStore" (dotcom) has listings for the 846 barebone but they have none left in stock.  I've been waiting for them to get more in but after talking with Nate it's quite possible they may not get more in, it's however the market goes.  The barebone listing they have includes the drive caddies, backplane, and power supplies.  This is $299.  They offer a switch out of the power supplies to SQ models which are more quiet.  This change makes the cost go up to $499 ($100 each power supply).  This is what I was going to buy if they ever got more in.  They also offer prebuilt systems in the +$550 range that include motherboard/cpu/ram/controller.  I was going to do that instead today, but thought I'd give the online chat a chance.  I told Nate what my needs were, and he offered to essentially gut a prebuilt system for me and sell me a barebone with the SQ power supplies for $499.  Total cost after shipping was about $550.  So I might have been better off getting the prebuilt afterall, since it would have come with the controller, but the power supplies wouldn't be the quiet ones, and I'd be stuck with a motherboard combo I'd have no use for.

 

As for the backplane..  I've been doing some research into that.  Trying to figure out what the best controller would be for it.  I guess I was mistaken about the SAS3/12gb thing.  It looks like that only works on the "TQ?" revision that basically has passthrough SATA ports.  Since this one has the 8087 and expander, I'm limited to the SAS2/6gb.  If I'm wrong here, please let me know!

 

Honestly, I'm not sure what my current setup has.  It's a Norco 4224, with a controller card and expander card.  I don't know what it's speeds/capabilities are, but I feel like it's a little slow.  It could be the controller or the expander though.  I just checked the specs on the backplane for the Norco and it reads 6G, same as the 846.  When it comes down to it though, a 12G card and backplane, while nice, probably wouldn't be noticeable with consumer SATA drives anyway.

 

I'm looking at your suggested controller and I'm a bit lost as to why you chose that one.  You also mentioned SAS3008, and he sells this https://www.ebay.com/itm/LSI-SAS9302-8i-ServeRAID-N2215-12Gbps-SAS-HBA-P16-IT-mode-for-ZFS-FreeNAS-unRAID/163040642059?hash=item25f5fac40b:g:-1QAAOSw6Wlezxw5:sc:USPSPriorityMailPaddedFlatRateEnvelope!43228!US!-1

Would that be a better buy for the speed, or would it be useless without the SAS3 backplane?

[ramblinreck47]

4 hours ago, hansolo77 said:

As for the backplane..  I've been doing some research into that.  Trying to figure out what the best controller would be for it.  I guess I was mistaken about the SAS3/12gb thing.  It looks like that only works on the "TQ?" revision that basically has passthrough SATA ports.  Since this one has the 8087 and expander, I'm limited to the SAS2/6gb.  If I'm wrong here, please let me know!

The Supermicro 846 backplanes essentially comes in 3 varieties:

 

"TQ" - 24 x Direct SATA3 connections...you need 24 SATA ports from your motherboard and HBA to get connections to all 24 drives...it gives you flexibility to add as you go and if you can find them, they are really cheap...the issue really comes with all the cables needed to make it happen and multiple HBA's or an expander

 

"A" - 6 x SAS ports connections...you need 6 SAS ports from either your motherboard or HBA's...if you have 3 x 2-port HBA's, you'll get a long of bandwidth and 6 total cables isn't a lot to deal with...the issue is the amount of PCIE lanes you're going to need to have available unless you're willing to spend some money on a 9211-16i (4 ports)  + a 9211-8i (2 ports) or 9305-24i (6 ports) or an expander...these backplanes are fairly cheap and very similar to probably how your Norco was setup

 

"SAS(X)-846E1" - Built in expander only needs 2 SAS ports to get data from all 24 drives...SAS2 will give the best balance of speed and cost with SAS1 being rather limited and SAS3 costing way way more...you keep cables to a minimum (only 2) and you're really only limited by the speed/bandwidth of the HBA you choose and the PCIE lane it is on

 

=> I think the SAS2-846E1 is a great model to go with because it keeps the amount of PCIE lanes taken up to a minimum on your motherboard and gives you a lot of flexibility to move up to a better HBA if you need more speed in the future.

[ramblinreck47]

5 hours ago, hansolo77 said:

Honestly, I'm not sure what my current setup has.  It's a Norco 4224, with a controller card and expander card.  I don't know what it's speeds/capabilities are, but I feel like it's a little slow.  It could be the controller or the expander though.  I just checked the specs on the backplane for the Norco and it reads 6G, same as the 846.  When it comes down to it though, a 12G card and backplane, while nice, probably wouldn't be noticeable with consumer SATA drives anyway.

 

I'm looking at your suggested controller and I'm a bit lost as to why you chose that one.  You also mentioned SAS3008, and he sells this https://www.ebay.com/itm/LSI-SAS9302-8i-ServeRAID-N2215-12Gbps-SAS-HBA-P16-IT-mode-for-ZFS-FreeNAS-unRAID/163040642059?hash=item25f5fac40b:g:-1QAAOSw6Wlezxw5:sc:USPSPriorityMailPaddedFlatRateEnvelope!43228!US!-1

Would that be a better buy for the speed, or would it be useless without the SAS3 backplane?

Look at it this way, a LSI 9211-8i (or SAS2008 variant) takes up a PCIE 2.0 x8 lane with a total bandwidth 4000 MB/s. If you were to use an expander to split those two SAS port links over 24 drives, you would get roughly a 140 MB/s (built in overhead that keeps it from being higher) from each drive if they are all running at the same time. That's not bad but a 5400rpm HDD should be running close to 185 MB/s at full speed. You're limited. If, however, you replace that 9211-8i with a 9300-8i/9207-8i (or SAS2308/SAS3008 variant) it takes up a PCIE 3.0 x8 lane with a total bandwidth of 4800 MB/s. Expanding out to 24 drives, that would be approximately 185 MB/s.

 

The reason I suggested the 9207-8i (SAS2308) equivalent is because it uses the same SAS connectors that the cables on your motherboard will already plug into, has higher speed than SAS2008, and is half the price of SAS3008 (still the same PCIE 3.0 x8). If you were to go to SAS3008, you would need to spend considerably more money and would need 2 cables like these (https://store.supermicro.com/cable/supermicro-minisas-to-minisas-hd-50cm-cable-cbl-sast-0508-02.html). The only benefit you would be getting is support for TRIM (not needed if you don't plan on connecting an SSD to the card) and a slightly newer chipset. That's all you get for double the price. 

Edited July 10, 2020 by ramblinreck47

[hansolo77]

Thanks again for the information!

 

I'm going to stick with the current backplane then.   I was also reading that the SAS3 backplane would only benefit if I'm using 24 SSD drives, and even then only if they're all being accessed at the same time.  I highly doubt I would ever be using all drives simultaneously let alone have them all SSD's.

 

For the controller, I get what you're saying.  I might still need to get cables, I'm not sure if the chassis they send me will have them.  And you're right, for half the cost I'll get something that's already compatible, without needing to use up extra slots which will be a premium once I start building out.  I'm still on the fence about the motherboard to get.  After looking at the chassis model revision, it looks like I have to rethink my plans.

 

Originally, I was going to mount my 2x12tb drives on the power supply wall using mounting brackets.  Those would be my parity drives.  After looking at the manual for the chassis, it looks like the revision I'm probably getting (B) will only support 1x3.5" drive and 1 or 2 SSD drives.  I guess it has to do with the expansion port on the back being equipped for an optional DVD drive.  So I'm kinda bummed about that, not being able to mount both of my 3.5" drives there.  That means I will have to use up one of my 24 bays for the other parity drive.  Unless I can come up with some kind of hack to mount the other drive.

 

I was also planning on getting an MSI X50 Unify motherboard to hold the Ryzen 3900X you suggested.  I narrowed down to this because of the available PCIe slots and 3 M.2 slots.  My thoughts were to use 2 of the M.2 slots to hold my cache and the 3rd to hold either the Plex metadata or the OS.  But I'm not sure about this.  I don't think I need such a huge drive (like 128gb minimum) for unRaid OS.  But I also don't really want to use USB either because I've had USB flash drives fail on me.  My current system has one of these 8gb USB2 drives.  I used it to install the Windows Server install software, so I could have an emergency rescue.  Problem is, it only works about 1/2 the time; and as luck would have it, not at all when I needed it.  So if I go that route again, I'm really going to need a solid/stable drive.  Any suggestions?