Build questions & blog

[ChatNoir]

After a month or so on the forum, I think I have enough information to get serious about building my first unRAID server. But before ordering stuff, I'd like to run things by you to prevent beginners mistakes.

Objective:

• File server
• Media player with Plex, Emby, Jellyfin or something else on a docker
• maybe some sonarr/radarr/or other stuff, I'd would have to take the time to understand and decide about it
• possibly some WireGuard or other thing of the sort, but not right now

Out of scope:

• No gaming VM: I have a one year old gaming rig that is pretty good
• No home automation: I live in a small flat, no real need for it.

Constraints:
I cannot find a good place in my place for the server except in my living room. So, no ultra loud stuff, it will have to be a tower and if possible not be super ugly. I am looking at a Fractal Design Define 7 XL, Define XL R2 or Silverstone CS380. Maybe server specific cases ?
I will use 2x 14TB (should have them tomorrow) and once it is stable enough, I'll reuse 2x 6TB from my NAS and maybe 2 or 3x 3TB from my NAS or spare. My NAS (DS414) will end up being a repository for the backup of the unRAID and some family members. It will also probably upload the backups to some cloud service (maybe BackBlaze ?).

Hypothesis:

• I will need 1 or 2 SSDs for the array cache and maybe 1 or 2 others for the dockers once 6.9 is out.
• It would be great if I can reuse my old GTX770 for Plex conversion, I'll try
• I will probably increase my array size both from adding drives and replacing others.
• It would be nice to have at least some 2.5G LAN on the motherboard since my ISP is offering 2.5G fiber and networking on the last router. 10G LAN would be better for futureproofing.

With the basics out of the way, I do have a bunch of things to clarify regarding the architecture, the possible expansions, etc.

I will keep the original post clean and hopefully up to date, so the questions will be in the next post(s).

Edited August 3, 2020 by ChatNoir
changed crappy title

[ChatNoir]

So taking all the elements above, I started to look at hardware.
My first approach was to look a brand new stuff and the various Ryzen do offer a lot for the price compared to their Intel equivalent. Sure, they mostly lack an IGP and there are some issues (johnnie.black must be sooooo bored to link the same posts again and again :D).
3900X or 3800X seemed nice for what they offer. B450 and X470 is out for 2.5G LAN (except one ASUS board), there is some choice with 2.5G LAN.

 

But looking at the number of drives I plan on using on the short term (2+2+2 HDDs, 2 or 4 SSDs) and some future array expansion I do find some limitations for the drives since I can at best go for 8 SATA + 2 M.2.

Sure, I could use less HDD, skip the redunduncy on cache drives; but I forsee that I would have to increased my array capacity, maybe go for twin parity and from what I see on the forums, RAID1 cache seems a pretty good idea.
There is then the HBA route obviously LSI 8i boards a accessible that's not a big deal.
But I will probably need a GPU for media conversion and maybe in the future a 10G LAN.

And if that doesn't seem to work so well with X570 (or Z490 for that matter even if the IGP might be of some help) ==> GPU PCIE 16x / HBA PCIE 8x / 10G LAN PCIE x4
All those boards can either offer 16x or 8x/8x.

 

So I spent quite some time looking at second hand server stuff, so original, nobody ever does that...
The more recent stuff with DDR4 is quite rare and expensive, so I looked at DDR3 stuff. But that's where I have some questions since I am out of my depth there. I did spend some time on ebay, Intel ARK and the Supermicro website.

I saw some boards that looked interesting (PCIE only, no PCI, recent enough, etc.) : X9DAX-7F / X9DRi-F / X9DAX-iF / X9DRH-7F. Much more PCIE extensions (2x 16x + 3x 8x + 1x 4x), more SATA, SAS some times.
I saw some differences in size, number and type of SATA connectors and some have a SAS controler...

1. are the attachement points of the server mother boards compatible with standard cases ? (think so^^)
2. are those boards compatible with standard with regular PSUs ?
3. those boards mainly use SATA2 (x8) with some SATA3 (x2), does this cause any issue ? As for speed on HDDs, I do not think it is an issue, for SATA SSDs, it might be an issue. What about TRIM ? It seems to be integrated in SATA 3.1.
4. on the Supermicro website, I see mention of SCU and AHCI SATA, is there any issue with SCU ?
5. is the Broadcom 2208 controler is useful ? Can it be used as an HBA or should I skip it ?
6. I am wondering what's the best choice as for CPU if I go this way. I made speadsheets taking into account nb of threads, base clock, cache and price. Sure E5-2697v2, 2695v2 or 2690v2 are the best in this range, but I am wondering if going for a pair of 2630v2 or 2637v2 on the cheap would be a good way to startup and learn? And maybe switching for more powerful stuff later on if necessary?
7. are there mono CPU server boards that offer more PCIE lanes than modern architectures ?

[Energen]

Lots of questions here.. a few I can't really answer, don't know enough about the subjects.  Here's what I can respond to..

 

First and foremost, plan out what you want now, and for the future.  Nothing is worse than spending X amount of dollars now just to want to change everything again for better hardware a year or two from now when you start wanting to do more stuff or play around with stuff.

 

That's the boat I'm in right now, I want new mobo/cpu which will also require new RAM.  $600 or so down the tubes because I didn't know I would want to do more stuff later on.  Plan now, buy once!

 

With that said -- ok.  Your hard drives, well if you plan on having a parity drive then one of those 14TB drives has to be your parity drive because parity has to be as large as the largest drive in the array.  If you had other plans for those drives then maybe you can use them outside of the array and not have to waste one for parity...

 

The Ryzen or Intel debate.. well, it sounds like you want GPU transcoding so my suggestion might be to focus on the Intel with igpu and a supporting board (not sure about those X9 boards, the new X11 boards with c246 chipset are what's talked about recently for igpu transcoding support) that way you don't need a graphics card and can use the x8/x16 slot for a SAS card if you need more SATA connections.  Some server boards have limited pcie slots so using one for a gpu might limit your speeds for a SAS card.. probably not, but maybe.

 

If you are more set with using a gpu for transcoding (which is probably easier anyways) then get whatever board you like that suits the cpu you choose..  plan ahead. Get the most capable cpu now that you can afford, and if you can't afford it then save up a little more and get it later.  Those cpus you listed all look good.. lots of cores, decent core clock speed.. I personally like the look of a cpu over 3ghz for a new build... and for whatever it is worth I look at the single thread passmark benchmarks .. I have no idea how those benchmarks affect anything in a practical sense but those all have single thread scores below 2000.  I don't know what that means but in my planning I looked for a cpu closer to 3000.  Does it mean anything?  I don't know.. but makes me feel better.

 

You haven't mentioned RAM other than DDR3/4.. so you'll have to think of how much you want and go from there.  I looked at one of those X9 boards and it had a ton of RAM slots so plenty of room to expand later, but if you had a board with only 4 slots then you'd have to plan up front for the limited ability to upgrade.  I started with 16gb of ECC DDR4, then eventually added another 16GB... hit my limit of slots there.  On my current setup I'd have to start over and buy 32GB sticks in order to upgrade.  ECC is $$.  Do you plan on ECC?

 

The general hardware questions -- as long as it's an ATX/mATX board then they will fit a standard case.  The SATA ports are the same.  The PSU will plug in the same (size your PSU accordingly for all the drives you want to add in the future). 

 

The SCU/AHCI stuff is about the SCU sas controller .. doesn't matter for anything except that using / not using / mixing drive types could determine your SATA speeds (3gbs/6gbs).  This is what I just read about it.. don't take it as gospel.

 

That's about all I can answer..

 

Hope this gave you something to think about and didn't make it more complicated for you.  I just like to type.

 

Good luck!

 

 

[ChatNoir]

Thanks for your feedback Energen.

I am doing this topic to cover as much stuff in advance and not having any buyers remorses.

 

Regarding the drives, I didn't went in too much details because I was focusing more on the general architecture and base components choices at first.

I bought those 2x 14TB to use as data + parity at first before getting too serious and removing the other drives from the NAS. The good thing is that I can transfer all my actual data on the 14TB data drive and build up the array later.

 

On the Motherboard side, what I like on those boards is the important number of expansion slots for GPU and future use.

On the hole placement, I am asking because they are not simple ATX size but "Extended ATX", meaning that they are 13.68" x 13" (34.74cm x 33cm) or 12" x 13" (30.5cm x 33.2cm) witch is between SSI MEB and SSI EEB.

 

I guess my question on the PSU wasn't super clear. I see there are 1x 24pin power connector and 2x 8pins. I think they are the same as regular consumer MB, just wanted to confirm.

 

On the topic of RAM, same as above, I was planning to tackle that question once the architecture was set. On those plateforms, it would be silly not to use ECC RAM and possibly lots of it. I'll look into it but probably big sticks on each CPUs but I have to consider the memory channels and not spend an insane amount of money. I'll look into it during the weekend or next week.

 

If someone has any more information on this first batch of question, I'll be glad to hear it.

[ChatNoir]

Ok with Energen answers, some time on the internet and reading the supermicro manuals I have narrowed my question list.

 

1. Is there a limitation on using SATA2 with spinners ? 3Gbps looks more than enough for me, is there a catch ?
2. The Broadcom / LSI 2208 looks OK and it seems it can be flashed to IT mode. Do you see an issue ?
3. Can I expect TRIM on any of those ports ? (SATA2, SATA3, SAS)

 

 

[Vr2Io]

1. Is there a limitation on using SATA2 with spinners ? 3Gbps looks more than enough for me, is there a catch ?

No limitation, but device may negotiate in 1.5Gbps if native interface speed not available. You need ensure enough bandwidth can share for all disks ( depends on how many disk serve )

 

2. The Broadcom / LSI 2208 looks OK and it seems it can be flashed to IT mode. Do you see an issue ?

Best use 2008 or 2308 controller, those are common in market.

 

3. Can I expect TRIM on any of those ports ? (SATA2, SATA3, SAS)

Not support TRIM, only high-end controller may support TRIM.

[ChatNoir]

So, with all the previous discussions I did more researches and I went with a more recent generation of board and CPU.

I also shifted from server board to workstation since I the specs and prices were more in line with what I wanted.

 

I got :

MB : Supermicro X10SRA-F (IPMI and VGA output, plus it was cheaper than the regular SRA boards ^^)

CPU : Xeon E5-2697v3, I doubt I'll ever lack horsepower but I could upgrade for a v4 in a few years if needed

RAM : 4x 16GB Samsung ECC RDIMM

GPU : I will make some tests to see if and how I can use the VGA from the MB and I will try to use my old GTX770 for encoding

Drives : 2x shucked WD140EMFZ  (will be 1 parity and 1 data) and I'll empty my Syno once every thing is running smoothly

SSDs : one WD Red 500GB for the moment, I will probably double it to have some RAID in a few weeks

 

Thanks for your help, hopefully everything goes well and I will not have to ask any questions on the forums (yeah ... not 100% sure here ^^)

Edited July 31, 2020 by ChatNoir

[ChatNoir]

Alright, it's alive !

Some BIOS issues during the weekend prevented me to progress much, but it is now OK with the help of a Supermicro rep.

 

Booted up during my lunch break and the HDD I plugged is not recognized, only the SSD. It was working fine in my USB dock so I suppose that I have to put Kapton on the 3.3V pin of the drives ?

[ChatNoir]

As I don't have Kapton on hand, I won't have an array for a few days and I will wait for the array and the initial data transfer before assigning the cache.

 

So, not much to do but better be productive in the mean time, what can I do to prepare the server ?

 

I was thinking about :

- configure notifications

- install and configure some plugins (unassigned devices, Preclear, Fix common problems, etc.)

- make sure the reported temp are OK

 

Any advices ?