Recommendations for new build


Recommended Posts

I've hit the point of upgrading my server a lot sooner than I expected (less than a year).  I'm seeing a fair amount of call traces and issues with maxing out my RAM, and there's not much DDR2 on the market for decent prices.  Given that the hardware (well, motherboard, RAM, and CPU at least) is well over 8 years old at this point, I've decided to start the upgrading process.

 

Now I have a fair amount of experience with building PCs, and have built a good many.  However, I have essentially zero experience with server class hardware, and I've been having trouble deciding what to get.

 

Right now, I'm running this:

CPU: Intel i5-2500k (no oc)

RAM: Corsair Dominator DDR2 16Gb (or something similar to those); I have all 4 slots filled with these, so I would need 4 sticks of 8Gb DDR2, which is not cheap if it can be found.

Motherboard: Asus P8P67PRO (or whatever)

2 ssds, 5 HDDs (two older ssds, one is 60Gb..., the parity and 2 other drives are brand new, the other 2 are older HDDs from my old pc)

500W Corsair PSU (new and good quality)

Standard size mid-tower case with a couple free bays

 

Basically, I built a new gaming PC and used the leftover parts from my old gaming PC to build the server.  The hardware does still seem solid, but is definitely at the edge of it's life.

 

So I figure it's time to set up with some new core components, and to get something with a bit more power and RAM.

 

I would like to keep using my current case, since it fits in the cabinet I made (with fans!), and because it's still fairly new.  I definitely want to do a rack build in the future, but apartment life is not friendly to that, and I don't have space anyways, so that's not an option for the moment and I don't know when it will be.  So it would be a nice bonus if I could port the new build to a rack case, but since that could be 5 or 10 years away, it's not a requirement.

 

My main uses of the server is Plex streaming (1 or 2 clients) with Sab/sonarr/radarr for downloading, and data storage/backup (crashplan pro).  Eventually I may have more uses for this, but I don't at the moment.

 

Obviously I don't need a ton of power, but I was still wanting to move to a xeon build with more RAM expansion options (bumping over 16Gb at the moment, but can't really see me needing more than 32Gb to be honest).

 

Power usage is a consideration, but I have no need of a super-low powered build.  That said, I don't want to just piss away money on power bills either.  My current build comes in around 100W, and that's fairly normal from what I have seen of unraid builds.  Lower would be great, but that area would be good.

 

Motherboard wise, I obviously need video output somehow, but I don't actually need a GPU with power since I don't really use VMs or game on it.  That said, I would like to have the option if possible.  Not a deal breaker though.

I also need a PCI slot for at least 1 gigabit NIC (I'd like to have an additional 2 ethernet ports to run a VM backup of pfsense to take over if my SG-1000 goes down, and possibly to take over DNS filtering since the SG-1000 does NOT have much power), but since 4 port gigabits tend to be pricey, I may get 2 instead.  That and I already have a card with 2 ports, so it makes more since to just buy a 2nd one instead of a 4 port.

I know I can get cards with additional SATA ports, so that's not a huge deal, but I will need the ability to add more hard drives, though eventually I will scrap the 2 older ones (500G and 1T, so pretty small) and replace them with bigger ones, but I'm sure I will keep adding drives until I need a rack server.

 

What would be a good choice for a xeon build?  More cores at a lower speed would be preferred to higher speed with less cores (unless that's wrong, but I'm pretty sure).  More cores would make more sense I think, since I can assign them to VMs/containers if I want.

Are there other good cpu options that I could use instead?  Maybe a Ryzen?

Do I need ECC RAM?  Would I have a choice with server components?  Never used it before.

Not sure I need a dual CPU build, but it would be kind of cool.  I imagine power usage would be higher though.  Single would probably be preferred just because of cost, but no real preference (unless power consumption would be significantly higher).

 

Not necessarily looking for someone to put everything together for me, but I'd love to see what has worked well for you in the past.  I would also welcome reading material that would assist if you want to link it.

 

Thanks a lot!

  • Like 1
Link to comment

Hi -

 

It sounds like your requirements are basic NAS, a couple of Plex streams and a VM.  If that's the case a Sky Lake or Kaby Lake Core i7 or E3 Xeon would be fine.  If you can wait, the 6 core Coffee Lake Core i7's will be available soon and hopefully they will announce a companion E3 Xeon line.  ECC is very nice to have in an always-on fault tolerant server.

 

You haven't mentioned anything that would require a Ryzen or dual CPU board, or a need for a large number of cores.  Things get complicated for those options, though for different reasons.  At point you pluralized to "VMs", though.  That's an important distinction - are you planning to go beyond a single backup pfsense VM?

Link to comment
14 hours ago, tdallen said:

It sounds like your requirements are basic NAS, a couple of Plex streams and a VM.  If that's the case a Sky Lake or Kaby Lake Core i7 or E3 Xeon would be fine.  If you can wait, the 6 core Coffee Lake Core i7's will be available soon and hopefully they will announce a companion E3 Xeon line.  ECC is very nice to have in an always-on fault tolerant server.

 

You haven't mentioned anything that would require a Ryzen or dual CPU board, or a need for a large number of cores.  Things get complicated for those options, though for different reasons.  At point you pluralized to "VMs", though.  That's an important distinction - are you planning to go beyond a single backup pfsense VM?

The short answer is that I don't know.

 

Right now I have no VMs at all, though I want one for pfsense.  That said, I may very well decide to add more in the future, so it's more for future proofing.

Of course, it would also be fun to setup a VM with active directory and maybe some client VMs for testing/learning/play (I work in IT, though have only been working in the industry a few years), so I sort of have ideas, but I have never been bothered to setup active directory or actually do anything.  Now I know I would not have the power currently, but that's something I would have found out pretty quick.

I've also thought it might be fun to have a windows VM with a GPU that I could start and lets friends remote in and game on while they are visiting, but I'd probably never end up using it and don't want the extra power draw from the GPU (probably not that much, but I just didn't need it).

 

Was thinking xeon mainly because lots of cores and I wanted to get an actual server class cpu.  Not really much more reason than that (well, future proofing), although I'd be looking for an older/cheaper one rather than a high end one.  Obviously I don't need a dual cpu build, but it would be pretty cool.  I assume they would draw roughly twice the power of a singe cpu, but two low power CPUs may end up being better / more efficient than a single (maybe, I dunno).

 

ECC does sound good, but I've never had a RAM issue in my life (aside from running out of it).  Not really sure what the price difference is, but I don't mind paying a little more for it.  Keeping with the server class hardware theme, ECC would make sense.  Again, I don't really have a better reason for needing it.  I'd skip it if it's like double the price, but I'm willing to pay a little more for it even if I'm not really going to need it to keep with the server class components.

 

So you are totally right,  I could get a newer i3/5 and more RAM and be working fine.  But that's not as much fun or as interesting, and limits my future options a lot more than an older xeon.  I also don't need speed so much as more cores.  Mine currently maxes out all 4 (so at least it's spreading the load unlike windows...), so I figure more cores makes more sense than more speed.

That said, I still need to find something else I can do with it that would utilize that power, but that's part of the experience.

 

Another question that I just realized: would I need a new PSU as well?  No clue if a server motherboard uses the same types of connectors and what not.

Link to comment
6 hours ago, lordbob75 said:

I also don't need speed so much as more cores.  Mine currently maxes out all 4

You're maxing out all 4 cores because you're running a 6466 Passmark CPU with 4 non-hyperthreaded cores running a heavy workload (Plex) that favors multi-threaded operations.  On the other hand, if you were running a gaming VM (for instance) you'd wish you had fast cores since gaming favors faster single threaded operations.  Not all work favors a big core count and cores aren't the only measure of the work a chip can do.  For instance, despite the fact that it has the same core count as your i5-2500k, a 12,113 Passmark Core i7-7700k can do almost twice the work simply because it is newer and faster and supports hyperthreading.

 

There's a big difference between E3 Xeons and E5 Xeons.  The E3 Xeons are basically rebranded Core i7s that support unbuffered ECC RAM.  The E5 Xeon is a different chip entirely, though even within the E5 line you have the 16xx series and the 26xx series.  The 16xx series is more of a workstation chip that favors fewer, faster cores (but still more than the E3) while the 26xx series is more of a enterprise server CPU that favors many cores.  If you just want a monster E5 system, then go for it - it's a great platform and there are lots of threads here about building an unRAID server with used E5's that come from enterprise decomms.  But I wouldn't necessarily call buying a several year old 2.0GHz, 20 core CPU that runs at 140w future-proofing, and I definitely wouldn't say that about the older dual CPU E, X, and W series systems.  A bargain, a great value, a fun system to experiment with - yes, but not really future proofing.

 

The ideal way to approach this is figuring out what work you want to be able to do, and then match the characteristics of the chip against that work (as well as your budget).  If you're not sure, I'd generally recommend a newer system as more future proof than an older system.  But if you just want a massive test bed then go for it - that's fun too.

Link to comment

Gaming is definitely not the purpose of this server, so I'd rather go with a cpu better for handling server work like vms and docker containers.  If I ever set up gaming capabilities, it's not going to be playing the latest AAA title with max graphics anyways, so it is what it is.  It's not something I care about at all for this build.

 

Plex is definitely a main usage, and I will for sure be setting up more docker containers in the future.  I think I definitely want to work under the assumption that I will have multiple VMs in the future too.  That is the primary purpose for this build, and I definitely would prefer to go with server oriented hardware.

 

I was not really sure, but I've been thinking about it.  I appreciate the quick explanation of the xeon line, and I took a look at the passmark page.  An E5 for around $150-225 would be ideal I think.  The ryzen actually looks like a decent choice as well.

 

Your line about the age makes me wonder about the E5 chip I was looking at, but it had a way higher score than my current one...  Though it doesn't look like it would be easy or as cheap as it showed on passmark for a new one.

 

I guess a better question then would be, what chip would you recommend in the 2-300 dollar range that would be a good choice and would be decently future proofed?  Or is that budget way too low for that?  I'm willing to put some bucks into this, but if the only decent choice is a cpu that runs for $500-1000, I may need to rethink some things...

That said, any thoughts on the ryzen?  I've never really gone with AMD chips before, but I've heard pretty good things about it and the price is pretty reasonable.  If there isn't a xeon that I can stomach the price of, would it make a good alternative choice?  I'm thinking it would, but haven't seen much about people using it with unraid.

Link to comment

You can get a Kaby Lake Xeon E3-1245 v6 for around $300 (if you don't need the integrated graphics, you can opt for a model that ends in 0 instead of 5).  It's the latest generation of the E3 Xeons with four cores and eight threads and it supports unbuffered ECC RAM.  This CPU has a passmark of roughly 10,000.  If you have a Plex Pass, you can take advantage of hardware encoding and run more transcoding streams in Plex with lower CPU usage.  One user reported running 9 simultaneous Plex transcoding streams with hardware encoding enabled.  I have personally tried it with four transcoding streams with less than 40% CPU usage. I also run two VMs (Win 10 and Ubuntu) and several dockers in addition to Plex but, they are not always on. See my sig for my hardware.

 

Coffee Lake i7s (six cores and 12 threads) are released at a price of $380. They support socket 1151 like Skylake and Kaby Lake, but, they require a new 300 series chipset. However, there is no Xeon E3 equivalent supporting ECC RAM and there likely will not be.  It appears that Intel is going a different direction now with Xeon processors and both the E3 and E5 lines may be "retired."

 

Now that the NPT problem that affects GPU passthrough in VMs appears to have a fix, Ryzen CPUs are a good choice as well.  You can get a Ryzen 1700x for the same $300.  This CPU has double the cores and threads of the Xeon E3 (8 cores and 16 threads).  Ryzen chips do not have integrated GPUs, but, they do, supposedly, support ECC RAM.  This CPU has a passmark score of around 15,000.

 

I have a feeling the next year is going to see more powerful chips at lower prices thanks to AMD being back in the game and putting pressure on Intel.

Edited by Hoopster
  • Like 1
Link to comment
21 hours ago, Hoopster said:

You can get a Kaby Lake Xeon E3-1245 v6 for around $300 (if you don't need the integrated graphics, you can opt for a model that ends in 0 instead of 5).  It's the latest generation of the E3 Xeons with four cores and eight threads and it supports unbuffered ECC RAM.  This CPU has a passmark of roughly 10,000.  If you have a Plex Pass, you can take advantage of hardware encoding and run more transcoding streams in Plex with lower CPU usage.  One user reported running 9 simultaneous Plex transcoding streams with hardware encoding enabled.  I have personally tried it with four transcoding streams with less than 40% CPU usage. I also run two VMs (Win 10 and Ubuntu) and several dockers in addition to Plex but, they are not always on. See my sig for my hardware.

 

Coffee Lake i7s (six cores and 12 threads) are released at a price of $380. They support socket 1151 like Skylake and Kaby Lake, but, they require a new 300 series chipset. However, there is no Xeon E3 equivalent supporting ECC RAM and there likely will not be.  It appears that Intel is going a different direction now with Xeon processors and both the E3 and E5 lines may be "retired."

 

Now that the NPT problem that affects GPU passthrough in VMs appears to have a fix, Ryzen CPUs are a good choice as well.  You can get a Ryzen 1700x for the same $300.  This CPU has double the cores and threads of the Xeon E3 (8 cores and 16 threads).  Ryzen chips do not have integrated GPUs, but, they do, supposedly, support ECC RAM.  This CPU has a passmark score of around 15,000.

 

I have a feeling the next year is going to see more powerful chips at lower prices thanks to AMD being back in the game and putting pressure on Intel.

I am definitely liking the sound of the ryzen a bit more, but as you say it may very well be worth the wait for some new chips next year.

I can afford to wait a bit, plus I can put together an actual budget for this.

 

12 hours ago, tdallen said:

Affordable E5 chips are eBay purchases, typically from enterprise servers which are being decommissioned. Be careful of ES chips, which are engineering samples.  

Got it.  Not sure I really want to go that route, but it is an option.

 

I appreciate all the cpu advice from you guys, but any tips on RAM and motherboards?  I'm not sure what to look for in those for this use case (though I'm sure it's similar to consumer stuff), and there seem to be a lot of options.  Single cpu board is probably the best choice, but what about chipset choices or what not?  I'll probably be needing a SATA card soon as well, unless these motherboards come with more than 8 ports?

Anything special to look for in ECC RAM, or is it the same things I would look for in a gaming PC?  I would imagine the main factors are frequency and time, not so much latency or whatnot.

 

Again, I appreciate the help.

Link to comment
On 10/28/2017 at 9:05 PM, lordbob75 said:

I appreciate all the cpu advice from you guys, but any tips on RAM and motherboards?  I'm not sure what to look for in those for this use case (though I'm sure it's similar to consumer stuff), and there seem to be a lot of options.  Single cpu board is probably the best choice, but what about chipset choices or what not?  I'll probably be needing a SATA card soon as well, unless these motherboards come with more than 8 ports?

Anything special to look for in ECC RAM, or is it the same things I would look for in a gaming PC?  I would imagine the main factors are frequency and time, not so much latency or whatnot.

 

CPU selection is where you need to start.  That will help you focus in on the appropriate motherboard, chipset and RAM.  All must support ECC RAM if that is what you want.  It is recommended in always-on server situations, but, it is not an absolute necessity.  However, if you choose a server CPU and a server motherboard, ECC RAM just kind of makes sense.  Unless you are doing heavy-duty gaming, shelling out big bucks for the fastest RAM is really not necessary.  Just go with a good moderate-speed RAM that is supported by the CPU and motherboard.

 

Of course, you could choose a motherboard you like with the features you need and then choose a supported CPU; however, I think the CPU is the most important consideration, so, I start there.  Since CPUs require a particular chipset, that will determine which motherboards you can use. 

 

I see you already have several hard drives you likely want to keep using, but, when I opted to build a new server I decided to replace the hard drives as well and go much bigger.  I personally would rather use fewer hard drives of a larger size than a larger number of smaller hard drives.  I went from 3TB drives to 8TB drives and re-purposed the smaller drives in my backup/test server.  Having fewer drives also means that you likely don't have to consider dual parity and it lessens the risk of more than one drive failing at the same time.  Fewer drives also means that, depending on the number of SATA ports on the MB, you can add a lot of storage before you need to worry about a PCI-E SATA card.  My MB has 8 SATA ports and I currently use only 5 with 1 parity drive, 3 data drives and 1 SSD cache drives.

 

Personally, I am very interested in what happens with Ryzen /Epyc (AMD server CPUs) in the future.  Because I prefer small servers, I like to build in the Mini-ITX form factor.  That pretty much requires a CPU with integrated graphics or a motherboard with built-in graphics.  Other than mobile chips, AMD does not yet offer that in Ryzen/Epyc line; however, who knows what's coming.  My next build will have a minimum of 8 cores/16 threads and right now AMD is definitely winning the price/performance crown in that slot .  Right now, my 4 core/8 thread Skylake Xeon is serving my needs well and I have time to wait and see.

 

 

Edited by Hoopster
Link to comment

For RAM, in a server the first thing I look for is stability.  After that, the next thing I look for is stability.

 

Pick something from the motherboard manufacturer's HCL or as close as possible.  Research what others are using for that motherboard. Speed is important but I never overclock the CPU or RAM on a server.  Also watch for what type of ECC RAM is required - unbuffered or registered.

Link to comment
On 10/29/2017 at 6:20 AM, tdallen said:

For RAM, in a server the first thing I look for is stability.  After that, the next thing I look for is stability.

 

Pick something from the motherboard manufacturer's HCL or as close as possible.  Research what others are using for that motherboard. Speed is important but I never overclock the CPU or RAM on a server.  Also watch for what type of ECC RAM is required - unbuffered or registered.

Thanks, I will keep that in mind.

 

On 10/28/2017 at 9:18 PM, Hoopster said:

Personally, I am very interested in what happens with Ryzen /Epyc (AMD server CPUs) in the future.  Because I prefer small servers, I like to build in the Mini-ITX form factor.  That pretty much requires a CPU with integrated graphics.  Other than mobile chips, AMD does not yet offer that in Ryzen/Epyc line; however, who knows what's coming.  My next build will have a minimum of 8 cores/16 threads and right now AMD is definitely winning the price/performance crown in that slot .  Right now, my 4 core/8 thread Skylake Xeon is serving my needs well and I have time to wait and see.

I'll definitely pick a CPU first, but it's a tough choice.  I'm considering a newer E3 (v4/5/6).  It would be nice to have more threads, but I'm not sure I want to shell out for an E5, if I can even find an affordable new one, not sure I want to get a used one.

I'm curious about your AMD comment, what sort of options do they have in the 8 core area?  All I know about is the ryzen, but I thought those had 6 cores.  I'd definitely consider an AMD chip as well.  Not sure how long I want to wait on the upgrade though, even if AMD comes out with an awesome new 8 core chip, it will probably be more than I want to spend and who knows how long it will be before they introduce cheaper versions.

Link to comment
On 11/3/2017 at 10:50 PM, lordbob75 said:

I'm curious about your AMD comment, what sort of options do they have in the 8 core area?  All I know about is the ryzen, but I thought those had 6 cores.  I'd definitely consider an AMD chip as well.  Not sure how long I want to wait on the upgrade though, even if AMD comes out with an awesome new 8 core chip, it will probably be more than I want to spend and who knows how long it will be before they introduce cheaper versions.

 

All Ryzen 7 CPUs (1700, 1700X, 1800X) are 8 core/16 thread CPUs.  The Ryzen 5 series are 4-6 core/8-12 thread and the Ryzen 3 are 4 core/8 thread.

 

The Ryzen 7 1700/1700X (8 cores/16 threads) compares favorably in price to the 4 core/8 thread Skylake/Kaby Lake (E3 v5 and v6) Xeons.

 

The advantage to the Xeons if you are planning on running a Plex server is that the Skylake/Kaby Lake Xeons with  integrated GPUs support Plex hardware transcoding.  I am very happy with my E3 1245 v5 Xeon and Plex hardware transcoding is great, but, I will be keeping an eye on the Ryzen/Epyc chips over the next year or so.

Edited by Hoopster
Link to comment

I've had good results with Supermicro motherboards.  For the E3-1200v5/v6 processors, something like their X11SSL works well, with 6 SATA ports.  Bump up to the X11SSM and you get 8 SATA ports.  The top E3/v6 Xeon will come close to doubling the processing power of your current processor.  The v5's will be a bit more affordable.

 

To go past 4C/8T in a Xeon I suggest the E5-16/2600-v3/v4 series.  The E5-1650v4 has good bang for the buck, at well over double your current processing power, and can be found at eBay in the $500 range.  Drop back to the v3 series for even more cost effectiveness.  Motherboards will be more expensive; even the smallest Supermicro model (X10SRM-F) will run you over $200 unless you find a good deal.  Plenty of SATA ports though.

 

For memory I tend towards Crucial, Samsung, and occasionally Hynix.

 

EDIT: Forgot to mention: onboard VGA for the motherboards I mentioned . . . no graphics card needed.

Edited by bobkart
Link to comment
  • 2 weeks later...

Hey, thanks for all the help guys.  Learned a lot about server CPUs.

 

I was trolling craigslist for server stuff and ran across a really good deal on a lightly used server with these specs:

Dell T320 Server
E5-2470 v2 XEON
10 Cores
20 Threads

64GB Ram
256GB SSD
2TB Hard Drive

Other Specs:

Brand: Dell
Product Line: PowerEdge
Product Type: Workstation
Form Factor: 5U (tower)
Processor Cache: Up to 2.5MB
Memory: Fully Expandable up to 192 GB of PC3-10600R or PC3-8500R memory
Hard Drives: Up to 8x 3.5" SAS/SATA hard drives
Maximum Internal Storage: Up to 32TB
I/O Slots: 5 PCIe slots:
Remote Management: iDRAC 7, iDRAC 7 Express, or iDRAC 7 Enterprise

 

Grabbed it for $550, which I think was a pretty good deal overall.  Should be pretty perfect for my uses, and was reasonably priced enough that if it doesn't work out it won't be the end of the world, but I think it will with all those cores.  Definitely got super lucky stumbling across it right after he posted it.  (It was a good deal, right?)

 

Again, I appreciate all the help.  I feel like I have a lot better understanding of server CPUs now.

 

Now I just need to figure out how to transfer my setup over to it without losing all my data....

Edited by lordbob75
Link to comment

Which HBA did it come with? Those dell servers usually have RAID controllers and many of them can't do JBOD mode which is what you need for unRAID. Some of them like an H310 can be flashed to work with unRAID, but older ones like a Perc 6i have to be replaced. Unfortunately these servers don't have usually more then a couple of onboard SATA ports either.

Link to comment
2 minutes ago, ashman70 said:

Which HBA did it come with? Those dell servers usually have RAID controllers and many of them can't do JBOD mode which is what you need for unRAID. Some of them like an H310 can be flashed to work with unRAID, but older ones like a Perc 6i have to be replaced. Unfortunately these servers don't have usually more then a couple of onboard SATA ports either.

Uh oh, way to burst my bubble of excitement....  How can I tell from BIOS?  Or do i need to boot to unraid?

Link to comment
8 minutes ago, ashman70 said:

You can look up the serial or service tag at Dell.com or just open it up, but dell.com should tell you at least what it shipped with. Unfortunately it's a pretty important piece for unRAID. Sorry to burst your bubble of excitement, nice score though, that is a nice server.

Well I checked both but can't find anything and I'm not 100% sure what I'm looking for.  Do they call it something else?  What component am I looking at in the server, the HDD controller?  I can't see any type or anything on it.

Link to comment
1 minute ago, ashman70 said:

If you want to message me the serial number I can look it up. It's usually described as Perc something depending on the model. It's entirely possible it never came with one originally, it depends on what they  used it for and what drives they ordered with it.

Ah, so Perc H710 then I think

Link to comment
Just now, ashman70 said:

That would be it and unfortunately it can't do JBOD mode, so you'll have to replace it. I would look on eBay for a used H310, then there is a guide on here on how to flash it to IT mode and you will be golden.

From my brief bit of research on it, it sounded like if I do that, then unraid sees all of the HDDs as one single disk reported from the controller rather than individual disks?

 

I'd much rather have it see the individual disks for spinning up/down, SMART stats, etc.  Is there a non-dell controller that you can recommend that would work with it?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.