1st attempt at 60TB server

[Tim Timmens]

I am looking to build my first ever unRIAD server. I have over 700 blu-ray movies that will be stored plus other data. I will then look to use separate media device from the cinema room and living room to play back the media.

 

I unfortunately have very little (make that none) understanding of building a pc or server, so all help is welcome.

 

So far this is what I have decided:

 

What is your budget?

 

I want to make in under £750 - $1200 (this does not include the cost of HDDs as I will buy them as and when I need the space)

 

How many drives do you want your server to be able to support and how much capacity do you need?

Is expandability important to you?  If so, what's your long term goal?

 

Initially I will buy 3 - 5 HDDs and add from there. I am thinking to have up to 20 HDDs thus using 3TB HDDs would give me storage solution of 60TB.

 

Are you interested in running any unRAID Add Ons?

 

I have read of people using YAMJ for their media. Is this a good option?

 

Do you want to run green/low power drives or faster 7200 rpm drives?

 

I have not quiet decided if I will buy WD green 3TB drives or WD red 3TB drives. The red drives cost just over £10 more/drive but it does come with an additional years warranty.

 

Do you have any spare parts laying around that you would like to apply towards your build?

 

None, everything will be bought new.

 

So these are the parts that I have decided so far are suitable to the server build.

 

Intel Core i3 3240 Processor £87.50 - $140

 

http://www.amazon.co.uk/Intel-3-40GHz-Graphics-Advanced-Extensions/dp/B008F65MQ8/ref=sr_1_1?ie=UTF8&qid=1383483198&sr=8-1&keywords=intel+core+i3+3240

 

ASRock B75 Pro3-M Motherboard £47.14 - $75.42

 

http://www.amazon.co.uk/ASRock-B75-Motherboard-Instant-Charger/dp/B007RQ0HSA/ref=sr_1_1?ie=UTF8&qid=1383483451&sr=8-1&keywords=ASRock+B75+PRO3-M+LGA

 

Corsair TX650V2 Enthusiast Series 650W £68 - $108.8

 

http://www.amazon.co.uk/Corsair-TX650V2-Enthusiast-Series-Bronze/dp/B004O0P9TO/ref=sr_1_1?s=computers&ie=UTF8&qid=1383492939&sr=1-1&keywords=Corsair+psu+650w

 

Antec Full Tower Case Twelve Hundred V3 £156.10 - $249.76

 

http://www.amazon.co.uk/Antec-Twelve-Hundred-V3-Washable/dp/B004INH0FS/ref=sr_1_1?ie=UTF8&qid=1383494891&sr=8-1&keywords=antec+1200

 

G-Skill 8GB Ripjaws X DDR3 £63.98 - $102.37

 

http://www.amazon.co.uk/G-Skill-Ripjaws-DDR3-2133-Dual/dp/B0050119O0/ref=sr_1_2?ie=UTF8&qid=1383495586&sr=8-2&keywords=G+Skill+Ripjaws

 

WD Green 3TB £88.40 - $141.44

 

http://www.amazon.co.uk/Western-Digital-Caviar-Cache-Internal/dp/B004RORMF6/ref=sr_1_1?ie=UTF8&qid=1383494178&sr=8-1&keywords=WD+3tb

 

WD Red 3TB £103.62 - $165.79

 

http://www.amazon.co.uk/Western-Digital-Desktop-SATA-Drive/dp/B008JJLW4M/ref=sr_1_3?ie=UTF8&qid=1383495354&sr=8-3&keywords=WD+3tb

 

unRAID® Server Pro Registration key 2-Pack £93.13 - $149

 

I would get the USB version but customs in the UK means price will increase by 25%!!!!

 

Can I just stick this on my own USB?

 

What else do I need to complete this server? If you have any suggestions with the parts I have picked or things I am missing your input is greatly appreciated.

 

Thanks

 

 

 

[Ford Prefect]

...looks like you've done your homework well.

Here are my 2 cents:

 

CPU + mobo is a good choice for a desktop combo, being Socket 1155.

I don't know if newer Socket 1150 combos will be around cheaper in the UK.

 

You RAM selection is overkill (but it will run fine, I believe) as your CPU will only support 1333 and 1600 modules.

You could save some little money when going for DDR3-1600 CL9 modules.

 

..not sure about the case, what number of disk will fit without an additional cage.

 

Yes, you can use your own USB stick(s) but you'll have to register their GID with limetech in order to activate the license.

When unRAID is installed, the GID can be retrieved via the Web-UI.

There are *some* USB sticks that do not have/present a unique GID, hence these are a no-go.

I *think* some good models are listed on the limetech page.

[ijuarez]

when this done a picture is a must to see this beast in action

[Tim Timmens]

CPU + mobo is a good choice for a desktop combo, being Socket 1155.

I don't know if newer Socket 1150 combos will be around cheaper in the UK.

 

Would this be suitable (price difference is minimal to my original plan)

 

Intel Core i3 4130 £91.78 - $146.89

 

http://www.amazon.co.uk/Intel-Extended-Technology-Execute-Disable/dp/B00EF1G98W/ref=sr_1_4?ie=UTF8&qid=1383528470&sr=8-4&keywords=intel+core+i3

 

ASRock H87 PRO4 Motherboard £63.85 - $102.16

 

http://www.amazon.co.uk/ASRock-H87-PRO4-Motherboard-Generation/dp/B00CYWEZXA/ref=sr_1_1?ie=UTF8&qid=1383528547&sr=8-1&keywords=ASRock+1150

 

 

You RAM selection is overkill (but it will run fine, I believe) as your CPU will only support 1333 and 1600 modules.

You could save some little money when going for DDR3-1600 CL9 modules.

 

..not sure about the case, what number of disk will fit without an additional cage.

 

The RAM, I have looked at alternatives but price difference is minimal:

 

Corsair CMZ8GX3M2A1600C9 Vengeance 8GB £62.13 - $99.41 (for a price difference of $4, should I just not get the higher spec?)

 

http://www.amazon.co.uk/Corsair-CMZ8GX3M2A1600C9-Vengeance-Performance-Desktop/dp/B004CRSM4I/ref=sr_1_1?ie=UTF8&qid=1383529615&sr=8-1&keywords=8gb+ddr3+cl9

 

I forgot to add in my original post:

 

I will be buying four of these 5 in 3 hot-swap caddys £59.99 - $95.98 / each

 

http://www.xcase.co.uk/x-case-5-in-3-hotswap-caddy-v2-fits-in-3-x-5-25-bays-sata-iii-3-5-and-2-5.html

 

 

 

 

 

[garycase]

Would this be suitable (price difference is minimal to my original plan)

 

Intel Core i3 4130 £91.78 - $146.89

 

http://www.amazon.co.uk/Intel-Extended-Technology-Execute-Disable/dp/B00EF1G98W/ref=sr_1_4?ie=UTF8&qid=1383528470&sr=8-4&keywords=intel+core+i3

 

ASRock H87 PRO4 Motherboard £63.85 - $102.16

 

http://www.amazon.co.uk/ASRock-H87-PRO4-Motherboard-Generation/dp/B00CYWEZXA/ref=sr_1_1?ie=UTF8&qid=1383528547&sr=8-1&keywords=ASRock+1150

 

No.  The motherboard you originally selected (Socket 1155 with an Ivy Bridge CPU) was fine, as it had two PCIe x16 slots that you could use for add-in controllers when you need to expand your SATA ports.    But this board only has one x16 slot.    Since most add-in controller cards only provide 8 ports, if you truly plan to migrate towards 20 drives, you should use a board that can support 2 of these cards.

 

 

Also, a 650w power supply is marginal if you're going to move to 20 drives.  I'd get at least 750w, and it wouldn't hurt to go with 850.  I'd use either the Corsair HX series or the Seasonic X series units.    These are both good choices:

 

http://www.amazon.com/Corsair-Professional-Series-Modular-HX850/dp/B0090I9W66/ref=sr_1_2?ie=UTF8&qid=1383561427&sr=8-2&keywords=Corsair+HX

 

http://www.amazon.com/Seasonic-ATX12V-EPS12V-Power-Supply/dp/B004ETEFTK/ref=sr_1_1?ie=UTF8&qid=1383561685&sr=8-1&keywords=Seasonic+X

[garycase]

Do you want to run green/low power drives or faster 7200 rpm drives?

 

I have not quiet decided if I will buy WD green 3TB drives or WD red 3TB drives. The red drives cost just over £10 more/drive but it does come with an additional years warranty.

 

Definitely go with the lower power drives.    Use only NAS-quality units ... either the WD Reds or the Seagate NAS drives.    ... and buy 4TB drives => there's no reason to use 3TB units.    The cost/TB is very close ... depending on where you buy them it may even be lower for the 4TB units;  and you'll be using fewer drive slots for the same storage capacity, so your server can grow even larger.    e.g. a 60TB capacity requires 21 3TB drives, or 16 4TB drives.

 

[Fireball3]

As already mentioned by garycase.

 

You will need some controller cards to be able to connect that many drives!

Therefore you will need suitable PCIe slots on your board!

The corresponding forward breakout cables will also cost some money.

 

You also might want to go for a gold or even platinum certified PSU.

They have a much better energy efficiency on <20% load.

May be relevant if your build is running 24/7.

A 650W PSU will probably be good if you stay with "green" drives.

 

You are probably aware of the dilemma in PSU dimensioning.

There is a need for high amps when the drives spin up, and afterwards you

have just a marginal power consumption. But the efficiency of power supplies

is good only above 20%. Going with an 850W PSU means your build should

have a consumption of at least 170W (20%).

 

When do you plan to reach 20 drives?

You will probably need to replace the PSU in some years so starting with

a small one can be an option.

 

Edit:

If you plan to use those 5in3 backplanes you definitely want to use low power (green) drives.

If not, cooling of server grade drives (10W) at a moderate noise level will be an issue!

 

If your rig is going to be up 24/7 you should also think of ECC RAM instead of that high-end RAM.

Better go for "conservative" chips with decent timings.

[Tim Timmens]

No.  The motherboard you originally selected (Socket 1155 with an Ivy Bridge CPU) was fine, as it had two PCIe x16 slots that you could use for add-in controllers when you need to expand your SATA ports.    But this board only has one x16 slot.    Since most add-in controller cards only provide 8 ports, if you truly plan to migrate towards 20 drives, you should use a board that can support 2 of these cards.

 

I have looked at the alternatives, I think this mobo has the 2 PCIe x16 slots. (how do I know if it does have it, amazon does not mention it.. so I have been looking at pics of other boards and trying to work this out  )

 

ASRock H87M PRO4 Motherboard £71.08 - $113.73

 

http://www.amazon.co.uk/ASRock-H87M-PRO4-Motherboard-Generation/dp/B00CYWEYOK/ref=sr_1_4?ie=UTF8&qid=1383569533&sr=8-4&keywords=ASRock+1150

 

As for the power supply, I will get the 850w. Its probably overkill but its only £10 - $16 more than the 750w so why not!

 

Corsair HX850 Professional Series - £129.84 - $207.74

 

http://www.amazon.co.uk/Corsair-HX850-Professional-Series-Modular/dp/B008O0RTFC/ref=sr_1_1?s=computers&ie=UTF8&qid=1383572283&sr=1-1&keywords=corsair+psu+850w

 

[Fireball3]

http://www.asrock.com/mb/Intel/H87M%20Pro4/?cat=Specifications

- 1 x PCI Express 3.0 x16 slot (PCIE1: x16 mode)

- 1 x PCI Express 2.0 x16 slot (PCIE2: x4 mode)

- 2 x PCI slots

The x4 slot will limit the bandwith of x8 SAS-controllers but it should be sufficient for unRAID even with x4.

 

Unfortunately no ECC RAM support:

- Dual Channel DDR3 memory technology

- 4 x DDR3 DIMM slots

- Supports DDR3 1600/1333/1066 non-ECC, un-buffered memory

[garycase]

Yes, the AsRock ASRock H87M PRO4 has 2 PCIe x16 slots, so that will do nicely.

 

I completely agree with the PSU "dilemma" of running below the 80+ certification thresholds if you buy a supply with plenty of "spinup" headroom ... but the two units I mentioned are still very efficient down to as little as 5% load factors [they lose 5-10% efficiency at those very low loads, but as they start at the Gold certification levels, they're still 80% efficient even after that loss].    If you stick with NAS quality drives, however, a 750w unit will be plenty for a fully loaded Pro system (24 drives).

 

http://www.amazon.com/Corsair-Professional-Series-Modular-HX750/dp/B0090I9VZI/ref=sr_1_2?ie=UTF8&qid=1383573473&sr=8-2&keywords=Corsair+HX+750

 

http://www.amazon.com/Seasonic-Retail-Power-Supply-80Plus/dp/B00E0OT6L6/ref=sr_1_2?ie=UTF8&qid=1383573507&sr=8-2&keywords=Seasonic+X+750

[Tim Timmens]

Thanks fireball3 for your input.

I will begin to rip my blu-ray and other media collection immediately on to the server once built. Thus as and when I need to buy a new HDD I will. I expect to fully reach 20 HDD's in 3-6 months but it could be longer.

 

In terms of what you mentioned about "green" drives, correct me if I am wrong but the person who introduced me to unRAID said the server only spins up the HDD in use others are all "off". If that is the case surely cooling should not be a problem?

I don't plan to have the server on 24/7, just have it on as and when I need to access the media or when I am uploading media on the server (so initially it will be on for long hours but otherwise not so much). Should I still go for the ECC RAM?

 

Thanks

[Fireball3]

the server only spins up the HDD in use others are all "off". If that is the case surely cooling should not be a problem?

 

That is correct, but you need to do a regular (monthly) parity check. Then all drives will spin for a long time. (4TB ~ 42 8 hours).

 

I don't plan to have the server on 24/7

ECC shouldn't be necessary then.

[garycase]

You need to design for adequate cooling regardless of how many hours you plan to run your server.  You'll still want to do a monthly parity check -- and ALL drives will be spun up for that for many hours.  After about an hour or so, they'll hit the max temp they're going to with your particular setup, and stay their for the rest of the parity chek => if this is too high, you'll effectively be "baking" your drives 

 

Nothing to panic over -- the setup you've selected will provide very adequate cooling;  but you DO need to pay attention to what temps you hit during parity checks.

 

You don't have an option for ECC RAM unless you change the configuration you've specified  (no Haswell desktop boards support ECC memory).    But I wouldn't worry about it => just use high quality modules and do NOT install more than 2 modules on your motherboard.    With unbuffered RAM, your memory subsystem will be FAR more reliable with 2 modules than it is if you install more.

 

[garycase]

... need to do a regular (monthly) parity check. Then all drives will spin for a long time. (4TB ~ 42 hours).

 

A parity check does take a while, but nowhere NEAR 42 hours !!    About 8 hours is more likely 

[Tim Timmens]

BTW,

 

The price disparity of 3Tb and 4TB in the UK is a joke!

 

WD RED 3TB costs £103.62 - $165.79

 

WD RED 4TB costs £153.76 - $246.02

 

In the grand scheme of things, it'll mean I pay about 15% more getting 4TB drives (to a total of 60TB) but leaves me the option of further expansion.. ahhh options!!

[garycase]

Yes, that's 11% more per TB ... a penalty of $6.24/TB

 

But I'd still go with 4TB drives, for 2 reasons ...

 

(a)  $6.24/TB is really pretty small in the overall scheme of things ... and saving the extra drive slots (thus providing 33% more potential capacity for your server) is certainly worth something.

 

and

 

(b)  The prices are very likely  to come down somewhat after WD starts shipping their 5TB drives next month, so the /TB cost of the 3TB and 4TB units will likely be much closer.  [i.e. you won't pay that "penalty" for all of your drives]

 

[garycase]

... as a minimum, I'd at least buy a 4TB parity drive => even if you want to buy a few 3TB drives, it'd be nice to have the option of using 4TB drives when the prices come down without having to change your parity drive.

 

[Fireball3]

A parity check does take a while, but nowhere NEAR 42 hours !!    About 8 hours is more likely  [img alt=]http://lime-technology.com/forum/Smileys/default/smiley.gif[/img]

Yes gary, I mixed it up with the preclear duration  .

Nevertheless, 8 hours is also enough to heat them up if not cooled properly.

[Fireball3]

... as a minimum, I'd at least buy a 4TB parity drive

For parity it might be reasonable to go with a server grade drive. Gary?

[garycase]

... as a minimum, I'd at least buy a 4TB parity drive

For parity it might be reasonable to go with a server grade drive. Gary?

 

A NAS quality unit [WD Red or Seagate NAS] is fine.  An Enterprise class drive would provide the advantage of 7200rpm for somewhat faster parity operations when doing multiple simultaneous writes from different clients; but would have NO impact on parity check or drive rebuild speeds.    I'd simply stay with the NAS units -- not just for parity, but for ALL of the drives.

 

[henris]

No.  The motherboard you originally selected (Socket 1155 with an Ivy Bridge CPU) was fine, as it had two PCIe x16 slots that you could use for add-in controllers when you need to expand your SATA ports.    But this board only has one x16 slot.    Since most add-in controller cards only provide 8 ports, if you truly plan to migrate towards 20 drives, you should use a board that can support 2 of these cards.

If you are planning on ordering the 5-in-3 cages from X-Case, then you could also use this LSI SAS 9201-16i HBA card. Good quality SATA/SAS cards are tough to find in Europe and though pricey this is definitely a quality card. Since you have an option to use a motherboard with multiple PCIe slots, a cheaper alternative is to buy popular Supermicro 8 port card from Ebay/States, even with customs/VAT it will be cheaper than the LSI card.

[Fireball3]

In Europe it's easier to obtain the FSC D2607 or the Dell Perc H310 on ebay.

People often pull them from new servers and sell them on ebay.

I think they are also good quality.

Got mine for around 30-40€ each.

You can crossflash them with LSI software and have some nicely working HBA's.

 

16-port adapters are rarely sold on ebay and if you intend to buy them online

they cost a fortune.

 

[henris]

The LSI SAS 9201-16i costs 365eur in X-Case. I haven't been able to find reliable source (in Europe) for the re-furbished cards which could go well below 100eur/8-port card. So 200eur for two used cards which require re-flashing or 365€ for a brand new ready-to-use one. In a 20-drive system the drives alone cost ~3000eur so it is all very relative what you define as a fortune

[Fireball3]

You can see it that way of course.

But let's say 100€ for 2 new (or refurbished) 8-port controllers vs. 365€ leaves 265€.

Fortune or not - all things are relative.

That difference makes ~7 TB in hard drives to me.

 

Of course, ebay is no reliable source and if you need things ASAP you won't be

able to make a bargain. 

[Tim Timmens]

Hey Guys,

 

QUICK UPDATE

 

I will be picking up a Lian Li PC-P80B case with 4 x Xcase 5in3 bays + 1 Supermicro AOC-SASLP-MV8 PCI-EXPRESS X4 Low Profile SAS RAID Controller + 2 breakout cables.  I am buying this all for total of £265 - $426.65. Of course it's used but I think it's a good deal 

 

before I purchase the other bits I have quick few questions.

 

I am buying the following;

 

Intel Core i3 4130

 

ASRock H87 PRO4 Motherboard

 

G-Skill 8GB Ripjaws X DDR3

 

Will this all compatible with UnRAID? My biggest fear is I buy this then begin to find out that there are issues of hardware incompatibility with UnRAID!

 

Same also applies to WD RED 3TB or 4TB HDD, I have read something about them needing ears (WTF?), so some light on both fronts would be very much appreciated as I can then go ahead with my purchase.

 

Pics of the case and other items will be on when I collect it later this week