Jump to content
Johnm

Goliath: My Norco RPC-4224 build.

67 posts in this topic Last Reply

Recommended Posts

Goliath,

 

Formerly My WHSv1 Media server named Leviathan.

The WHS server was a test of WHS's Drive Extender. Previously I was hosting my media off of a windows 2008r2 box with 16 2TB Samsung F4's in raid 6.

I wanted a greener alternative to having to spin 16 drives 24x7 just to watch a movie or access a single file. in my house there is almost always someone on the media server.

Besides greener, I wanted simple, set and forget. I work in an enterprise high stress server environment. The last thing I want to do when I come home is work on my own network.

 

WHS was my first step to testing that. It was not a total failure, it did what it was supposed to do. It served up my files and was a no brainer, anyone in the house could use it.

I just did not like the way it handled file duplication and god awful network performance. Copying files to and from it made me want to pull my hair out it was so slow.

 

My next step was to be WHS2011 and Flexraid. I thought I'd get the performance boost of 2K8 and the pooling of Flexraid. Well the flexraid site was down for over a week at time of building. forget that.

 

I went to step 3 bypassing step 2. unRAID.

after weeks of trolling the forums and playing with the free version, I feel am ready to buy my key and go into pre-production. I recycled most of my WHSv1 box. Moving it to a norco RPC-4224 The downside to this is I am limited in the number of Hard Drives I can use on this motherboard.

I know it will be short term before the next upgrade. This gives me time to decide if unraid is truly my savior and I buy a motherboard  for unraid specs or i need to look down another road with a different hardware option.

 

Right now, I feel pretty positive about this step, enough so that ill be putting my data on it. I really wish there was some option, say a 30day trail of one of the Purchase versions of unraid so I can get a full hands on experience of it can offer.

But, I understand the route they took. I'm just used to the enterprise. you don't buy it unless you're satisfied.

 

My network consists of several 2008r2 boxes, several with raid 5 arrays. An ESXi box, 2 WHSv1 boxes for PC backups and office type files and photo backups, a WHS2011 box for additional PC Backups and remote access.

 

 

Goliath

 

OS at time of building:

5.0-beta6a Pro License (I live on the edge.)

5B14 now.

 

CPU:

Intel Atom D510. Built into motherboard.

Intel Core i3-2100 Sandy Bridge 3.1GHz 2 x 256KB L2 Cache 3MB L3 Cache LGA 1155 65W Dual-Core  $100 at microcenter

 

Motherboard:

SUPERMICRO MBD-X7SPA-HF-O Mini ITX Intel Atom D510 processor Server Motherboard

SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard $108 open box from New egg (motherboard was still factory sealed, only the manual missing, go figure)

 

RAM:

4 Gigs of used DDR2 Samsung memory from trash at work (pulled from Thinkpad or Mac?). 2x2Gig sticks, motherboard Max

Kingston 2x 8GB (2 x 4GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 (PC3 10600) Server Memory Model KVR1333D3E9SK2/8G

8Gig Board max is 32Gig

16Gib (Added 8 more that i pulled out of Atlas when it was upgraded. This is overkill)

 

Case:

Norco RPC-4224

 

Drive Cage(s):

24 built in.

 

Power Supply:

CORSAIR CMPSU-750TX 750W ATX12V v2.2

SeaSonic X750 Gold 750W ATX12V V2.3/EPS 12V V2.91 SLI Ready 80 PLUS GOLD Certified Modular Active PFC Power Supply Sale: $119.99 W/ Free shipping.

 

Uninterruptible Power Supply (UPS):

APC 1300VA Power Saving Green Back-UPS XS - with LCD Display $109.99 w/ free shipping

 

 

SATA Expansion Card(s):

2 X SUPERMICRO AOC-SASLP-MV8

1 x IBM M1015 (LSI 9220-8i) Flashed with IT firmware $75 off EbayRemoved from server but is a good replacement for the SASLP-MV8

Cables:

4 x NORCO C-SFF8087-D SFF-8087 to SFF-8087 Internal Multilane SAS Cable Im only using 2 for now.

2 x NORCO C-SFF8087-4S Discrete to SFF-8087 (Reverse breakout) Cable

2 x NORCO C-P1T7 4Pin Molex 1 Male to 7 Female Power Extension Splitter

2 x Monoprice SFF-8087 1M cable (the norco one was to short for the M1015)

 

Fans:

6 x ENERMAX UC-8EB 80mm Case Fan

(These are not fully tested for adequate cooling.) (these proved a failure in the long run. drives got a bit toasty)

Running stock fans in mid fan plate. (new 120's on order.)

 

Server is almost totally Silent, I can't tell if it is on or off

 

3x120mm Fan bracket ($20ish shipped)

3x "pressure optimized" Noctua NF-P12-1300 120mm fans I picked up for $15 each plus $5 shipping for all 3.

2x ARCTIC COOLING ACF8 Pro Pro PWM 80mm Case Fans back on the rear.

 

 

 

Parity Drive:

Still testing, 2TB Samsung F4 for now.

It is the coolest and second fastest green drive I have.

My Seagate 5900's are faster but run a lot hotter.

since the Parity drive is almost always running,

I wanted to try cooler over heater.

Hitachi 3TB green Drive 3TB moved to ESXi

Seagate LP 2TB

 

Data Drives:

7 Hitachi 3TB green Drives moved to ESXi box

 

11x 2TB drives Mix of WD EARS, Seagate 5900's, Hitachi's and Samsung F4's (All LP drives)

5x 1.5TB Samsung F2's (LP

 

 

upgrading those to 2TB (or 3TB if possible at that time) when I need more space.

 

 

Cache Drive:

WD 500GB 7200 RPM.

 

Preclear Bays:

Two (unless they up the drive limit)

 

Total Drive Capacity:

29.5TB Usable

(it went down in size. I do have more drives i can put into this box sitting in storage)

 

Primary Use:

Media Hosting for 4 XBMC boxes.

Xbox 360 ISO storage.

Backup of my PC Backups (copy my WHS back ups here)

Backups of Ghost images from ghostcast server

Long term storage of older seldom used files.

Possible Back up of raid arrays.

(I might build a second unraid for the last 3 plus backup of media from this server)

 

Likes:

Low Power!

Lots of storage!

Parity!

Single drive pool

Server is almost totally Silent, I can't tell if it is on or off.

 

 

Dislikes:

Expensive!

Big

burn in takes forever

I need 2

 

Not sure yet. I'll keep you posted.

 

Add Ons Used:

unMenu

Install Screen

Mail and SSMTP

SSH

apcupsd

Clean Powerdown

unRAID Status Alert

Monthly Parity Check

unRAID Power-Down on disk overtemp

Screen

 

 

Future Plans:

Upgrading to a SUPERMICRO MBD-X9SCM-F-O Sandybridge with I3 Sandybridge or Xeon (if i run VM's) CPU

If that board will not support unRAID, then the X8 with a I3

I am not in a rush so I'll shop the sales.

Possibly An SSD for Cache drive.

Depends on sales and 7200 RPM drive performance.

 

Boot (peak):

Idle (avg):

Active (avg):

Light use (avg):

 

 

 

Rolcu.png

 

New Configuration:

zNRwm.jpg

 

13ocG.jpg

 

I should have taken the pictures AFTER I blew the case out with compressed air.. not Before... i definitely am pulling gunk into the server through the drive bays.

 

The fan just sitting on my MV8's.

It has not moved yet. it has dropped the temps way down.

If yours vibrates or moves, you could zip tie it to the cards.

Yz0jb.png

 

My foam insulation and new fans

sSgxX.png

 

gGRtY.png

 

uGelA.png

 

 

 

Old Configuration:

1Hg0c.png

 

 

74AjD.png

 

 

Copy Speeds with Current setup:

KevHi.png

 

 

 

This will be a work in progress. I am sure I'll be updating this post as I change things about.

 

 

Share this post


Link to post

Nice. I love that case.

 

 

Big upgrade from WHS. I probably would have never even found Unraid if M$ had decided to keep drive extender in WHS 2011.

Share this post


Link to post

Ditto.

 

I am mid preclears with new drives and the new fans. I am doing 5 2TB EARS and the seagate 1.5TB 7200RPM drive at once. I was weary of running more then 6 at a time.

 

The WD are all at 38C-41C, the Seagate is at 44C. I know that drive runs hot and is rated up to 60C operational.

 

I am not sure if i want to put the loud factory fans back in or stick with totally silent. Those are not dangerous temps, just higher then I want. I guess a full system parity check will tell.

 

I think I'll start looking at silent 120's just in case. I'm now spoiled by a server so quiet i can put it in the living room and forget it is there, until I trip over it.

 

LoL that motherboard looks so small in there.

Share this post


Link to post

The WD EARS are at 38-41 Celsius?  What is your room temperature?

 

I'm unsure what the CFM on the stock fan wall fans are, but, 24 CFM may very well been a reduction in airflow.  I believe I read someone stating that the stock ones run 36 CFM..  12 CFM * 4, 48 CFM loss.  Combine that with the severe reduction in airflow replacing the dual stock rear exhaust.  Yes, I know they're loud as all get up.. but..

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16835226005

 

Just as quiet, but 8 CFM more per fan than those Enermax's. 

Share this post


Link to post

Thats one reason why I think I'll go with the 120's. get more CFM for similar DBa.

 

That room is 23C. it is in the livingroom for testing. It will end up in the basement later.so I might put the factory mid coolers back in.

 

If I do buy more 80's, I'll try and get 4 pin connectors so I can use the quick change mounts.

 

Edit.

I just went downstairs to look at that physical server and removed the napping cat from in front of the server and the temps all dropped a degree or two in the first 10 min.  The server is on the spare couch. I'm not going to move it now.

 

the downside of IPMI. you cant see the server, and the cat.

Share this post


Link to post

The WD are all at 38C-41C, the Seagate is at 44C. I know that drive runs hot and is rated up to 60C operational.

 

I believe that Seagate will trigger the SMART temperature attribute if the temperature is above 55C. When the preclear is done you can post your SMART status and we will see if they are rated at 60C.

 

Share this post


Link to post

Nice. I love that case.

 

 

Big upgrade from WHS. I probably would have never even found Unraid if M$ had decided to keep drive extender in WHS 2011.

my WHS is stll my primary storage for my media. At least my BD ISOs. Since it can still do a lot of things I can't with unRAID. But I am using my unRAID Storage for a lot of my other file storage now.

 

Sent from my HTC Incredible using Tapatalk

Share this post


Link to post

To me the OS that you use is like a tool in the toolbox, you use the tool that works best for the job.

 

I still have WHS boxes, I wont get rid of them. So far unRAID fills a gap i need.

 

I'm just a geek with OCD in general...

Share this post


Link to post

The temps dropped down overnight durring the rest of the writing of zeros and the second read.

 

the Seagate dropped down to 41 and the EARS to 34-38 range.

 

I didnt change a thing, I'll just assume the room temp and sun effected the temps for now.

I'll move the server to its home and run the next batch of preclears tonight or tomorrow.

Share this post


Link to post

Well, I found a bargan on the SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 and ordered it for just over $100.

 

I don't see any indications that anyone has tested the Intel C204 chipset or the Sata3 connectors that come with the chipset. I guess I'll Guinea pig it. If it fails for unraid, my 2008r2 or ESXi servers would love it.

 

I also picked up 2 x 4Gig of Kingston Value Ram EEC Memory sticks. overkill for Unraid, but it was cheap.

 

I have an I3 2100 that i can put on it, if I repourpose the mobo, I'll toss a Xeon on it.

 

A Sata3 SSD for a cache drive on the Sata3 port sounds like over kill, but sounds like a fun test. even a mid range one should exceed the Gig network.

 

 

EDIT:

I see the Intel 82579LM  might be an issue. the second NIC should work fine.

Share this post


Link to post

Well, I found a bargan on the SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 and ordered it for just over $100.

 

Ok, Board and ram both came today. I ripped out the Atom motherboard. put the new X9SCM. Pulled the I3 I had in my trunk that I was going to return to microcenter (I wanted the Xeon for the ECC).  and fired it up.

 

As expected, Unraid didnt like the first NIC, the second one worked fine. it worked better then fine.

I am now getting 90MB/s transfer rates from my 2008 servers to my unraid. slightly better from my desktops SSD.

 

I also added an APC BX1300G UPS.

 

I updated my build list to reflect the changes.

 

I also have my factory mid fans back in. i was not sure about the  enermax's

 

I'll post some pictures when I find my iPhone.

 

 

Edit:

I already noticed this  setup sucks more power then the ATOM.

 

Using the UPS  as a kill-o-watt (I know not efficient)

 

Idol, drives spun down:

Old board: 36watts

new board: 48watts

 

Parity check:

old board: mid 80's i never wrote it down.

new board: 98watts

 

Changing my fans might have added a watt or two also.

Share this post


Link to post

My 2 "SeaSonic X750 Gold 750W ATX12V V2.3/EPS 12V V2.91 SLI Ready 80 PLUS GOLD Certified Modular Active PFC Power Supply" That was on sale at Newegg for $119 W/ free shipping showed up today.

 

OMG! overkill on the packaging. I got this HUGE Box from newegg?!?! I don't remember ordering a new Case/Chassis..

I open it up and it was the 2 SeaSonic's I ordered. Nice packaging!

Unlike my Corsair's I get Two Velvet bags and a Two pouch nylon Bag with Velcro closures full of  Cables.

I think half of the cost went to the packaging.

 

I really didn't have a clue what i was going to do with this Power Supply. I just knew I would need one soon for my second Norco.  at that price, I'll get two.

 

I pulled the  Corsair TX750 out and tossed one of the SeaSonic's in. I decided that while the Corsair was rated the same, the SeaSonic is "Gold Certified" and the Corsair was only "Plus Certified".

 

For how much this beast draws and being on 24x7. This should give me cleaner, more efficient power and save a few bucks on my electric bill.

 

It is almost a shame to use a modular Power Supply in a 4224 when there is plenty of room for the extra cable mess.

 

PS, the velvet bag comment was sarcasm, seriously? do we need these things? I'd like to pay less money on a Power Supply, not more for packaging!

Share this post


Link to post

OK,

 

I tested an SSD today for a cache drive.. SEE HERE.

 

It was not as I planned. My 7200RPM 1.5TB seagate POS spinner out performed it. something is seriously wrong here. and thats is a quick SSD for its age.

Edit 10/20:[After further testing... I Can Saturate the SSD by copying from a SATA3 SSD on my desktop to unraid Cache SSD. In addition, the SSD could saturate the gigabit NIC at 99%+ when copying from 2 sources at once (SSD or Mechanical). The 7200RPM mechanical cache drive would have a heart attack doing that. I could also copy at full speed to the cache drive while the mover is running. the SSD will stay in the system!]

 

Also my temps are way down now.. the stock fans are good for something besides noise..

 

I am not sure I can tweak this any better?

Maybe jumbo packets?Old photo with Mechanical Drive.

qn8w9l.jpg

 

Share this post


Link to post

Posting my syslog for the last 24 hours.

 

I rebooted yesterday, ran a parity check.

I came home today and rebooted.

I don't see any alarms going off in here but maybe the trained eye can catch somthing i missed?

I am pretty sure this board has not been marked as Level 1 yet.

 

 

Edit: if someone can tell me what the non-existant user errors are?

syslog-20110526-184205.zip

Share this post


Link to post

Here's what I saw in your syslog:

 

Drive ST31500341AS (1.5 TB Seagate)

ID# ATTRIBUTE_NAME          FLAG    VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE

184 End-to-End_Error        0x0032  100      100      099      Old_age  Always      -                        0

 

I don't know if the above is an issue or not.  None of your other drives even list that SMART parameter.  I'm guess it is fine, but I still think someone with more experience should look at it, since the value is so close to the threshold.

 

Drive ST32000542AS (2 TB Seagae LP) is running firmware CC34.  This is a bad idea, it will cause the drive to fail prematurely (I had one that failed after just a few months of regular use).  I recommend you upgrade the drive to firmware CC35.  The process is non-destructive, it won't erase your data.  Details here.

 

All your other drives look healthy.

 

That's all I could recognize.  There's a lot of stuff in your syslog that is over my head.  Hopefully someone more experience can take a look at it.

Share this post


Link to post

Hey thanks for taking time out to look at it.

 

 

I am aware of the firmware on the seagate. I had left it at the older firmware intentionally.

Most of the rest of them in my possession are updated.

All of my chinese ones are updated and my japan ones are not (Or whatever the 2 countries were that made them)

I really hate that model.

 

I am wondering if it is ok for level 1 cert.

Share this post


Link to post

It looks Level 1 worthy to me, but I would still prefer another more experienced person taking a look at it.

Share this post


Link to post

It is attributed to running ECC RAM, that 184 that is.. basically the parity between the RAM and HD mismatched.  It has a raw data value of 0 (lower is better for this SMART id [ie. 184]).

 

Share this post


Link to post

I did the next upgrade..  ;D

 

I installed the 3x120mm Fan bracket ($20ish shipped) and three "pressure optimized" Noctua NF-P12-1300 120mm fans I picked up for $15 each plus $5 shipping. I also put my 2 ARCTIC COOLING ACF8 Pro Pro PWM 80mm Case Fans back on the rear.

 

Huge difference!

 

I should have done this from day one, I was just being cheap at the time.

 

It is not dead silent. it has a slight wind noise to it. But I can definitely live with it.

It is almost silent if you're comparing it to the stock mid wall fans.

It is about the same noise level as my main 3d rendering desktop.

 

 

The real factor is hdd temps. i tried a few parrity checks with 3 simultaneous pre-clears and nothing got over 37. most drives were in the mid 30's. this is about the same or 1-2 deg higher then the stock mid wall fans.

 

I can feel the suction on the front of the drive bays. This was money well spent IMO.

 

I am still using the Noctua's at full speed (1300ish RPM). I might try the L.N.A adapter provided with the fans and drop them to 1100ish RPM and check my temps again. I don't think I can get much quieter and maintain these temps.

 

I also noticed that i had a fan wire stuck in my CPU fan from the last time i moved the case. It has been running for a week or two with that jammed up stopping the fan from spinning. CPU core temps have been fine without it. I am just surprised the server didn't freak out and shut down on me for the fan being "seized".

 

EDIT:

Oh, I forgot to mention;

I plugged the open wire holes in the 120mm bracket with insulation foam.

 

and...

I added another 4 1.5TB drives I just got back from warranty repair and I have 3 New 2TB drives running preclears. This should bring me to 22 usable terabytes.

 

current drive config is:

Parity drive:

1x Seagate 5900 2TB

Cache Drive:

1x Seagate 7200 1.5TB

Data Drives:

5x Western Digital EARS 2TB

4x Samsung 5400 F3 1.5TB

3x Samsung 5400 F4 2TB

 

This maxes out my 14 drives on the Motherboard pluse One MV8.

I'll have to install the second MV8 or the 2 port card for the next drive I add to the server.

I might still have to pull my old WD 2TB green drive. I am getting parrity errors in Beta7. I think that drive is the source of the issues? (bringing me down to 20TB if that is the case)

Share this post


Link to post

Hi Johnm,

Glad to see you went ahead with the Noctuas, and they didn't disappoint. THe L.N.A adaptors are a nice option to drop the speeds (and noise) with a fixed voltage. Although my first impression is that they should be ok for your intended level of control, I'd suggest you try something like 3 Zalman Fanmate 2 individual fan controllers (one per 120mm).

 

That's what I did with my gaming rig once I went RAID 5 with it. with this, it became flexible enough in maintaing case temps. If I go no A/C where this pc is located, I can easily adjust the fan speed as needed without taking anythig apart. I have the little controllers taped with double-sided tape to the back of my case-hidden.

 

Maybe that's an option for you.

Share this post


Link to post

I am running them full speed and I have to look to see if my server is on unless i am right next to it.

My drives all stay around 31-34c during parity checks.

 

I could not be happier right now with my server.

OK, i would love it totally filled with 3TB drives... But i have a good start. 8 now for 21TB.

 

Right now it is just rock solid.. hardware wise at least.

 

One of these days I'll have to post the specs on Spartacus, my second unraid.

 

Share this post


Link to post

Johnm,

 

Did you have any issue's with the Supermicro board and running memtest?  Memtest seems to hang before it can even start one test, though this is a know issue with some mobo's and legacy USB support.  However, if I disable legacy USB support, my flash drive stops working.  Makes it hard to test if that's the problem :P

 

Thanks in advance.

Share this post


Link to post

Johnm,

 

Did you have any issue's with the Supermicro board and running memtest?  Memtest seems to hang before it can even start one test, though this is a know issue with some mobo's and legacy USB support.  However, if I disable legacy USB support, my flash drive stops working.  Makes it hard to test if that's the problem :P

 

Thanks in advance.

 

Yes.

The memtest with unraid is a bit outdated.

It does not know the newer memory controllers like those with the sandybridge chipset.

I am not sure if it has been upgraded in the latest beta (5b10).

I upgraded manually to ver 4.20 and it works with no problems. (there might even be newer by now?)

 

You can get upgrade instructions in this thread http://lime-technology.com/forum/index.php?topic=2538.0

Share this post


Link to post

What is the length of the cables that you have used.  I can get some cables that are 30cm long, but I am concerned that they may be too short.  Others are 60cm and 1m.  I want to try an aoid excessive cable in the case that would prevent air flow.

 

Mick

Share this post


Link to post

What is the length of the cables that you have used.  I can get some cables that are 30cm long, but I am concerned that they may be too short.  Others are 60cm and 1m.  I want to try an aoid excessive cable in the case that would prevent air flow.

 

Mick

 

This is from the Newegg page:

 

Norco Technologies Inc. C-SFF8087-D

Model

 

Brand

    NORCO

 

Model

    C-SFF8087-D

 

Type

 

Type

    Cable

 

Features

 

Features

    Cable Length: 1.64ft / 500mm.

    RoHS Compliant

    Connector on First End: 1 x 36-pin - SFF-8087.

    Connector on Second End: 1 x 36-pin - SFF-8087

    Use for server case or enclosure with MiniSAS SFF-8087 Backplane, such as RPC-4220 storage case (RPC-3216, RPC-2208, RPC-1204......)

 

Manufacturer Warranty

 

Parts

    1 year limited

 

Labor

    2 years limited

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.