N54L with unRaid 5.0 and 4x4 TB, 1st impressions

[imrevo]

Good evening,

 

currently, I'm building my first unRaid-Server and I want to share my thoughts and experiences.

 

Hardware:

- N54L HP Microserver, 4GB RAM, No-HDD, with DVD-RW, Model #708245-425

- 4 x 4TB ST4000DM000

- Bios-Mod for full speed at all 6 Sata-Ports

 

Further additions, already bought but not installed yet:

- Samsung 840 EVO 120GB as cache disk

- another 4TB ST4000DM000 as manual "hotspare"

- Raidsonic ICY BOX  IB-172SK-Bas replacement for the DVD-RW drive

 

unRaid 5.0 is placed at a SanDisk 8GB stick plugged into the internal(!) motherboard usb slot of the N54L, no problems with booting or sharing. The bios settings have been changed according to the link above and, VERY IMPORTANT, disk caching has been enabled in the bios (only do that if you have some form of ups!). Changed the parity creation from 38MB/sec to 80MB/sec @ the test setup with 3 Samsung HD753LJ.

 

I already added unMenu, openssh and some other plugins, no big deal but worth mentioning as it was a simple click-and-click procedure and not what one might expect from a linux system :-)

 

I started the pre-clear-Script about 28 hours ago, for all four 4TB discs at once (4 telnet sessions) and here's the main reason for starting this thread:

 

- none of the 4 disks ever went above 35C

- the pre-read and the post-read speeds are ~140MB/sec per disc

- the N54L is still very silent (located ~50 cm to the right of my head at the moment)

 

The pre-clear script will be finished tomorrow and then I will do some benchmarking that will be published here, of course.

 

After 4xN40L with Windows 2012 and StarWind iSCSI as data pool for my two Dell servers running ESXi 5, this N54L is my first server running unRaid. The tests with the free license and my 3 Samsung HD753LJ were very successful and promising.

 

One question may be permitted:

Is there really no possibility to define a hotspare disk that jumps in automatically in case of a hard disk failure? Of course, this disk has to match the largest disk in the array, i.e. the parity disk. Any plugin, perhaps? We are talking about data, maybe valuable data and this data should be protected best effort.

 

cu

Volker

 

[neilt0]

You have made a wise choice, young Padawan. 

 

See my sig for a thread all about unRAID on Microservers and mods and all that. I'm currently running 6x4TB drives and am preclearing a 7th (a WD Red 4TB which started the preclear at 150MB/sec).

If the PSU will take it, I'll be running 7x 4TB drives on my N54L.

 

[imrevo]

Good evening,

 

short update: preclearing is done, now it's sync'ing parity at ~170MB/sec, see attached image.

 

Some hdparm benchmarks:

 

/dev/sdb:
Timing cached reads:   3236 MB in  2.00 seconds = 1618.20 MB/sec
Timing buffered disk reads: 454 MB in  3.01 seconds = 150.96 MB/sec

/dev/sdc:
Timing cached reads:   3198 MB in  2.00 seconds = 1598.90 MB/sec
Timing buffered disk reads: 464 MB in  3.01 seconds = 154.39 MB/sec

/dev/sdd:
Timing cached reads:   3188 MB in  2.00 seconds = 1594.39 MB/sec
Timing buffered disk reads: 468 MB in  3.01 seconds = 155.50 MB/sec

/dev/sde:
Timing cached reads:   3270 MB in  2.00 seconds = 1635.11 MB/sec
Timing buffered disk reads: 452 MB in  3.02 seconds = 149.88 MB/sec

 

more to come....

 

@neilt0: 7th drive? Where do you connect that one? 

 

cu

[neilt0]

If the PSU takes it, it'll be attached to a PCI-E HBA card. One of the drives will sit outside the case, attached using the e-SATA port.

[stourwalk]

Why put the 7th drive outside?  I've got 6 x 2Tb + 320Gb cache drive inside my N36L - is there less space in the N54L?

 

Just about to start the slow process of upgrading to 4Tb drives...

 

Chris

 

[neilt0]

You can't fit 7 3.5" drives inside the NxxL series unless you cut out some of the internal structure like so: http://forums.overclockers.com.au/showpost.php?p=15449310&postcount=9059

[stourwalk]

Ahh I see - I used a 2.5" drive for the cache so it fits in the space under the 2 x 3.5" easily

 

I nearly said 'full-height' drive then - then promptly remembered my first '5.25" full-height' harddrive.. ahh those were the days..

[tr0910]

Yes I had a few full height "machine guns" too. No silent drives those ones.  (Wistfully remembering the 60 megabyte Tommy gun)

 

Sent from my mobile

 

 

[neilt0]

Ahh I see - I used a 2.5" drive for the cache so it fits in the space under the 2 x 3.5" easily

 

I nearly said 'full-height' drive then - then promptly remembered my first '5.25" full-height' harddrive.. ahh those were the days..

You mean like this? http://lime-technology.com/forum/index.php?topic=11585.0 

 

I now use a 4TB drive for the cache. That might sound odd, but I write a lot of data to it as a temp drive -- well over 10TB a month and sometimes 500GB to 1TB a day.

[stourwalk]

Yes - just like that... ok I admit it - you gave me the inspiration for my setup

 

Do wonder whether I should have grabbed a N54L when they were cheap tho as a second setup.. oh well..

 

 

[neilt0]

Actually, I don't really need 2 servers now, but I do appreciate the extra CPU power in the N54L vs. the old N36L.

[imrevo]

As promised, find some benchmarks attached: without cache disk, with Samsung 750GB as cache disk, with Samsung SSD as cache disk.

 

I am very glad about these results and I'm a little bit surprised that it doesn't really matter if you use a ssd as cache disk or an old Samsung HD753LJ when writing to the raid. Both are fine for increasing the write speed.

 

N54L-Benchmarks.zip

[imrevo]

...and after applying that cpufreq-"fix", I wonder why so many people waste so much money on Synology, QNap and Buffalo :-)

 

The "fix" for the N54L (from some post in this forum) and maybe as well for the N36L / N40L is adding these lines to the go script:

 

echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold & logger Go Script - ondemand up_threshold set to 50
echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor & logger Go Script - ondemand sampling_down_factor set to 50

 

and of course you should execute these 2 lines once (or reboot, whatever you prefer).

 

That made the benchmark for "writing to the unRaid with cache disk" look like this:

 

Seq. Write Speed: 95.4 MByte/sec

Seq. Read Speed: 98.7 MByte/sec

 

Without the fix:

Seq. Write Speed: 54.3 MByte/sec

Seq. Read Speed: 97.3 MByte/sec

 

The original value for "up_threshold" was "95" on the N54L.... way too high, if you ask me (and the benchmarks :-) ).

 

cu

Volker

[aaronwt]

...and after applying that cpufreq-"fix", I wonder why so many people waste so much money on Synology, QNap and Buffalo :-)

 

The "fix" for the N54L (from some post in this forum) and maybe as well for the N36L / N40L is adding these lines to the go script:

 

echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold & logger Go Script - ondemand up_threshold set to 50
echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor & logger Go Script - ondemand sampling_down_factor set to 50

 

and of course you should execute these 2 lines once (or reboot, whatever you prefer).

 

That made the benchmark for "writing to the unRaid with cache disk" look like this:

 

Seq. Write Speed: 95.4 MByte/sec

Seq. Read Speed: 98.7 MByte/sec

 

Without the fix:

Seq. Write Speed: 54.3 MByte/sec

Seq. Read Speed: 97.3 MByte/sec

 

The original value for "up_threshold" was "95" on the N54L.... way too high, if you ask me (and the benchmarks :-) ).

 

cu

Volker

 

Where can I find out more info on what I need to do to add this script to my N54L?