Poor performance with my new Lime server


Recommended Posts

Hi

 

I got few weeks ago a server from Tom

MD-1510/LL + CPU & 4GB Memory upgrade

 

My internal network is 1000MBps and ping completes in <1ms

 

Still, read rate I get is 46 MB/sec, and writing is merely 10 MB/sec.

 

I have 4 x 1.5TB Western Digital disks for data + 1 x 1.5TB for parity

Each disk has 100 MB/sec read rate.

 

What is wrong with the system?

Why is it so slow?

 

Where can I see CPU, Memory, Network, Disk utilization as well as some other diagnostics on Lime server? Can I see it through Web interface?

 

It is quite an expensive system for home use, so I am a bit frustrated

 

 

-- Michael

 

Link to comment
  • Replies 83
  • Created
  • Last Reply

Top Posters In This Topic

Which WD 1.5TB disks? Are these the 5400RPM green drives?

I always recommend the fastest drive you can afford for the parity drive.

 

46MB/s over the network or local?

 

10MB/s write seems about correct for a parity protected drive.

If you were to add a cache drive this speed would be the equivalent of a single drive.

 

 

Link to comment

According to the wiki: http://lime-technology.com/wiki/index.php/FAQ#How_fast_is_unRAID.3F

 

Your performance is exactly as expected.  Quoting from the wiki:

Some general guidance as to what typical users can expect (assuming gigabit networking), read speeds *from* your server and writes to non parity-protected drives (such as the Cache drive) should typically be between 22MB/s and 35MB/s. Write speeds directly to parity-protected drives should be between 8MB/s and 15MB/s, with write speeds to User Shares a bit slower.

 

Writing to the array between 8MB/s and 15MB/s... you're in the middle with 10MB/s. 

 

Reading from the array between 22MB/s  and 35MB/s  you're better than average here with 46 MB/s... nice. 

 

Does not look like anything is wrong at all, in fact, you are doing better than average.  You cannot use the raw 100MB/s disk rate... they are not that fast.  That is a marketing "burst" rate, not a sustained data transfer rate and it is also without any operating system or networking overhead.

 

Enjoy your new array.  If you wish, post a syslog and somebody will let you know if they see anything out of the ordinary.  From the performance, it looks like it is doing well.

 

Joe L.

Link to comment

According to the wiki: http://lime-technology.com/wiki/index.php/FAQ#How_fast_is_unRAID.3F

 

Your performance is exactly as expected.  Quoting from the wiki:

Some general guidance as to what typical users can expect (assuming gigabit networking), read speeds *from* your server and writes to non parity-protected drives (such as the Cache drive) should typically be between 22MB/s and 35MB/s. Write speeds directly to parity-protected drives should be between 8MB/s and 15MB/s, with write speeds to User Shares a bit slower.

 

Writing to the array between 8MB/s and 15MB/s... you're in the middle with 10MB/s.   

 

Reading from the array between 22MB/s  and 35MB/s  you're better than average here with 46 MB/s... nice. 

 

Does not look like anything is wrong at all, in fact, you are doing better than average.   You cannot use the raw 100MB/s disk rate... they are not that fast.  That is a marketing "burst" rate, not a sustained data transfer rate and it is also without any operating system or networking overhead.

 

Enjoy your new array.  If you wish, post a syslog and somebody will let you know if they see anything out of the ordinary.  From the performance, it looks like it is doing well.

 

Joe L.

Your "Green" drives seem to be slower 5400 RPM drives...(actually, the marketing material does not even mention the speed other than to say it saves power.  They call it Intellipower since they do not want it known it is rotating slower than the competition)  since the platter MUST rotate at least once between reading the existing data and writing the new to generate parity, the max "write" speed of an array will always be dictated by physics and the rotational speed of the disk. 7200RPM drives will be faster than slower drives.

 

 

Link to comment

..this still does not justify the speed I get on read & write

 

10 MB/sec is awfully, and unexpectedly slow

 

Every appliance like QNap does much, much better

Qnap NAS is not parity protected.  and does not consolidate multiple disks as a single view of data.  Can't compare apples and oranges.

 

Nor can you compare unRAID with standard RAID5... they do not read then write the sector, but instead write in parallel to all the disks in their protected array.   They must use identical size disks in the array, or be limited by the smaller disk.   Again apples and oranges.   unRAID allows non-identical disks, and ease of expansion... and better protection against loss of all data if multiple disks fail...  unRAID was designed for the home media server market.

 

If you absolutely need raw speed, and protection of your data, unRAID was the wrong solution for you. You need hardware RAID5 instead.

 

Joe L.

Link to comment

No, I do not want RAID-5, I am well aware of the benefits of unRAID.

 

I read about cache disk.. It looks to me like a hole. I do want the data to be unprotected not for a fraction of time

I think you meant you do NOT want the data unprotected.

Is there anything else I can do to improve the disk writes?

Yes, Use true 7200 rpm disks.  No matter what you do, you are limited by rotational speed and basic physics. 5400 RPM disks will be slower than 7200 RPM disks...

 

Write directly to the disk shares, do not use user-shares... they add overhead.

 

Joe L.

Link to comment
Quote

Is there anything else I can do to improve the disk writes?

 

Yes, Use true 7200 rpm disks.  No matter what you do, you are limited by rotational speed and basic physics. 5400 RPM disks will be slower than 7200 RPM disks...

 

Write directly to the disk shares, do not use user-shares... they add overhead.

 

Joe L.

 

Here is what I do, set my media shares to be Export, read only under shares. I set my disk shares to be hidden read/write. When copying data to the disk shares I use \\server_name\diskx\media_name.

 

Where servername is unraid servername (tower, etc..), diskx is the disk number (disk1, disk2, etc...) and media_name is the type of media (movie, music, music_video, photo, etc...). 

 

Having the disk shares hidden means casual users dont know they exist. Media shares are read only so data isn't deleted by finger slippage on a front end device like a media player. I am the one who uploads media to the system, I have a windows share whether other people can dump media to be uploaded and I copy it across to unraid and update the home automation stuff so it can accessed easily (Cimemar online for anyone interested). 

 

Writing to a disk share is about 3x the speed of writing to a user share on my system. The downside is you need to do make sure the disk has enough space before copying data, wheras when writing to a user share unraid will do the math and decides where to place the data.

 

For me personally, I'd rather write the data faster and know where it is, so I can reference it from the home automation software.

Link to comment
Kaygee

 

what do you do if the share splits across few disks?

 

That is exactly what I have, each disk has an upper folder called "sharename". I use unmenu - check for open files. Choose a disk not in use and copy to the relevant folder.

 

Under each user share add each disk (disk1-7 or disk1, disk2, disk4 etc...).

 

Open \\tower\movies and the contents of movies folder from each configured disk is displayed.

 

Works a treat.

Link to comment

Sanity checked my figures:

4.3GB (4,587,734 bytes) file copy takes 3mins 5secs which is just over 24MB/s. Same file to a user share takes just under 7 mins, or around 10.5MB/s.

 

1GB RAM, Sempron 3500+ CPU.

 

 

 

Quite a difference! I've only ever written direct to the user shares, I think I'll try direct to a drive share and see what happens!

Link to comment

Sanity checked my figures:

4.3GB (4,587,734 bytes) file copy takes 3mins 5secs which is just over 24MB/s. Same file to a user share takes just under 7 mins, or around 10.5MB/s.

 

1GB RAM, Sempron 3500+ CPU.

 

 

 

Quite a difference! I've only ever written direct to the user shares, I think I'll try direct to a drive share and see what happens!

Writing to usershares is a lot easier, since you don't have to think about free space, but has serious performance hit. When used with a cache drive, this issue is solved since the files are transferred to the cache drive first, then during off hours to the parity drives. Unfortunately, due to a bug in the 4.5 beta the cache drive function performs the same or sometimes even worse than writing to a usershare without cache drive.

 

These are my findings:

 

Write:

Usershare without cachedrive: ±10 MB/s

Diskshare without cachedrive: ±15 MB/s

Usershare with cachedrive: ±7 MB/s

Diskshare with cachedrive: ±15 MB/s (cache drive is bypassed when writing directly to diskshare, so perfomance is the same)

Writing directly to cachedrive: ± 35 MB/s

 

Read:

Diskshare: ± 75 MB/s

Cache disk: ± 75 MB/s

Usershare: ± 40 MB/s

 

This is on Gigabit with Jumbo frames (MTU: 9000) enabled.

 

Link to comment

I have more powerful hardware (LIME system with all available upgrades), but the speed rate I receive is exactly the same if I am writing direct to the disk or through share..

 

What I do is writing 500MB file once to \\server\disk1\share and 2nd time to \\server\share

 

in both cases it is somewhere between 10MB/sec and 12MB/sec..

 

Something wrong.. Read rate is quite good, though. unRAID OS is (I assume) the latest one

Link to comment

I have more powerful hardware (LIME system with all available upgrades), but the speed rate I receive is exactly the same if I am writing direct to the disk or through share..

 

What I do is writing 500MB file once to \\server\disk1\share and 2nd time to \\server\share

 

in both cases it is somewhere between 10MB/sec and 12MB/sec..

 

Something wrong.. Read rate is quite good, though. unRAID OS is (I assume) the latest one

Have you tried NOT using jumbo frames?

 

I know it sounds strange, but worth a try.  Too big a packet size is just as bad as too small.  If fragmentation occurs, then you will just end up with the network having to resend.

 

Here is one post with the same type of performance slowdown with jumbo frames:

http://forum.qnap.com/viewtopic.php?f=23&t=263&p=1963

 

For kicks, please post the output of the following command

ifconfig

Link to comment

 

VAULT login: root

Linux 2.6.27.7-unRAID.

root@VAULT:~# ifconfig

eth0      Link encap:Ethernet  HWaddr 00:30:48:b0:ad:08

          inet addr:192.168.3.102  Bcast:192.168.3.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:3287130583 errors:0 dropped:0 overruns:0 frame:0

          TX packets:1123489433 errors:0 dropped:46 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:3299461697 (3.0 GiB)  TX bytes:2256782712 (2.1 GiB)

          Interrupt:219 Base address:0xa000

 

lo        Link encap:Local Loopback

          inet addr:127.0.0.1  Mask:255.0.0.0

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:31 errors:0 dropped:0 overruns:0 frame:0

          TX packets:31 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:6432 (6.2 KiB)  TX bytes:6432 (6.2 KiB)

 

root@VAULT:~#

 

 

 

 

BTW, why the directory is so slow? I had 4GB installed by Tom, shall not all the files to reside in RAM? How can I speed up opening the directory? S-L-O-W!!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.