[SOLVED] Need help solving write speed issue

[somail]

My rig has displayed write speeds from 8-15MB/s for the last 3 years.  I have always felt it should be faster but have never had time to troubleshoot.  Below are my speeds, and these are consistent across every Unraid version.

 

Parity Sync:  ~70MB/s Can drop to high 50's during late stage of sync

Read Speed: ~50MB/s but can get up to high 60s depending on drive being read from.

Write Speed (NO PARITY): Anywhere from 35-45 MB/s

Write Speed (Parity): 8-15MB/s sometimes will drop down to low single digits.

 

Current specs:

 

5.0-rc16c

Motherboard: ASUS M3N78-VM

Processor: Athlon 64 LE-1600 (2.20 Ghz, single core)

RAM: 2 x 1GB DDR2 800

 

7 HDD's:

 

On Motherboard SATA:

2TB WD EZRX  <------Parity Drive

1.5TB WD EZRX

1.0TB WD EARX

1.5TB Seagate (The crappy old ones that clicked, but this particular one does not seem to)

500GB WD AAKS

 

On PCIE x1 generic rosewell SATA card:

640GB Samsung HD642JJ

500GB WD AAKS

 

I originally thought my slow parity write was due to a 1.5TB seagate being used as a parity drive.  I swapped it out with a new 2TB WD EZRX and write speeds became more consistent but I am still stuck at an average ceiling of about 13MB/s.

 

I am attaching a syslog for a 25GB single file transfer with parity on.  Write speeds started out at 4MB/s and then over a few minutes ramped up to a max of 15MB/s which is probably the fastest I have ever seen.  The write was being done to the 1.0TB WD EARX.

 

Any ideas on where my weak link is? Do you think the average 13MB/s write speed is normal for this type of setup?

 

Thanks for the help

syslog.txt

[mr-hexen]

That's a pretty weak processor is it not? 

[somail]

Sure is.  But I did not think unraid was 'that" taxing.  It seems even the most basic setups here get faster than 15MB/s.  But then again I have never paid attention to people's processor speeds.

 

I guess I could try to O/C the cpu a little bit and see if speeds improve.

 

[mcs]

It's not likely to be a CPU issue.

 

I'm also running a basic system:

MB:        Asus M4A78-VM

CPU:      AMD SEMPRON LE-1200 CPU, 2.1GHz (45W)

PCI SATA:  Adaptec 1430SA - PCI E x4 (installed in M4A78-VM 16x "graphics" slot)

 

And getting, similar results that you are reporting except for parity protected write speed:

Parity Sync:  ~70MB/s

Read Speed: 50MB/s+

Write Speed (WITH PARITY): 30-50 MB/s

 

Check your BIOS settings, make sure it has disks set to AHCI if it is available. I originally had mine set to SATA and suffered a 20%+ performance hit.

 

[dgaschk]

The syslog does not appear to include anything after the array is started. Please attach a longer syslog that includes a slow transfer event.

[somail]

Hi guys thanks for responding.

 

@MCS - I checked and I have AHCI set. That is good to hear on the CPU.  I would be very excited to get your speeds.

 

@dgaschk - you may need to help me get a proper log. I have used the instructions on your sig both times to pull the log from the web "//tower/log/syslog".  Both times I get a log which stops after the array is started. See attached for a new log just in case I am wrong. 

 

EDIT: to clarify, both logs I have posted should show both a single file read and a single file written to the array.

 

Do I need to change the type of logging?

 

Thanks for the help

syslog.txt

[dgaschk]

The logging is ok but the log is too short. What is the last time stamp in the current log.  When was the first?

[mcs]

Another couple of thoughts:

 

The (30-50 MB/s) with parity write figures I provided below are for writting to disk shares, as I don't use user shares.

I believe user shares can cause a performance hit, especially on low end CPUs.

Are you writting to disk shares or user shares?

 

There are also a number of oldish (500GB) disks on your system which may have slower write speeds.

 

[garycase]

Looked this over and will add a couple thoughts ...

 

=>  Your CPU is indeed low-end, but is on a par with a typical Atom, which can easily support FAR better write speeds than you're seeing.

 

=>  With your mix of older and various size drives, the parity check speeds seem fine.

 

=>  Your low write results are consistent with a faulty pair in an Ethernet cable, which will still allow transfers, but at much lower speeds.    The fact that it's very variable may mean it's an intermittent fault.  In any event, I'd definitely try a different Ethernet cable between the UnRAID box and your router.

 

=>  To help eliminate the add-in SATA card, do a read and write test to the 1.5TB EZRX drive --  write a couple fairly large (GB+) files to it;  then copy them from the disk to your PC.    Do this directly to the disk share;  NOT through a user share.

 

[somail]

@mcs - You are correct, I am using "user shares"

 

@dgaschk - I am not sure about your question.  The logs I am attaching are the ones I get from the web browser  or by doing cp /var/log/syslog /boot/syslog2.txt ...... both result in the same log.  Do the logs normally show file transfers? I am not used to reading unraid logs, but as you said I do not see my transfers. These logs should show several (if transfers are logged). 

 

@garycase 

 

I swapped to a newer Cat6 cable and did some transfers to multiple disks.  Some speed differences, but all within a tight margin. Also, on the wiring side I have used multiple Cat5e pairs, routers, and switches with this server over the years and average write has usually been 12-15MB/s.

 

I transferred and copied back 2 x 3.11GB files using three disk shares

 

WD 1.5TB EZRX (This is on the SATA expansion card, my initial post was incorrect)

Write: Average 20MB/s but it can get down to 14MB/s

Read: High 50's

 

Seagate 1.5TB (On Motherboard port)

Write: Average 15MB/s

Read: Low 60's

 

WD 500GB AAKS (On Motherboard port)

Write: Started Mid 20's but ended in the high teens (The winner!)

Read: High 50's

 

After all this I also performed a user share write which resulted in 15MB/s (just to confirm my past readings).  This write was done to the 1TB WD EARX (Motherboard port), which was not tested above. I then wrote to this drive directly "drive share" and got about 17MB/s.  So I guess the overhead for "user shares" on my system is about 2-3 MB/s.

 

I am attaching a new syslog for this period.

 

I also performed a Top command on the system.  Notable was 25%sy , 73%wa .  The high wa seems correct since it's a file transfer.

 

Thanks for the continued help, and let me know what else I can provide.

syslog2.txt

[garycase]

Your parity check speed is fine for that drive complement.    But your other speeds are perplexing -- ALL should hit notably higher peak speeds than what you're seeing.

 

Since you've essentially eliminated the Ethernet cable, the next thing I'd do is switch from the Realtek adapter to an Intel one ... it may not make any difference, but at least you'll have the "best" adapter in your system and can feel confident this isn't an adapter issue.

 

e.g.  http://www.newegg.com/Product/Product.aspx?Item=N82E16833106122

[somail]

Well I got the Intel PWLA8391GTL as suggested.  Write speeds still slow.

 

Since I was on a shopping spree I picked up an extra 2TB WD EZRX.  I used it to replace my 500GB WD AAKS.  After rebuilding the array I did a test transfer and..............Peak speeds of 60MB/s and average of 35MB/s !!!!!!!!!!!

 

That is huge compared to the peak of 15's and average 12's.  No idea why having that drive on the array slowed everything down. It had been full for years so it was never being written to.

 

Either way I am a happy camper.

[garycase]

Depending on the specific revision, your old 500GB AAKS was either a 3-platter, 167GB/platter drive, a 2-platter 250GB/platter drive, or a 1-platter 500GB/platter drive.    The 00TMA0/40TMA0/00YGA0/22YGA0/65YGA0/75YGA0 revisions are the oldest (167GB/platter).

 

Your new 2TB EZRX is a 1TB/platter drive ... so it has FAR better areal density than that old AAKS.

 

You should also notice a nice improvement in parity check time, although that old Samsung drive may mitigate that [that's the next one I'd replace].

 

 

[somail]

Thanks. It was a 75YGA0 (still have one more in there too).  Here I was not even caring about platter size.

[garycase]

Thanks. It was a 75YGA0 (still have one more in there too).  Here I was not even caring about platter size.

 

Since you still have the same drive in the system, your parity check speed won't change at all, since the other one will still limit the speed for the first 500GB of the checks.    When you eventually get rid of that last 75YGA0 unit, you'll be very pleasantly surprised at what happens to your parity check speed 

 

I took a look at your other drives as well.  The Samsung is a 2-platter, 320GB/platter unit, so once you get rid of the other 75YGA0 it will be the limiting drive for parity check speeds (until the check passes the 640GB point).    After that, your 500GB/platter Seagate will be the slowest unit.