Transfer speed without parity drive

[Nokoff]

Hi Guys, Just a quick question.

 

What will the bottleneck be when copying data over the network to my array before I have added the parity drive?

 

Small files really clog up the transfers, and on larger files the max throughput i get is about 54mb/s copying to various drives, although I previously copied at speeds between 60-70mb/s on windows. Any ideas? Will a cache drive improve this perhaps? Or should I just be getting the speed of each individual drive.

 

Side note: When you access a large share spanned across all drives, does unraid need to spin up all drives involved to list folder contents?

 

Thanks for your time

Nokoff

[garycase]

If you copy data before you've assigned parity, there's no reason to have a cache drive => the data will write as quickly as it can to each of the drives, limited only by your network transfer rate and the speed of the drives.

 

Once you assign a parity drive, write speeds should (on a Gb network) be in the 30MB/s range ... possibly as high as 40MB if you're using 7200rpm drives.  That's when you may want to use a cache drive to speed things up a bit.  [recognizing that using a cache drive means writes to UnRAID are "at risk" until the files are moved to the protected array]

 

[garycase]

Side note: When you access a large share spanned across all drives, does unraid need to spin up all drives involved to list folder contents?

 

That depends on what it currently has in the directory cache ... but the answer is that it might.

 

You may want to add Cache_Dirs to your Go file.    Read the details here:  http://lime-technology.com/forum/index.php?topic=4500.0

[Nokoff]

If you copy data before you've assigned parity, there's no reason to have a cache drive => the data will write as quickly as it can to each of the drives, limited only by your network transfer rate and the speed of the drives.

 

Once you assign a parity drive, write speeds should (on a Gb network) be in the 30MB/s range ... possibly as high as 40MB if you're using 7200rpm drives.  That's when you may want to use a cache drive to speed things up a bit.  [recognizing that using a cache drive means writes to UnRAID are "at risk" until the files are moved to the protected array]

 

So no explanation really as to why I'm currently experiencing lower than windows speeds writing to the drives?

 

I'll check out the cach directories thanks! Looks like just what i was asking.

[garycase]

So no explanation really as to why I'm currently experiencing lower than windows speeds writing to the drives?

 

Could be a variety of things ... but most likely it's simply the fact that you have to send everything across your network in addition to writing to the disk.    Could also be that your Windows drives are 7200rpm, whereas your UnRAID drives are 5400 or 5900rpm "green" drives.    Could also be that the writes you're timing on the Windows box are being written to the outer cylinders of the drives, and the UnRAID writes are to middle or inner cylinders (slower).

 

But the bottom line is it seems like it's working just fine.

 

[Nokoff]

This is really unbearable. The topic you touched on with now writing to the outer cylinders of the drive is plausible as I haven't really replicated that with the same drives in windows.

 

Is there some software that I could use to run a benchmark or just measure the time taken to copy over a few sample files/folders that I can run to illustrate just how bad the problem is?

 

Just doing a manual test, What i've done is select 2 folders containing 331 files, size 70.3mb to copy over.

 

This literally takes under a second to copy from one drive to another on my pc. Trying to transfer those same 2 folders to my unraid box is near impossible.

 

I have no plugins loaded, just clean unraid 5RC (latest).

 

It is worth noting that I initially configured the array in Unraid 4.7, then upgraded to 5.x however I still have no parity drive so I assumed it was quite straight forward to upgrade. Is it possible that something changed from the way 4.7 configured my drives to the way 5.x is now reading them or trying to write to theM?

 

Eventually i get this error:

 

 

 

 

 

[Nokoff]

I can confirm that the problem lies in accessing the Tower over SMB share.

 

To test I connected via FTP to do a transfer and got much greater speeds.

 

See Screenshots: (FlashFXP used for FTP)

[garycase]

I don't have a system without a parity drive to test on, but I did do a few tests with smaller files -- 550 files in 30 folders totaling 400MB -- and indeed it slows down to below 10MB/s, whereas a single 500MB file copies at max speed (~ 35MB/s writes to a parity protected array) ... with a cache drive this would almost certainly be much faster.

 

I suspect what's happening is simply that the directory writes aren't being cached ... so there's a directory update and write after EACH file -- which slows things down a LOT.    If you want to see the impact of this, change your Windows write policies to turn off write caching; then do a disk-disk copy in Windows 

[Nokoff]

I don't have a system without a parity drive to test on, but I did do a few tests with smaller files -- 550 files in 30 folders totaling 400MB -- and indeed it slows down to below 10MB/s, whereas a single 500MB file copies at max speed (~ 35MB/s writes to a parity protected array) ... with a cache drive this would almost certainly be much faster.

 

I suspect what's happening is simply that the directory writes aren't being cached ... so there's a directory update and write after EACH file -- which slows things down a LOT.    If you want to see the impact of this, change your Windows write policies to turn off write caching; then do a disk-disk copy in Windows 

 

Gary the speeds are terrible though, and that is on a LARGE file transfer. 2+GB. I can't live with -10mb/s. Please someone must be able to help!

 

Read is just as bad as write. Here is copying a ISO file from the unraid server to my pc over network. The problem lies in the windows shares I'm sure of it, if I use midnight commander to copy between the drives on my server the transfer speeds are 60-70mb/s.

 

 

 

[dgaschk]

 

I have no plugins loaded, just clean unraid 5RC (latest).

 

 

5RC is not the latest. The latest in RC16c.

[garycase]

Agree -- if you're getting the slow speeds even with large files (100MB+) then something is definitely not working correctly.    Just for grins, I tried copying a large file to/from the array from a Windows 8 system (since that's what you're using) ... but it works fine on mine, so that didn't help provide any clues.  This almost has to be a write caching (or lack thereof) issue ... but I'm not sure how to alter this for network transfers.

 

[garycase]

 

I have no plugins loaded, just clean unraid 5RC (latest).

 

 

5RC is not the latest. The latest in RC16c.

 

I think "5RC (latest)" means v5, Release Candidate, latest version 

... I suspect he's running !6c  [Nokoff -- is that correct?]

 

[garycase]

This thread is a bit dated, but addresses the same issue (slow SMB write speeds).  Might want to try a couple of the modifications suggested in it:

 

http://social.technet.microsoft.com/Forums/windowsserver/en-US/46898c7f-92e0-4c99-98d2-18a7458a7d2d/slow-network-write-speeds-via-smb-cifs

[Nokoff]

This thread is a bit dated, but addresses the same issue (slow SMB write speeds).  Might want to try a couple of the modifications suggested in it:

 

http://social.technet.microsoft.com/Forums/windowsserver/en-US/46898c7f-92e0-4c99-98d2-18a7458a7d2d/slow-network-write-speeds-via-smb-cifs

 

I had a look at some of the things in that thread and nothing worked/applied for/to me Any other suggestions? I know I'm asking a lot but if someone else could assist please maybe we could collaborate our ideas.

[dgaschk]

Attach a syslog.

[Nokoff]

Does Tower kernel: sky2 eth0: rx error mean anything? 

 

And I don't know if it helps to know that file transfers seem to take 30-60 seconds to eve nstart transferring any data at all, like it's timing out? 

syslog.zip

[dgaschk]

This is a bug. See http://comments.gmane.org/gmane.linux.network/228957

 

Try the latest release, RC16c.

[Harpz]

I can confirm that the problem lies in accessing the Tower over SMB share.

 

To test I connected via FTP to do a transfer and got much greater speeds.

 

See Screenshots: (FlashFXP used for FTP)

 

Little off topic sorry what is the top screen shot using and the other 2 showing the file transfer using also, the transfer with the graphs.

 

I may have to try to FTP to my box and send files that way the max transfer i ever seen is 48MB/Sec copying over loads of rar files and late 50s when copying over large mkv files and thats with a cache drive.

 

Your getting 55.87MB/Sec I'll have to see what i can do, was it easy to set up the FTP?

 

 

[Nokoff]

I can confirm that the problem lies in accessing the Tower over SMB share.

 

To test I connected via FTP to do a transfer and got much greater speeds.

 

See Screenshots: (FlashFXP used for FTP)

 

Little off topic sorry what is the top screen shot using and the other 2 showing the file transfer using also, the transfer with the graphs.

 

I may have to try to FTP to my box and send files that way the max transfer i ever seen is 48MB/Sec copying over loads of rar files and late 50s when copying over large mkv files and thats with a cache drive.

 

Your getting 55.87MB/Sec I'll have to see what i can do, was it easy to set up the FTP?

 

The Unexpected network error message was using windows 8 file copy (maybe appears differently due to Classic Shell Application). So was * SMB slow.PNG using windows file copy.

 

The ftp application used was flashfxp. I couldn't connect via ftp in 5RC15, but after downgrading to 4.7 I was able to ftp in with username root, pass blank. That 55mb/s is reading from the unraid hey, not writing.

 

dgaschk I tried upgrading to the latest RC16, one transfer seemed to go well, the next (writing to unraid) went wonky, but I accidentally assigned my cache drive to disk 5 instead of cache, how do I undo this? When I try to unassign disk 5 and assign it to cache disk it says "Too many wrong and/or missing disks!" - nvm fixed this by going to utils->New Config and reassigned the drives.

 

I'm testing this now, will report back. Confirmed still getting errors trying to write to the disk.

 

 

 

[dgaschk]

Attach a new syslog

 

[Nokoff]

New Syslog attached. Here is the behaviour demonstrated before generating it.

 

Copying files from unraid to my pc over the windows network.

 

[Nokoff]

More attachments.

syslog.zip

[Nokoff]

Hello? Could a lime-tech person please step in here and offer some assistance?

[limetech]

More attachments.

Is the system log before or after the transfer was terminated?  After transfer terminated (or let it run few minutes), what is the output of:

 

ifconfig eth0

 

and

 

ethtool eth0

[Nokoff]

I got really desperate and I tried changing a lot. I knew the problem was somehow on my end and that I could fix it if i changed something....

 

I downgraded to 4.7

I moved the PC upstairs to my main network switch (same as the one my windows 8 pc connects to)

changed the network cable

disabled the secondary onboard network adapter

 

All in one go. SOmething here seems to have worked! I will keep testing and report back to confirm. Thank you for all the people that took time to have a look at my problem guys! I appreciate it.