So I bought a WD Red 3TB to replace an older WD Green 3TB as the cache drive.  I read up on how to transfer from cache drive to a precleared drive for an easy swap out.  I have programs running on the cache drive and would like them to move to the new drive including sabnzbd, sickbeard, couchpotato, dropbox, etc.

 

I tried using midnight commander as I thought it would be the easiest option but for some reason it is very slow to copy. 

 

Max transfer I saw was 2.2MBps and It constantly goes down to the mid KBps range.  I want to copy about 1.4TB worth of data but at these speeds it seems not worth waiting for.  I tried copying the relevant folders directly to the new drive (drive has been precleared and has reiserfs)  and also to an existing drive in the array but the same slow speeds are there.

 

Even tried mounting them on a computer with ubuntu for a direct copy.  The permissions in ubuntu won't allow the copy so I don't want to mess around with that.

 

I have also copied about 700GB worth of data over the network in a few hours to an external portable drive and the speeds there were fine (almost 90MBps)

 

Is there anything I can do to speed it up because obviously this is not normal to have such a slow movement of data within the server itself?

 

 

 

syslog-2013-06-26.txt.zip

Here is the hdparam of the relevant drives.

 

Input is hdparm -Tt /dev/sd(letter of drive)

 

Cache WD Green

/dev/sda:

Timing cached reads:  23836 MB in  1.99 seconds = 11980.64 MB/sec

Timing buffered disk reads: 344 MB in  3.00 seconds = 114.54 MB/sec

 

Future cache (not in array but mounted) WD Red

/dev/sdk:

Timing cached reads:  23300 MB in  1.99 seconds = 11680.86 MB/sec

Timing buffered disk reads: 420 MB in  3.01 seconds = 139.56 MB/sec

 

Drive currently in array also used to check speed WD Red

/dev/sdj:

Timing cached reads:  21662 MB in  2.00 seconds = 10857.32 MB/sec

Timing buffered disk reads: 456 MB in  3.00 seconds = 151.92 MB/sec

update to the latest release.

I have updated it to RC15a and tried copying again.  On certain files it transfers around 75MBps and on others it can be 300KBps.  I think it has something to do with how midnight commander handles small files.  The larger files seem to transfer easily whilst the small files appear to bog down the system.

Many small files will be much slower than one large file.

I saw on Midnight Commander how a lot of the files being transferred that were slow happened to be in the .apple* folders.  I ran this

 

find . -name \.AppleDouble -exec rm -rf {} \;

 

and it deleted most of the problem files.

 

Then I copied each application folder one by one.  Probably averaged 45MBps overall because of the small files clogging up the system.  Everything has been copied over and the drives have been successfully swapped over.  Now deleting everything off the old WD Green cache drive to add it to the array.