Slow Data Copy Rate To unRAID Array?

[woolybully]

Hi Guys,

 

I have a new rig (my first) and have followed instructions in the forum about copying large amounts of data to the aray. un-allocating parity and using TeraCopy, however I'm getting transfer rates of about 3.3MB/s.

 

The data is currently on a Buffalo Linkstation Quad (running as Jbod). I am using TeraCopy installed on Vista to move the data.

 

LAN is 1Gb (switch reports 1Gb to unRAID rig, Linkstation and Vista PC) AHCI has been enabled on the Mobo's SATA ports.

 

Is this right to you guys? Any ideas? I've seen posts with much higher rates.

 

The performance of my Linkstation is crappy  (hence the move to unRAID, which I'm loving btw  ) could it just be this slowing things down?

 

Your thoughts and hints would be appreciated.

 

 

[kizer]

My first inital questions are.

 

1. What is the speed of the disks in your array. All 7200, all 5400 or a mixed bag.

2. Are you copying via wireless?

3. Have you tried copying files directly from your PC not from the other device.

 

Either way your saying 3.3MB/s on Terracopy right? That does seem awefully slow.

[bubbaQ]

Likely a crappy switch coupled with a crappy NIC.  Not to mention you are copying up and down the same wire to your PC.

 

Map the TC drive directly from unRAID, and pull the data over the wire that way.

[woolybully]

kizer

 

Thanks for the reply.

1. All 7200

2. Not wireless all over 1Gb copper connections with 1Gb switch and NICs

3. No, good point though :-D

 

bubbaQ

 

Again, appreciate your reply. I'd be suprised if it was the switch or NICs, they have been rock solid and the unRAID mobo (NIC onboard) is brand new.

 

Could you please elaborate on and instruct how to

Map the TC drive directly from unRAID, and pull the data over the wire that way.

or is there already a thread/post on this?

 

Many thanks

[bubbaQ]

You are doing something now, that you have not done before, so prior "acceptable" operation is not an indicator of equipment stability.

 

You are sucking a bunch of data down into your PC from TC, and then sending it right back out over the SAME wire to the AME port in the switch.  This is very different than reading or writing so/from unRAID or TC.... you are doing BOTH at the same time.

 

Use samba to mount the TC drive directly in unRAID, and copy the data that way.

[woolybully]

Thanks bubbaQ

 

I'm really new to unRAID and Linux, could you please tell me the syntax required to mount the network drive and then copy? I'd be logged on via the console

 

Would it be something like this?        mount -t cifs -o guest //server/share /x/tempfiles

 

Thanks

 

 

[bubbaQ]

Yup

 

Nohup your copy command in the background, and then let it run for days if needed.

[woolybully]

Many many thanks

[woolybully]

Hello again,

 

Now I have been using the command

mount -t cifs -o guest //server/share /x/tempfiles

to move data and have found then when I navigate to the folder from a Windows PC the directory structure is there but not the files within. The file browser in unMENU shows the files there! OK now I'm confused 

 

I'm guessing permissions and have googles chmod but can't seem to get things right... help please oh wonderful forum 

 

 

[Joe L.]

Hello again,

 

Now I have been using the command

mount -t cifs -o guest //server/share /x/tempfiles

to move data and have found then when I navigate to the folder from a Windows PC the directory structure is there but not the files within. The file browser in unMENU shows the files there! OK now I'm confused 

 

I'm guessing permissions and have googles chmod but can't seem to get things right... help please oh wonderful forum 

 

 

Do you have the window's explorer set to show "hidden files" (you should)

Do you have the window's explorer set to show "system files" (you should)

[woolybully]

Hi Joe,

 

You nailed it...

 

I had "Hidden Files" set to show and didn't think that "System Files" would make a difference, but I set it to "Show" and hey presto.... Many thanks

 

[BRiT]

On the unRAID server you can try the following to perm remove the System and Hidden mapped attributes (replace /mnt/user with any specific root share or directory you want to fix) :

 

find /mnt/user -type f -exec chmod 0644 {} \;

 

If you're running an unRAID 4.5 you can change the way SAMBA maps hidden and system and archive bits by using a 'smb-extra.conf'. For more information see this thread: http://lime-technology.com/forum/index.php?topic=6860.0

 

It could look like this:

 

[global]
map hidden = No
map system = No

 

This is how SAMBA maps between unix and Windows file permissions:

 

 

[woolybully]

Many thanks BRiT. I'm a real newbie to anything other than a Windows OS so info like this is really valuable.

 

I'm loving learning and loving pulling away from MS 

 

 

 

[lionelhutz]

Out of curiousity, what is the transfer speed when you read from that Buffalo to you PC? If that unit is any slower than about 30Mbps then you might as well enable parity because it won't slow anything down.

 

I have moved data between drives on my unRAID server at about 25Mbps with parity enabled using my PC. But copying between 2 different network devices using a PC could be slower.

 

Peter

 

[woolybully]

Hi Peter,

 

You're right the Buffalo unit is sloooooow hence the move to unRAID 

 

Buffalo --> PC (using TeraCopy) = ~5.0MB/s

Buffalo --> PC --> unRAID (using TeraCopy) = ~4.0MB/s

Buffalo --> unRAID (using cp command on console) = Much better!

 

I have re-enabled parity to no ill effect of transfer speeds. I have nohup'd the cp command at the console so can't give you an actual speed reading (there probably is a way to get this info but it's unknown to this linux/unRAID newbie) but from my experience the transfers are no slower now we parity back.

 

Just waiting now to empty the contents of the Buffalo on to the unRAID rig (slowly ) then I can steal the HDDs and put them in the unRAID array and get rid of the Buffalo chasis 

 

 

 

 

 

[bubbaQ]

I have nohup'd the cp command at the console so can't give you an actual speed reading

 

Check free space and note exact time.  Wait 10 minutes.  Check free space again and time.  Calc the rate.

[Joe L.]

install bwm-ng

 

Type:

mkdir /boot/packages

cd /boot/packages

wget  -O bwm-ng-0.6-i486-2bj.tgz  http://repository.slacky.eu/slackware-12.1/utilities/bwm-ng/0.6/bwm-ng-0.6-i486-2bj.tgz

installpkg /boot/packages/bwm-ng-0.6-i486-2bj.tgz

Then just type:

bwm-ng

 

It will tell you the transfer rate dynamically.

 

If you wish a 30 second average rate, type:

bwm-ng -T avg

 

Joe L.

 

[woolybully]

Thanks, love the add-ons...

 

Buffalo --> unRAID (using cp command on console) = 4.78MB/sec

 

Not a good as I thought, but will be interesting to see when copyng from another location. Hoping it will confirm that it is the Buffalo that is slow and that is where the bottleneck/slow point is.

 

 

 

[lionelhutz]

Yup, that Buffalo unit is slow to your PC so it's your bottleneck. I'd bet that going PC <-> unRAID will be much faster.

 

If the drives are JBOD, IE a seperate file system on each disk then you should be able to connect them in the unRAID box and copy the contents that way. I guess knowing which file system the drives are using would help. You basically mount the drive and copy the contents off then you can clear it and put it in the array.

 

Peter

 

[kizer]

What kind of speeds do you get copying files from your PC ---> unRAID?

[woolybully]

PC --> unRAID (with Parity ON) fluctuates dramatically.

 

It peaked at 21MB/sec but spent most of the time down around 2-3MB/sec

 

 

[kizer]

Heres my current setup.

 

1.5TB WD 7200RPM

1.5TB WD 7200RPM

500G WD Variable 7200 RPM nonsense

500G WD Variable 7200 RPM nonsense

 

I normally get around

18MB/s upload from another NAS with Parity and I've seen it spike up to around 24MB/s

26MB/s upload directly from my PC. Its not constantly that speed, but its normally in the 20's

39-41MB/s download with of course Parity

 

Are you running any funky type of Anti-Virus or any software firewall software that might be picky?

[kizer]

What kind of speeds are you getting when you download from your unRAID to your PC?

[woolybully]

Hi Kizer,

 

Currently clearing a new HDDto add to the array so it's offline atm so can't test.

 

I have at present:

 

1TB Samsung 7200RPM

80GB WD 7200RPM

2TB WD WD20EARS 7200RPM (Parity)

 

PC has 500GB WD5000AAKS (Caviar) in it

 

I use AVG (Free Ver) anti-virus and no special firewall. Am running Win Vista (*shudder* but that's another story ) on my PC.

 

 

[kizer]

LOL, man you should so use that preclear script.