Tranferring files from XBMC to UNRAID/movies share using WINCSP - its displaying the speed as 1,114 Kib/s is whats expected?

 

UNRAID Specs

 

Unraid v 4.5.4

Gigabyte GA-MA74GM-S2H

1GB RAM

Sempron 1.2Ghz

1 x WD 750GB RE3

3 x WD 750GB Cavier Black

 

Netgear router 100/MB

CAT5e

100 MB Switch connected to Server

 

The parity drive is the RE3

I presume winscp is literally using scp as the file transfer mechanism? as opposed to straight ftp?

 

In which case your box is having to run encryption over the traffic and I'm not sure how up to the job your sempron would be. Long time since I looked at anything with sempron in the name so I could be very mistaken.

 

You can tweak your ssh config at server side and / or at the winscp client side to use a different, faster, algorithm which might help speeds. You could also look at enabling / disabling compression over ssh which may or may not help too.

Your sir are correct, just downloaded and checked it at work, it has protocol drop down box which defaults to scp!

 

Will go home and change to ftp!

 

But also considering putting in an extra gig!

And upping the cpu?

 

Would this also help with write speeds?

I'm making a relatively sweeping assumption (without really knowing much about that particular cpu) that your write speeds are bottlenecked by the CPU being too slow for the encryption overhead of scp via ssh.

 

In which case more ram is unlikely to help. But a faster cpu *probably* will.

 

If you're switching to normal FTP all that goes out the window - see what rates you get, ftp should be relatively unaffected by CPU or memory (certainly at 100mbit speeds) though more memory will always help in general / will allow the filesystem to cache more.

Your problem is more likely the 100MB switch, not the CPU.  Switch to Gigabit ethernet the whole way.

 

You can run top during a file xfer and prove that the CPU is (or is not) a bottleneck.

Am I getting my notations mixed up?

 

He's transferring at ~ 1 megabyte per second.

 

100mbit should be capable of 10 times that speed.

100mbit should be capable of 10 times that speed.

 

Nope.  Ethernet is a CSMA/CD scheme and with the max window size, it runs into problems at about 1/3 of max, or 30mbit/sec.  Slower when you have multiple devices in the same collision domain (but can be cumulatively faster).

 

A crappy hub, a bad crimp, some other issue, and 8mbit/sec on 100megbit Ethernet (8% utilization) is not unexpected.

I've read those figures alot on these forums and don't agree based on real world examples of 100mbit or gigabit links reaching saturation at very close to their rated speeds. But that's a separate argument

 

In this instance :

 

- he's not using a hub so CSMA/CD is less (arguably not at all depending on the backpressure on the switch) of an issue

- the collision domain in this instance is tiny, presuming each machine is on it's own switch port.

 

I agree in checking the cabling (I had made the perhaps foolish assumption that had been checked) and also the CPU load.

 

I'll happily eat my words if an upgrade to gigabit makes it appreciably faster, but a 100mbit switch should be able (pending backplane speeds) to chuck 10 megabytes per second between two ports all day long.

 

I have seen terrible sftp / scp speeds in practice on fast links by overloaded cpus, underpowered cpus or heavy encryption algorithms being used. Which is why I'd lean towards that.

 

8% utilization on a 'clean' 100mbit network segment is not right from a networking point of view.

 

Both easy to check though  - happily be proved very wrong

100Mb "switches" are generally very expensive.  Consumer-grade 100mbit is most likely a hub (shared collision domain) and not a switch.

 

GB Ethernet is a always a switch (cutthrough mode or s&f)... there is really no such thing as a GBE "hub."

 

Yes, you can get higher than the 1/3 throughput, but only by doing things not available to the general consumer (i.e. require equipment or hacking the registry).

 

I agree 8% util on a "clean" install is poor.... I unfortunately have found many "poor" installations.

I bought a 100mbit switch off the shelve years ago for the low low price of £20. Whilst never using all it's 8 ports I did saturate 4 of them at full 100mbit line rate with ease.

 

It's easy to buy a 100mbit hub thinking it's a switch, but if you're sure it's a switch then...

 

1/3 throughput on a hub I will defer to you on as it's been so long since I've seen a hub passing traffic I really can't remember much about their peak efficiencies. They did come with collision warning led's if I recall which is never a good sign!

 

But 30% on a switched network, in my experience, has purely been a figure for capacity planning rather than what you can actually shove down a port.

Ya, I don't buy that speed arguement. It's can't be the network unless you've got a problem with the router or with a bad cable. I agree that something is likely tying up the resources on one end.

 

I was using a $40 (bought years ago) Linksys WRT54 router with no special setup and could easy run run around 11MBps copying to my server.

 

I then replaced that router with a cheap $60 Trendnet, I think the TEW-639GR model, and it allows reads from my server faster than 60MBps. I did some tests but that was a while ago. I believe it might have been over 70MBps but I just can't recall the actual number.

 

I don't use any special program to monitor but the network utilization line in the task manager at the high update speed is a flat line. I highly doubt it dips between each sample. Also a quick size vs time check confirms the speed number is quite reasonable. I don't use any parts better than readily available cheap consumer grade stuff.

 

A low speed number can occur on a bad installation but then the installation needs to be fixed. Do a check on the ethernet statistics to see if there are transfer errors.

 

Peter

 

Hi,

 

The Netgear Wireless G router downstairs has got 3 ports used

- PS3

- XBMC

- Cable (CAT5e) going upstairs (15m) to my cheapo 10/100 hub

 

My laptop is connected wirelessly to the router

 

Some findings;

-Using WINSCP (FTP)  on my laptop - transferring files to UNRAID from XBMC = 1,154 KiB

-Mounted share on my laptop copied a 700mb avi from Laptop to XBMC = 2.5MB

- Bypassed Hub connected cable straight to server then using WINCSCP (STP) on my laptop

 to UNRAID from XBMC = 1,114 Kib

 

What do you suggest i try next?

- Plug cable direct into server and try a ftp transfer

- Thinking of taking my server downstairs and connect directly into router - so at least i will know for sure whether its the 15m CAT5

 

 

Just had another thought!

 

Plug my laptop into the server via CAT5 and transfer the 700mb avi file!

Considering both the server & laptop have gig nics it should whizz through

The plot thickens!

 

Now it *is* a hub and not a switch. And we have wireless.

 

All bets are off.

 

If you're doing wireless laptop -> any target then I'd say that's an acceptable throughput.

 

Apologies to bubbaQ - still disagree with the switched speed theories but you were very right to suspect foul play with the network setup based on the original description!

I dont get it?

 

If initiate a transfer from my laptop from one wired client (XBMC) to another wired client (UNRAID),

the data goes does not go through the laptop? correct?  

 

Also apologies over my initial description if it misled anyone, sorry

How are you initiating the transfer?

 

-Using WINSCP (FTP)  on my laptop - transferring files to UNRAID from XBMC = 1,154 KiB

 

The only way I can think of you doing that is using FXP but that requires 'straight' FTP - which you weren't using, you were using scp / sftp.

 

http://winscp.net/eng/docs/faq_fxp

 

It all sounds like the transfer is going via your laptop one way or the other. Even if you are using FXP it's easy to get your config wrong / port options mixed up enough to run the transfer via the control host. But as winscp doesn't support this it's a moot point.

 

If you're running WinSCP..and connecting to XBMC in one pane and then in the other connecting to unraid and dragging and dropping, it will transfer via your laptop. It is not transferring directly between the hosts. So your laptop will be downloading from XBMC (via wireless) then sending to unraid (via wireless again). The wireless being the obvious sore point.

 

I'm as confused as you are as your networking has changed completely from your initial post, I seem to be digging myself into a large hole!

 

Tips :

 

- completely remove the laptop from the equation. Do direct transfers from host to host, do not initiate anything using the laptop. Unless you can give us more detail on exactly what you're doing I suspect it's not working as you assume / routing through the laptop.

 

- if you must use the laptop don't use wireless, plug it in.

 

- find out if you have a hub or switch. If it's a hub throw it in the bin and replace it with a switch. From the new description I doubt it's causing too much trouble but as bubbaQ has already stated they're very inefficient and for the price of a cheap 100mbit or even gigabit switch..

 

- If you're trouble shooting your unraid speeds, plug the laptop directly into the unraid server and copy a file from the laptops local disk to unraid.

 

- If you're trouble shooting your network then you have multiple places you can plug things in to test each segment. Do all this and it should become apparent if or where there is an issue. Also make sure you know exactly where and how traffic is flowing - don't assume what it's doing, use netstat / other network monitoring tools to make sure.

 

I would still expect better speeds that you're seeing, but definately not if your transfer is going via wireless.

 

If the suspected hub is also 'busy' with traffic from other clients at the same time then, as bubbaQ has pointed out, it's not unlikely that your throughput could be decimated by collisions.

 

Thanks for the info

 

Will try filezilla as it seems to have fxp support!

The hub should be fine as it shares the connection to the network printer, which is in sleep mode

 

Im going to do the laptop into server test, too give me an idea of what speeds to expect anyway

in my wired network.

 

The *ftp* server's at both ends will need to support fxp.

 

I don't know how you've configured xbmc but I don't think unraid supports it by default.

 

pasv_promiscuous

Set to YES if you want to disable the PASV security check that ensures the data connection originates from the same IP address as the control connection. Only enable if you know what you are doing! The only legitimate use for this is in some form of secure tunnelling scheme, or perhaps to facilitate FXP support.

 

Default: NO

 

You would need to edit the vsftpd.conf and restart.

 

And you cannot use fxp with sftp or scp.

 

Far easier would be to telnet to the unraid server and then ftp from there into XBMC and pull the files down directly. This would completely avoid any laptop shenanigans.

Thanks again!

 

I wish i could get Telnet to work but thats another story! lol

 

Would i need to install a ftp client on UNRAID?

telnet should be enabled by default!

 

There is an ftp client as part of stock unraid.

telnet should be enabled by default!

 

There is an ftp client as part of stock unraid.

 

Did a search for ftp, went through 10 pages of results and could not

find how to use the client.

 

Could you point me in the right direction?

It's just the basic linux / unix client.

 

http://linux.about.com/od/commands/a/blcmdl1_ftpx.htm is probably as good a page as any!

 

 

Having problems ftp'ing

 

I manage to connect fine and browse XBMC, I set the local directory using the LCD command

to /mnt/user/movies

 

Then i use

 

prompt (turns off prompt)

 

mget *

 

It says its downloading bourne.mkv for example but then it seems to hang

so I have to do a hard reset. And when I check my movies share only 530mb

has been transferred!

 

 

Back to using winscp

Just to say thanks to boof for the help, managed to get the files across using a portable usb drive!

And I just finished off my Sabnzbd installation!

 

Everything seems to be coming together nicely!

 

Many Thanks