[Solved] What kind of data rate...

[ctviggen]

Should I be getting for copying from one computer to the Unraid server?  I have one computer hooked up to a 3TB backup drive (WD green drive) over eSata.  The server is using three 1.5TB WD green drives, two for data and one for parity.  I'm copying from the 3TB drive to the server.  The parity check was completed yesterday.  Both computers are connected to cat 6 cabling and to a 1gbit/100mbs switch.  There is no cache drive as of yet.  I may install an SSD cache drive (and another 1.5TB drive; I have four, but one did not pass preclear).

 

I'm seeing 10.5 MB (megabytes or megabits?) per second.  

 

Does this sound reasonable?  If not, is there anything I can do to speed up the process?  On a daily basis, I should be copying recorded TV from two computers to the Unraid server, and will copy movies and music when I buy them.  

[stomp]

This is pretty low but you're transferring chain is not optimal e.g. it's sub-optimal (I wanted to use this word, sorry).

 

What you should do is trying to copy some files from your eSATA HDD to one of your PC's internal HDD (report the speed here) then copy these same files from your PC's internal HDD to unRAID server (again report the speed here).

 

I'm assuming you're using Samba to transfer the files (and not SMB2). But considering your speed, it might be possible that your network is capped to 100Mbit (which have a theoretical bandwidth of 12.5MB/s). In this case, something on your network is not Gbit compliant.

[ctviggen]

Thank you. I'll check this out.  According to the computer I'm on, it says it has a 1GBit connection.  I assume the Unraid server has a 1Gbit connection and I'm using a Gbit/100Mbps switch, but I'll check to ensure I haven't done something wrong.  I'm getting even slower speeds now, down to 6 MB/sec.

 

I installed the network myself, but I do have a mixed network of cat 5e/cat 6.

 

I can put the eSata drive into the unraid computer, but I was hoping not to do so, since I have to take down the server.  However, I need to get rid of a 1.5TB drive that doesn't work, add in a 350GB drive a cache (once it's precleared), and add in a fan to replace one of the fans that isn't working so well.  So, maybe I'll do that and then copy from there.

[chickensoup]

The type of cable won't cap it at speeds like that. My guess is the ethernet on your unRAID is 100m/bit.

[Johnm]

It does sound like you are running at 100Mb (thats megabits) network speed.

 

Be aware that using an older drive (I am assuming it is older) as cache drive can sometimes be slower then no cache drive at all.

[olssy]

I've been using iperf to debug my network issues, you should give it a try. Install it on both machines(UnMenu offers it and it is available for windows and linux), run one as client and one as server

iperf -s

iperf -c {ip of server machine}

 

It will at least let you know if it is a netowrking issue or a samba issue as it takes samba out of the equation.

[ctviggen]

Thanks, I'll give that a try.  There's one switch (for my whole network), and it says both the Unraid server and the computer with the eSata drive are operating at 1gigabit (each port has one light for 100mbit and two lights for 1gigabit).  So, I'm not sure where the holdup is.  Now, I'm also using the eSata drive to serve some of the house for pictures or music, so perhaps that's the problem.  I turned off the one computer that would request pictures, but I'm still serving music.  I do have music copied finally to the Unraid server, so I'll remove that load from the eSata drive soon. 

[ctviggen]

It does sound like you are running at 100MB (thats megabits) network speed.

 

Be aware that using an older drive (I am assuming it is older) as cache drive can sometimes be slower then no cache drive at all.

 

I'm not sure how "old" it is.  It's a 350 gb drive that uses SATA.  I'm about to pull it out and see what it says on it, so I'll get the drive info.  I was thinking of buying a SSD, but I'd need 128GB just in case, and that's $200 or more.  If the 350GB doesn't work, then I'll try the SSD.

[joshpond]

Thanks, I'll give that a try.  There's one switch (for my whole network), and it says both the Unraid server and the computer with the eSata drive are operating at 1gigabit (each port has one light for 100mbit and two lights for 1gigabit).  So, I'm not sure where the holdup is.  Now, I'm also using the eSata drive to serve some of the house for pictures or music, so perhaps that's the problem.  I turned off the one computer that would request pictures, but I'm still serving music.  I do have music copied finally to the Unraid server, so I'll remove that load from the eSata drive soon. 

 

that could be your problem. If the drive you are copying off is being read by something else as well that can cut your speeds drastically. The head on the HDD has to keep switching between 2 areas.

 

When I copy from my cache to the array and something like transmission is writing to the cache my copies are slow.

 

Josh

[ctviggen]

Thanks, I have the eSata connected to the Unraid server.  Just need to figure out how to mount it and get copying.  I'll let everyone know what happens.

 

As for the cache drive, this is the drive I was thinking of using:

 

Western Digital Caviar SE16 WD3200KS 320GB 7200 RPM 16MB Cache SATA 3.0Gb/s 3.5" Hard Drive -Bare Drive

 

It's been sitting in the current computer (made into an unraid server this week), not connected and without power, for quite some time now.  Assuming it passes preclear, is it suitable for use as a cache drive?

[Johnm]

hrmm..

 

that is one of those 2005-2007 ide  to sata transition drives that came as both IDE and SATA flavors. My guess is it would not be a stellar performer..

 

the best way to test it is to try it. copy some files to the array and time it. add the cache drive. erase those files, copy them to server again and time it again.

 

there is no reason you cant use it as cache for the time being or as a storage drive. with drives priced as they are now, every gig counts...

[stomp]

I have one of those 3200AAKS. Performance is ok but not stellar. I personally use a Samsung HD320GJ (1 platter of 320GB, high density per platter). This drive is capable of excellent read and write performance. SSD would be overkill, the bottleneck sits in the network. You would hardly be able to use more than 90-95% of your Gbit link even though your cache disk could write as hell.

 

For your speed problem, cables don't matter. It seems to me that unRAID may recognize your NIC as 100Mbps. What's your motherboard model? Also as mentioned, you can't judge transfer speeds if the source (your eSATA drive) is accessed at the same time from other devices. You should turn everything off and try again.

 

Now with 6-10MB/s, it's not a cache drive issue... What kind of drive delivers such poor performance?! My 5-year old 3200AAKS can do 100MB/s...

 

Please provide an unRAID log so we can clear if your link is really running at Gbit speed.

[ctviggen]

Thank you.  I've copied all of the data that another computer in the house would use to the Unraid server, and I've configured those computers to use the Unraid server instead of the computer I'm typing this on (which had the eSATA drive).  There should be no accesses to the eSata hard drive.

 

Well, I attached the eSATA drive to the Unraid server, only to realize that I have the drive formatted with GPT (since it's a 3TB drive).  My version of Unraid is 4.7 (only because it's the latest non-beta).  I couldn't quickly determine how to mount the drive, so I've reverted back to copying the files over the network.  As a father of two young children with a wife who's mad at you because you're spending too much time at the computer, I need to do everything as quickly as possible.  And, this computer that I just finished building is having random reboots that I haven't tracked down yet.  I'm still getting poor network performance. 

 

Thanks for the info about the drive.  I'll try to fix the network issue first, then determine whether the drive should be a cache or another drive in the array (it did pass preclear). 

[Johnm]

As a father of two young children with a wife who's mad at you because you're spending too much time at the computer, I need to do everything as quickly as possible. 

 

you set the copy and walk away for 12-24 hours to hang out with them... multitasking in a good way.

I think everyone here is guilty of watching a data transfer for 12hours... why do we do that?!?!

[ctviggen]

As a father of two young children with a wife who's mad at you because you're spending too much time at the computer, I need to do everything as quickly as possible. 

 

you set the copy and walk away for 12-24 hours to hang out with them... multitasking in a good way.

I think everyone here is guilty of watching a data transfer for 12hours... why do we do that?!?!

 

Well, she has a pretty good reason.  I bought a fanless computer.  It took two nights just to build it, only to realize the motherboard was bad.  Then received the new motherboard, and got it working.  Installation of all my programs takes hours (unfortunately, can't leave at that time).  I tried to parallel process by taking the old computer and setting it up as Unraid at the same time, but that's slow too (been a while since I've telnetted anywhere; a command line interface?  I don't even remember those).  Then, one drive didn't pass, and I had to determine what to do.  Then, I started copying folder-by-folder.  Meanwhile, my new, fanless computer would randomly reboot (which I believe -- fingers crossed -- I may have fixed).  That means you'd start copying 500GB of material, only to come back and have the computer rebooted with an error message.  You can restart and it takes less time, but windows stops partway through and asks if you want to copy over files.  And it waits until you answer.  So, I was constantly coming here and staying here.

 

On the other hand, all the data appears to be on the Unraid server.  This will allow me to set everything up and make the network easier to use. 

 

By the way, "ethtool" says Unraid is operating at 1gbps.  So, I still have some testing to do.  I'm still getting less than 10MB/s copy rate copying from the eSata drive to the Unraid.  Once the final files are copied off the eSata drive, then I'll set up a test from this computer's internal drive to Unraid to see what I get.

 

Again, thanks for everyone's help.

[ctviggen]

hmmm...Tried a copy directly from the C drive of this computer to the unraid server (that is, without the eSata drive), and I'm still getting less than 10 MB/sec.  I enabled the cache, and it's about the same.  I'll have to do more debugging tomorrow. 

[ctviggen]

I marked this as solved so that I could ask questions directed more to my network and it's issues.

 

By the way, when I copied a file from the eSata drive to the internal drive of the computer connected to the unraid server, the filed copied at 100 MB/sec (approximately).  So, that drive likely isn't the problem (once the data was off that drive and onto the unraid server and no computer was connected to the drive).