I have a Gig lan switch but my transfers have been exceedingly slow I feel. I am even wondering if I should have bothered with a Gigabit switch right now... my problem or Unraid's problem?

What program are you using to calculate your speeds??

I have no idea but I can tell you it's crap......!  The reason is that the data is moving from and or thru a windows box onto the UnRaid.  Moving data from drive to drive within the UnRaid itself and NOT using a windows machine to do it but from a telnet session is very fast.  I think it took me over two weeks to move about 130 ripped and striped DVD's, yeah frustrating but it's a one time deal.

 

It's always been high on my list of enhancements to have the ability for the UnRaid software to see a new drive is NTFS (or whatever) and offer to copy the data to an UnRaid drive.  Not sure if Tom will get that done or not.

What program are you using to calculate your speeds??

 

I am using DuMeter

 

http://www.dumeter.com/

I have no idea but I can tell you it's crap......!  The reason is that the data is moving from and or thru a windows box onto the UnRaid.  Moving data from drive to drive within the UnRaid itself and NOT using a windows machine to do it but from a telnet session is very fast.  I think it took me over two weeks to move about 130 ripped and striped DVD's, yeah frustrating but it's a one time deal.

 

It's always been high on my list of enhancements to have the ability for the UnRaid software to see a new drive is NTFS (or whatever) and offer to copy the data to an UnRaid drive.  Not sure if Tom will get that done or not.

 

Gigswitch does not mean transfers of up to 1000Mb. Read this interesting article

 

http://www.anandtech.com/showdoc.aspx?i=2154&p=4

 

Using a gigaswitch the reported speeds go up to a maximum of 54/56 mB/sec.

So far my maximum transfer speed has not touched 17Mb.

Also the article above highlights issues with the transfer speed of the hard disk itself. From the article: "With a gigabit LAN, the biggest bottleneck becomes not the network, but the hard drives. A simple estimate of the upper limit of a gigabit LAN network is the maximum sustained transfer rate of the slowest hard drive. The ST340014A's used in our test machines start at approximately 47 MB/s in the beginning of the transfer and slows to approximately 33 MB/s transfer rate at the end."

That speed is closer to the speed I have experienced, but my speed is still half of the speed anandtech witnessed.

 

Hello all,

 

I am using DU Meter as well and when writing I see large spikes and then complete stops up and down over and over again.  Here is what my graph looks like:

 

 

Is that normal?  Also, running top shows that the CPU is not being used very much.  I would expect the parity calculations to really peg it when writing.

 

Thanks.

Nice graph.  Spikes, stop/up/down, are all symptoms of the fact that data is being written to the server faster than the server can write it to the disk.  What happens is eventually the server memory fills up and the writer is forced to stop transfers.  When the memory drains back down, the writer can start again.  The exact points at which these events happen are somewhat "elastic" because the amount of memory dedicated to write buffering varies a bit so that there is still memory available for read streams.

 

You will see similar results for any system where a producer is faster than it's consumer.

 

Here is a brief explanation why this happens.  Let's take an example where we are writing say a 1GB file on an Windows PC to an unRAID disk share via GigE.

 

- Let's say on the unRAID server side, instead of writing the data to disk, we just discard the data.  If we measured this rate we'd see around 40-50MB/sec or so, depending on how well the TCP/IP stacks were tuned.  This is real close to the raw rate of the hard drive on the Windows PC.  The bottleneck here might be the network, but of course we could never go faster than the source disk (which could never really be as fast as the raw disk because of the file system and where blocks are physically located on the disk).

 

- But 40-50 MB/sec is faster than can be written to an unRAID disk share because in order to maintain array parity, we must read each block from disk before we write it, and we must read corresponding block from parity in order to update it as well.  This results in an absolute maximum write performance to a single disk to be 1/2 the rate of the disk, e.g., for a typical 7200 rpm drive this amounts to around 30 MB/sec.  So already we are slower than the source.

 

- In reality we get substantially less than 50% rate of a single disk - more like 25-40%, depending on several factors.

 

But, the write performance is still faster than most media sources: tuner cards, DVD drives, CD drives, video cameras, etc.

 

Also, parity calculations using a 2Ghz processor are negligible.

 

-------

 

Here is why 1/2 speed is the best we can get.  We write data in "chunks" to the disk.  Let's say that the size of our chunk is 128KB, which is close to the amount of data on a single track on a typical disk.  Let's say we are going to write this data starting at sector N on the disk.  Let's say the initial disk seek operation to move the r/w head to the cylinder containing our starting sector has already occurred.  Let's say the disk has rotated so that at "time0" sector N is just under the r/w head.

 

OK, first we read 128K starting at sector N (this will be 256 sectors, BTW), and save this in a buffer.  Let's say it took exactly 1 disk revolution to to read the 128K. Now we need to wait for sector N to come back under the r/w head so we can start writing our data.  But by the time we issue the write command to the disk, and the disk starts executing the command, we "missed" sector N.  So what does the disk do?  It waits until sector N comes around again.  So now we have a full disk revolution time where no data transfer is taking place.  Finally sector N comes around and we write our data.

 

Now what happens next?  Well if this is a large sequential transfer we are going to want to write the next 128K which will start at sector N+256.  We need to first read this data so we send a read command to the disk.  But sector N+256 just flew by the r/w head before the disk got the read command, so guess what? The disk waits another full revolution before it can start reading.

 

So adding things up, it takes 4 disk revolutions to write the data content of 1 disk revolution - which means data is being transferred at 25% rate of the drive.

 

There are some things we can (and do) do to improve this.  But even if we didn't skip any revolutions at all, we'd still only get 50% rate of the drive (2 disk revolutions to write data content of 1 disk revolution - because we need to read first).

 

-------------

 

Lest you think raid5 is better - well it can be - UNLESS there is fragmentation of the target file system.  There are a couple reason raid-5 can be faster, but chief reason is that there's the possibility of filling up a full "stripe" of data across all the array drives, and thus avoiding the necessity of reading the old data first.  But if the raid-5 subsystem is unable to accumulate complete stripes, it will have to also do read-modify-write.

 

Raid-1 (mirroring) doesn't suffer any of this slowdown because there's no read-modify-write.

 

If you think SATA NCQ (command queuing) will help this - well probably not.  This is because you want the disk to be able to start reading the very next sector following one being written, as well as start writing the very next sector it just read.  I don't believe any disk is capable of that because it takes time to switch the r/w head from read mode to write and vice versa - more time than is available in the inter-sector-gap.

 

----

 

A better solution to this problem is to solve it at the system level, rather than via low-level disk operations.  We are working on code which will let you set up two drives in a raid-1 configuration & the rest of the drives in normal unRAID.  The way it works is that writes to user shares will go to the raid-1 volume which will be fast and still have redundancy in case a disk fails.  Then a background process will migrate data off the raid-1 volume onto the unRAID array.  In effect, this extends the size of the memory buffer by a huge amount.  One disadvantage of this is that more disks need to be spun up.  [Actually our algorithm is far more clever - but you get the idea.]

 

 

 

I have no idea but I can tell you it's crap......!  The reason is that the data is moving from and or thru a windows box onto the UnRaid.  Moving data from drive to drive within the UnRaid itself and NOT using a windows machine to do it but from a telnet session is very fast.  I think it took me over two weeks to move about 130 ripped and striped DVD's, yeah frustrating but it's a one time deal.

 

It's always been high on my list of enhancements to have the ability for the UnRaid software to see a new drive is NTFS (or whatever) and offer to copy the data to an UnRaid drive.  Not sure if Tom will get that done or not.

 

You forgot to mention this was over a 100Mbit/sec network.

Thanks Tom, I feel better now that I know no one can transfer the data any faster than I do....

;-)

More tests

- I have used Windows Explorer copy and paste functionality with a maximum transfer speed of 10 mB/sec as result

- I have tested xcopy on another PC with a 100MB network card, and the maximum speed I was able to obtain was 8 mB/sec (I had to reboot Unraid so parity check was ongoing but I stopped it to continue the transfer, and speeds did not improve)

Any other applications you suggest to test in order to see if we see improvements in transfer speed?

So far xcopy for me wins hands down.

More tests

- I have used Windows Explorer copy and paste functionality with a maximum transfer speed of 10 mB/sec as result

- I have tested xcopy on another PC with a 100MB network card, and the maximum speed I was able to obtain was 8 mB/sec (I had to reboot Unraid so parity check is ongoing wich could affect performance)

Any other applications you suggest to test in order to see if we see improvements in transfer speed?

So far xcopy for me wins hands down.

 

To make sure I'm clear on units...

 

- a lower case 'b' means 'bits'; upper case 'B' means 'bytes'.

- 'M' means 'mega', as in 1,000,000 [in windows M means 1,048,576 which is 2**20]

- 'G' means 'giga', as in 1,000,000,000 [in windows G means 1,073,741,824 which is 2**30]

- 'K' means 'kilo' as in 1024 - where this comes up is on the Main page of the Management Utility, where all sizes and transfer rates are given in units of 1024 byte blocks.

 

Two kinds of ethernet:

 

- 'Fast' ethernet: 100Mb/sec (100,000,000 bits per second)

- 'GigE': 1000Mb/sec, or 1Gb/sec (1,000,000,000 bits  per second)

 

The limit of Fast ethernet is 10MB/sec (because of encoding), but actually less (because of packet overhead).  Best I've seen with Fast Ethernet is around 9MB/sec.

 

The limit of GigE is 100MB/sec, but again less than that because of encoding and overhead.  At this speed however, other factors come into play: CPU & memory performance, protocol stack overhead, etc.  Best I've actually seen is around 55MB/sec, though googling you'll find folks claiming to reach 80MB/sec.

 

If I read your post correctly, 8MB/sec on a 100Mb/sec fast ethernet link is reasonable.

I am looking at DU Meter right now and the speed reported is 8.19 mB/sec, not MB/sec.

Also I must correct my previous post, as the "fast" ethernet connection I am using on the other PC is a 100 Mb/sec, and not MB/sec.

Why did they have to make networking speeds so confusing? I never get it right, but thanks for pointing that out Tom, I have added it to my favourites as it was a quite good and concise explanation.

I am looking at DU Meter right now and the speed reported is 8.19 mB/sec, not MB/sec.

...

 

They probably "mean" 8.19 mega-bytes/sec.

 

You want to know the most trustworthy, reliable, repeatable method of timing large data transfers?  A stopwatch and a calculator!  I'm NOT kidding, try it.

A better solution to this problem is to solve it at the system level, rather than via low-level disk operations.  We are working on code which will let you set up two drives in a raid-1 configuration & the rest of the drives in normal unRAID.  The way it works is that writes to user shares will go to the raid-1 volume which will be fast and still have redundancy in case a disk fails.  Then a background process will migrate data off the raid-1 volume onto the unRAID array.  In effect, this extends the size of the memory buffer by a huge amount.  One disadvantage of this is that more disks need to be spun up.  [Actually our algorithm is far more clever - but you get the idea.]

 

Tom,

 

Why not do it with an additional parity drive? that is reserve parts of two mirrored parity drives with a swap space? Kinda kill two birds with one stone... Alternatively, why not let us buy more ram? Use the extra gig or three as a ram drive? The speed of that "cache" would be phenomenal I'd think.

 

BTW, I mentioned NCQ in the other thread specifically to address a possible boost for some SQL DBs, which are in fact given to NCQ kind of  serialization (multiple instances of ODBC/Java access to the same disk); in the enterprise, as you probably know, this is especially used in SAN type configurations, fibre channel etc. Maybe for a transaction backend, NCQ+iSCSI may make sense, as opposed to traditional home theater applications. Lastly, the latest gen of the Maxtor pro drives show slight improvement in traditional apps (=non composite tests) in NCQ. hopefully, WD can catch up to Seagate/Maxtor in that department, since I buy only WD 

 

Oh, almost forgot, were going to get hybrid drives in the near future, wouldnt be surprised if some of the enterprise models get 128MB+ cache on them, if Samsung keeps pumpin those NANDs out (and apple doesnt buy all of them  ); meaning that NCQ could have a nice boost with the large cache, by then it may be NCQ II tho.

 

-Alex

 

Tom,

    Your explanation above is quite helpful to the semi-uninitiated amongst us.... not to pick nits (really!) but your explanation seems to tip the tilted eyebrow to the non-rounded multipliers like 1048576 and 1073741824 (attributing them to Windows jargon) in favor of the more human 1,000,000 and 1,000,000,000.  Actually, in computerese (aka Binary notation) a "Killo" is 1024 and a million is a Killo-Killos and so on... the rebirth of the nice rounded numbers like 1,000,000 came about when the Disk drive manufacturers found they could inflate their drive sizes this way

 

Jim

 

You have correctly discerned the direction of eyebrow tilt, at least as where it involved Windows.  Windows, being the operating system of the masses, should display values in the units understood by the masses.  JMHO - always been a pet peeve - how many times have you had to try and explain to a non-technical user why "mega" means 2**20 and not 10**6?  For me, at least a gazillion times, which is a lot no matter what units you use

NERD FIGHT!!

 

"NERD FIGHT !!"

 

NEVER !!!!!  I want to make sure that Tom doesn't put a skull-and-crossbones next to my registration info !!!