Hi all,

 

I am new with unRAID and last week I built my first unRAID server. I used the following components ("The Beast" from recommended builds):

 

• MB: Supermicro X8SIL-F
  
• CPU: Intel i3-550
  
• RAM: 2 x Kingston KVR1333D3S8E9S/2G (2GB)
  
• PSU: Corsair Enthusiast Series TX750 V2
  
• Parity: WD Green WD30EZRX 3TB 5400rpm
  
• Data: WD Green WD25EZRX 2.5TB 5400rpm
  
• Flash: Sandisk Cruzer Fit 4GB USB-Stick
  

Using unRAID 5.0 Beta 14 Basic.

 

The system is working, but the write-performance to the server is bad. In average I am getting 10.5Mb/s when writing to the server. Reading gives me an average of 90-100Mb/s, so I think it is not a problem with the LAN or something.

 

I tried writing with Windows 7 on SMB and also tried writing with Ubuntu on NFS, both gave me the 10.5Mb/s average.

 

The thing what's seeming odd to me is that the activity-indicator-LEDs on the drive bay show activity for 3-4 seconds, then there is no writing or reading activity for 3-4 seconds, and then the writing continues. So there are time intervals where both harddisks do nothing.

 

I already read the guide to increase performance, but that didn't help. And of course I know that it is normal that writing-performance is significantly slower than the reading performance, because every write results in several reads etc. But shouldn't it give me at least 20 or 25Mb/s when writing?

 

Parity Check Speed begins at 130 Mb/s and drops slowly (end of the disk) to about 80-90Mb/s.

 

Has anybody an idea why I am getting those slow writing-speeds?

 

Best regards,

Pascal

what is the LAN speed of the PC you are copying from?  What is the LAN speed of your router?

If you type:

ethtool eth0

what do you see?

 

A 100MB/s lan connection would result in exactly the speeds you are describing.

It is possible that the cabling is bad in one direction I suppose.

 

It is normal for parity check to be much faster on outer cylinders of the disk than inner ones.  (data density is much higher on outer tracks)

 

I would agree that you should be getting write speeds of 25MB/s or higher.  The last time I had 10MB/s speeds was when I had an IDE disk in my array, perhaps one of your drives is configured as legacy IDE mode in the BIOS instead of full SATA (AHCI) mode?

 

Networking (as already suggested) is often the culprit.  You could rule this out by doing a file copy from one place to another on your unRAID box through the console interface so the data is not transferred over the LAN and timing the copy by hand.

 

Regards,

 

Stephen

 

Hi,

 

it is a gigabit network.

ethtool eth0 gives me:

 

Supported link modes: 10, 100 and 1000base/full

Speed: 1000Mb/s

Duplex: Full

Port: Twisted Pair

 

None of the drives run in IDE mode, all in AHCI.

 

I tried several things now:

1. Updated BIOS to newest version

2. Used other S-ATA cables

3. Used other LAN cables

4. Used other switch

5. Used second NIC on the board

 

The only thing that changed something was option 5. With the second NIC on the board I get a better performance now (about 25Mb/s). BUT at the moment (the only data drive is quite full now, 200Gb free), I am getting always an error when copying new files to the server: "There was a problem with the network resource." Any idea about that?

 

I also installed Windows on a SSD I got lying around and did some tests there: Write and read speed via SMB in average 80Mb/s and 90Mb/s. So this seems to be an unRAID problem?

 

Thank you for your help and best regards,

Pascal

Post a syslog.

If you are getting 25MB/s performance you are probably at the limit of your disk rotational speed.  In other words, so not try to find another bottleneck.  The 5400 RPM disk speed is the bottleneck.

 

You have 5400 RPM disks.  They will get 2/3rds of the performance of those users with 7200 RPM disks.  If you had 7200 RPM disks you would probably be getting 33.3 MB/s, and that is about as good as it gets for sustained writes to a parity protected drive.  Remember the parity and data disk must spin around twice for every sector written (once to read it, a second time to write it)  The faster the two disks spin, the faster you can write to the array.  You are limited by the slower of the rotational speed of the two disks involved.

 

Joe L. 

That seems a little low, but not that far off the mark. It's been some time since I benchmarked my server, but earlier on I was getting somewhere in the 30-35 MB/s write speeds once memory buffering is out of the way with my WD 5400rpm drives.

Remember the parity and data disk must spin around twice for every sector written (once to read it, a second time to write it)

 

I keep seeing this and mean to address it.  This is not true in practice with track buffering when writing to files spanning multiple consecutive sectors.

 

Think about it... if it were true, you could only write 5400/2 sectors per second!

 

When you read the first sector, the rest of the track is read into the track buffer.  All in 1 revolution.  You will write back (to buffer) before the first revolution is completed.  You will read the second sector (from buffer) and write it back (to buffer) too in the first revolution, etc.  In practice, it should only take 3 revolutions to process all the sectors in a single track (or cylinder with full cylinder buffering). Modern drive algorithms take into account the time needed to flip a head from read to write and elevator sorting with NCQ can be very efficient and it can take only 2 plus a fraction revolutions (the fraction is necessary for the head to flip from read to write).

 

In any event, the approximate theoretical maximum (tMax) speed for parity-protected writes is still:

 

    tMax =  (drive read rate + drive write rate) / 4

 

Also, a 5400 RPM drive does not give you 2/3 of the performance of a 7200 RPM drive in unRAID, because the SPT and SPC are different.  It may work out that way for various reasons, but what it comes down to is total number of bytes passing under the heads per second.  That involves number of surfaces, areal density, and rotation rate... not just rotation rate.

Hi,

 

thank you for all your answers :-)

 

Sorry for not being clear, 25MB/s is the speed that I was counting with and which is absolutely ok for me. The other thing that I mentioned yesterday is the thing that I am worring about a bit now. It's exactly the same behavour as described here: http://lime-technology.com/forum/index.php?topic=18747 and here: http://lime-technology.com/forum/index.php?topic=17568

 

But unfortunately for me the solution is not the split level (as is it in the second linked thread), because I set it to 0 and I've got only one data disc which is signifcantly smaller than the parity disk.

The first thread linked is descriping my problem from yesterday exactly, but there is no solution :-(

 

Today I observed a perfect behavior without changing anything on the system. Right now I am copying 336Gb to the server with an average speed of 30Mb/s. And I tried several times to provoke the "Network resource is not available any more" error, but everything was going fine.

Edit: Ok, the problem exists again. In the middle of copying it suddenly stopped, saying "A problem has occured when trying to access the network drive. I think that this happens only when the next file is going to be copied. If one file starts copying, it copies completely and successfully.

 

I attached a syslog of today where everything is going fine. If it falls back to a bad mode where performance is low or the error appears again, I will post another syslog and we can compare them.

 

What I noticed is that I have to use the second NIC. Even if both NICs are the same chip (INTEL), the first one will not work properly (10Mb/s and getting errors).

Unfortunately the mainboard is not able to disable only one NIC (can only disable both or none), I always have to manually disalbe the first NIC and enable the second one (using ifconfig). Any suggestion how this can be automated?

Edit: Copied the ifconfig commands to the go script and now it seems that unraid automatically chooses the second NIC.

 

Thank you again for your great support!

 

Best regards,

Pascal

syslog-fine.txt

Remember the parity and data disk must spin around twice for every sector written (once to read it, a second time to write it)

I keep seeing this and mean to address it.  This is not true in practice with track buffering when writing to files spanning multiple consecutive sectors.

That's a very important correction/clarification.

...  In practice, it should only take 3 revolutions to process all the sectors in a single track (or cylinder with full cylinder buffering).

While there is one head per surface, only one of those heads can be active (ie, engaged in reading or writing) [at a time]. (Notwithstanding esoteric/experimental/extravagant drive examples.)

In any event, the theoretical maximum (tMax) speed for parity-protected writes is still:

 

    tMax =  (drive read rate + drive write rate) / 2

 

This is not a quibble, but a (hopefully thought-provoking) observation:

 

A reasonable, and practical, maximum speed for (unRAID)parity-protected writes is:

 

  pMax =  min(drive read rates , drive write rate)

 

[ Only hint: "There's more than one way to skin a cat." ]

 

(The fact that unRAID does not provide for this is one, of several, reasons I am not a user.)

 

... but what it comes down to is total number of bytes passing under the heads per second.

heads => head . That might be a typo, but it does jibe with the cylinder issue above.

 

-- UhClem

 

Well to be exact the actual (not approximate as used above) value is:

 

  1 / (1/read-speed + 1/write-speed)

 

But the approximation (2 posts back) is fine when the read and write speeds are close to each other.

 

Consider a drive that reads at 100MB/sec and writes at 75MB/sec.  So theoretically it takes 10 seconds of reading and 13.3 seconds of writing for a total of 23.3 seconds to write 1GB to unRAID.  That means 43MB/sec of maximum theoretical throughput (i.e. ignoring latency).  Using this formula, and I/O specs for your drives, you can easily calculate the maximum theoretical throughput for your system.

 

The importance of tMax is that it lets you know how close you are to perfection, and informs you of whether increased performance is likely with the existing drives, or whether you need to go to faster drives.

 

unRAID write performance should really be reported as percent of tMax.  This is kind of like people who report their CPU temps with various heat sinks.  Actual temperature doesn't help you compare yours to someone else's.  The better measure is delta-T above ambient.

 

So I can get 30MB/sec and you can get 20MB/sec.  What does that tell you?

 

Now if I say I add that my 30MB/sec is 60% of of my tMax, and you report your 20MB/sec is 80% of your tMax, does that give you more useful information?  I think it does.

I appreciate your mathematical rigor, but I didn't have any issue with your initial formula (remember, I wrote "This is not a quibble"). In practical fact, both it and your revised formula are equivalent, in the context of this discussion. From the perspective of a low-level kernel driver, write speed equals read speed (for a given range of LBAs on any specific drive). (And, those speeds are predetermined by the factory "formatting" of that specific drive.)

 

When I used the word Sub-optimal in my Subject: line, I meant it -- and, not related to the OP's query, but regarding unRAID itself. [Plug your numbers into my formula, and realize that is a Practical maximum (not theoretical) for a proper/enlightened implementation.]

 

Just trying to provoke some thought, and maybe get someone's light-bulb to flash. (But, please don't ask me [or try to goad me] to throw the switch.)

 

-- UhClem