Hi all me again trying to sort out some slow write speeds!

 

I've posted before about slow write speeds to the sever. Usually only get about a 20MB/sec write. My typical use case is that I download a large file to my server and want to make a copy of it in a different user share on my server. I've always just drag and dropped files between shares in windows explorer (SMB). After all these years I've learned that that is actually slowing things down because essentially I'm doing double the work, and Midnight Commander is the best way to move or copy files on my server.

 

So I used Putty to login and moved files between DISK shares using Midnight Commander and sure enough I STILL get 20MB/sec write speeds. This is the same if I "move" or "copy" the large file(s). Transferring many small files yields approximately the same write speed. As a data point, I can copy a large file from my unRaid server to a desktop PC with and saturate the gigabit ethernet speeds (~118MB/sec).  If I copy the same file back to the server, again I'm down to about 20MB/sec. 

 

From my limited knowledge and testing, it is not one particular drive that seems to be the culprit. I'm really looking to see if there are any other tools at my disposal to try to track down where this problem is coming from.

 

I've attached a syslog to this post that I captured after a fresh reboot. And here is some more information:

 

unRaid 6.1.9

Supermicro D525 (Atom) board with IPMI

Array of 5 data disks (mix of WD greens and reds)

PCIe SSD cache (confirmed that the cache is enabled for all shares, and that freshly copied files wind up on the cache)

Gigabit ethernet

syslog.txt

I likely have the same d525 board as you.    That atom is BARELY capable of running v6.    To test I moved the array to a low end i3 and speed skyrockets.  Moved back to d525 and molasses river.  Reverted to v5 and speed skyrockets.  Move back to v6 and again slow.    Even writes to a SSD cache under v6 on the atom are painfully slow.    Sticking with v6 as the added functionality outweighs the slower performance on the atom.

I had been toying around with the idea that the board was too slow, but so many other peopled used it (way back when) and had fine results. Makes sense with the v5/v6 switch! I built this back in the v5 days. Thanks for confirming... I wish I would have learned that about 2 years ago!

 

Either way, I've attached my diskspeed results showing how fast the read speeds are

diskspeed.pdf

Looks inline with what I get on the atom and v6...  Note that my writes to the cash from an external source are much lower than yours....

Looks inline with what I get on the atom and v6...  Note that my writes to the cash from an external source are much lower than yours....

 

The Diskspeed.pdf I attached is read speeds, not write speeds

Sorry,  reading via phone and opening paragraph was all about write speeds.  Also,  the PDF is a bit difficult to read on my device,  but it appears you have about 18 mb/s read....  Which is low....  And outside my realm of experience

Sorry,  reading via phone and opening paragraph was all about write speeds.  Also,  the PDF is a bit difficult to read on my device,  but it appears you have about 18 mb/s read....  Which is low....  And outside my realm of experience

 

I did a bad job of conveying information in a logical manner!

 

Read speeds are excellent! 90MB/s on green drives and 120MB/s on red drives. 400MB/s on SSD cache.

 

Write speeds are terrible. ~20MB/s no matter how I write; midnight commander, SMB, cache or no cache.

 

I'm fairly confident I've eliminated network issues because I can read over the network so fast and copying files via midnight commander is not affected by network speed. You had mentioned the D525 was giving you poor performance on v6 as well, but it sounds like your performance is both read and write.

Nope,  my read is good enough that I've not measured it.

My write is 20 to an array disc,  and at most 40 to a SSD cache....  But usually closer to 25-30.  Still,  I accept the slower write to this hardware server as an acceptable tradeoff for v6 features. 

If it is just write then try remove the parity disk and do the write again. If write speed is good then it's your parity disk.

 

Also, attach the full diagnostic zip. Syslog alone doesn't help much.

Nope,  my read is good enough that I've not measured it.

My write is 20 to an array disc,  and at most 40 to a SSD cache....  But usually closer to 25-30.  Still,  I accept the slower write to this hardware server as an acceptable tradeoff for v6 features.

 

So I'm definitely kicking a dead pig at this point, but the unRaid GUI only shows around 30% CPU utilization when writing/copying files to the array using Midnight Commander. If the CPU was the bottleneck, wouldn't this be 100%?

 

I'm pretty amateur at this stuff, so there may be something else going on besides "utilization" that may be causing the bottleneck. Just trying to learn

 

If it is just write then try remove the parity disk and do the write again. If write speed is good then it's your parity disk.

 

Also, attach the full diagnostic zip. Syslog alone doesn't help much.

 

testdasi, thanks for chiming in! How do I remove the parity to test this? Also, please see attached for the full diagnostic report. The last hour or so I've been copying large video files from on disk to another using Midnight Commander.

tower-diagnostics-20160813-1101.zip

Bump.

 

Per my previous post, how do I remove the parity to check write speeds without it?

 

Also, I attached a full diagnostic report in my previous post.

 

Thanks again for all the help!

• Stop array.
  
• Change parity to no device.
  
• Start array.
  
• Experiment like he said.
  
• Stop array.
  
• Add parity drive back and rebuild parity.
  
• Check parity once the rebuild is complete.
  

Think that will do it.

 

 

The other option is to do a "new config" and then just assign the data drives back to their respective slots but don't add the parity drive.