Write speeds?

[opentoe]

Anyone build a server with new hardware lately, using hard drives and no cache drive? I'm curious as to what top write speed you are getting on a large single file copy. Like an ISO file or something. Like what is your write speed at %90 of the file copy?

 

Been trying to help a friend remotely and he's telling me on how slow his write speeds are. I asked him to send me his logs, but haven't gotten them yet. I told him a system with all mechanical drives and no cache drive would start out about 100-120MB/sec and then drop down to about 40-60MB/sec.

 

I sent him the speeds I was getting, but if I show him more examples maybe he will relax.

 

 

[Frank1940]

I have watched this in the past and my experience is similar to yours.  The other thing to realize is that all unused RAM is used a write cache so the more memory the system has the more stuff that will be written at the 100+ Mbps speed that you see at the beginning.  When that cache is full then the speed drops back to the speed of the physical disk writes. 

[schford]

I haven't activated my parity drive yet and I am maxing out my Lan connection and getting pretty much 114MB consistently copying stuff over - and that so far is a couple TB into a 4 TB copy for this folder.

[garycase]

I wouldn't consider write speeds for a non-protected array relevant.

 

I agree that 40-45MB/s is pretty typical, although there's one interesting thing I've noted:

 

(a)  First, if you have a 2-drive array, the write process is different -- UnRAID recognizes that it's effectively a RAID-1 in that instance and writes are VERY fast, since it's only doing 2 disk I/O's instead of the normal 4.

 

(b)  Second, even if you have more than 2 drives, if you upgrade to a larger size parity drive and only change ONE of the array drives to the new size, you'll get the same performance bump if you're writing to the new drive at a point beyond all other drives in the array.    I recently upgraded parity and one data drive to 4TB on a server that otherwise had all 2 TB drives.    The 4TB data drive I added replaced a full 2TB drive; so when I wrote new data to it it was all beyond the 2TB point.  I copied over a TB of data to the drive ... and the entire write was done at about 65-70MB/s !!  Writing to another new 1TB/platter 2TB drive in the same system was at the usual "fast start ... then slowdown to ~ 45MB/s" speed.

 

[luvmich]

I get a constant 100Megs/sec when I turn on turbo writes.  Not the easiest the thing to enable, but if you login to a SSH or console session and type "mdcmd set md_write_method 1" it will turn on turbo writes.  Change the "1" to "0" or reboot to turn off turbo writes.  What turbo writes does is spin up all the disk to calculate parity during writes.  (Not really energy efficient)

 

Anyway, I only turn on turbo writes when transferring large amounts of data.  Normal writes only spin up the disk being wrote to and the parity disk.

[opentoe]

I get a constant 100Megs/sec when I turn on turbo writes.  Not the easiest the thing to enable, but if you login to a SSH or console session and type "mdcmd set md_write_method 1" it will turn on turbo writes.  Change the "1" to "0" or reboot to turn off turbo writes.  What turbo writes does is spin up all the disk to calculate parity during writes.  (Not really energy efficient)

 

Anyway, I only turn on turbo writes when transferring large amounts of data.  Normal writes only spin up the disk being wrote to and the parity disk.

 

Really? Never knew that was even an option. I keep my drives spun up 24/7 so if this really works would be pretty cool. I'll try it myself, then offer the suggestion to my friend too. I don't know if he keeps his running all the time though.

Thanks.

 

UPDATE: Does this affect read speeds when this mode is turned on? I remember this option from V5.03. I just turned it on with a large file and received 114MB/sec for the entire file copy. Pretty nice if moving large files. I wonder if this is something that can be added to the GO file upon bootup?

 

[StevenD]

 

. I wonder if this is something that can be added to the GO file upon bootup?

 

 

Sure. Thats how mine gets set since I never spin anything down.

 

[garycase]

UPDATE: Does this affect read speeds when this mode is turned on?

 

No, it doesn't have any impact on read speeds.

 

... and, as already noted, you can certainly add it to the GO file if you always want it enabled.

 

[opentoe]

UPDATE: Does this affect read speeds when this mode is turned on?

 

No, it doesn't have any impact on read speeds.

 

... and, as already noted, you can certainly add it to the GO file if you always want it enabled.

 

Cool. I did add it to my GO file and is nice seeing a constant 112MB/sec write speed. I do enough large file copies weekly for this to put a big smile on my face. Not sure why I didn't remember this one. Thanks for who ever reminded us earlier.

 

[opentoe]

Did something happen to the turbo write method? mdcmd set md_write_method 1

I remember when using v6.1.8 and prior I could get a sustained 112MB/sec write speed to the array with large files. I added the command to my GO file so I wouldn't need to strike it up via the terminal. I've noticed after I've upgraded to 6.1.9 turbo write speeds are gone. Large file copy starts as normal, then will drop, like a normal typical file copy without turbo mode.

 

Anyone else seeing this?

 

[JorgeB]

Turbo write works the same on v6.1.9, to add the command to the go file you have to insert a delay or check if the array is online before executing it, or it won't work.

 

On v6.2-beta there's a WebGUI setting to turn on/off.

[Frank1940]

Turbo write works the same on v6.1.9, to add the command to the go file you have to insert a delay or check if the array is online before executing it, or it won't work.

 

On v6.2-beta there's a WebGUI setting to turn on/off.

 

Is this done by changing the 'Tunable (md_write_method):' on the 'Settings' > 'Disk Settings' page to 'Reconstruct write'? 

 

If this isn't the proper setting, where is the setting? 

[JorgeB]

Is this done by changing the 'Tunable (md_write_method):' on the 'Settings' > 'Disk Settings' page to 'Reconstruct write'? 

 

If this isn't the proper setting, where is the setting?

 

Yes, reconstruct write=turbo write

[Frank1940]

I get a constant 100Megs/sec when I turn on turbo writes.  Not the easiest the thing to enable, but if you login to a SSH or console session and type "mdcmd set md_write_method 1" it will turn on turbo writes.  Change the "1" to "0" or reboot to turn off turbo writes.  What turbo writes does is spin up all the disk to calculate parity during writes.  (Not really energy efficient)

 

Anyway, I only turn on turbo writes when transferring large amounts of data.  Normal writes only spin up the disk being wrote to and the parity disk.

 

I am doing a study on the use of Turbo writes and I want to point out some other factors that have to be factored into this simplistic remark.  To get speeds close to what Turbo Write can deliver, you would have to install a cache disk.  Now you have the time that the cache disk has to spun up for the initial transfer of the data to the server.  Then at later time, the cache disk, the parity  and data disks have to be spun up for the transfer to the array.  The time for this transfer is roughly twice what it was the initial transfer.  Plus, you have to include the time to spin-down delays into the energy equation.  Plus, the data is unprotected until this second transfer can be made.  With Turbo Write, the data is protected as soon as the write operation is completed.  I have confirmed that all of the disks will spin down at the end of the spin down delay so that using using Turbo Cache will not require that the array disks being spun up all of the time. 

 

It appears from my study that the power usage of a modern HD is about 3.5W when spun up with no R/W activity and about 4W with R/W activity.  If you looking at it from an economical standpoint, how many hours of writing activity would you have to have to to be saving money to justify buying and installing a cache disk?  I have the feeling that folks with small number of data disks would be much better off with Turbo Write rather than installing a cache drive just to get higher write speeds.  For those folks using VM's and Dockers which are storing their configuration data, they could now opt for a small (and cheaper) SSD rather than a larger one with space for the cache operation. 

 

I can also tell you that the speed gains with Turbo Write is not as great with transfers involving large number of files and folders.  The overhead required on the server to create the required directories and allocate file space has a dramatic impact on performance!  The largest impact is on very large files and even then this impact can be diminish by installing large amounts of RAM because of unRAID usage of unused RAM to cache writes.    I suspect many users might be well served to install more RAM than any other single action to achieve faster write speeds! 

 

So things are not as simple as they first appear. 

[opentoe]

Turbo write works the same on v6.1.9, to add the command to the go file you have to insert a delay or check if the array is online before executing it, or it won't work.

 

On v6.2-beta there's a WebGUI setting to turn on/off.

 

Hey Johnnie- I've had the setting in my GO file for quite some time. I never spin down my drives, but I do not have the delay in my file. I'll try adding that or run the command manually to see if the speeds return.

 

Thanks for the suggestion.

[JorgeB]

I could never get the command to work on the go file without the wait for the array to be started first.

 

There's probably a command you can run to check if it's on or off but I never seen it and there's another easy way:

 

-clear read/write statistics

-copy a large file to a specific disk

 

-if there are approximately the same amount of read/writes in both the parity and destination disk and all other disks are at zero you're using normal write.

-if there are only writes on both the parity and target disk and reads in all others you're using turbo write

[Frank1940]

I could never get the command to work on the go file without the wait for the array to be started first.

 

There's probably a command you can run to check if it's on or off but I never seen it and there's another easy way:

 

-clear read/write statistics

-copy a large file to a specific disk

 

-if there are approximately the same amount of read/writes in both the parity and destination disk and all other disks are at zero you're using normal write.

-if there are only writes on both the parity and target disk and reads in all others you're using turbo write

 

An even quicker way if you have more than one data disk using basically your method.  Make sure all of the drives are spun down.  (If not, they can be spun down from the 'Main'-'Array' page.)  Then do your copy.  Now look at the Main page.  If all of of the disks are spun up, you are in Turbo mode.  If only the Parity and one of the data disks are spun up, you are in the Normal mode. 

 

It is really quite surprising that no one has written a plugin to change the write modes from the GUI for versions 6.1.9 and earlier... 

[RobJ]

It is really quite surprising that no one has written a plugin to change the write modes from the GUI for versions 6.1.9 and earlier...

So we're waiting for someone to give us a plugin that puts a Turbo button or slide on the Main page and the Dashboard, and clearly shows which mode it's currently in.

[Frank1940]

It is really quite surprising that no one has written a plugin to change the write modes from the GUI for versions 6.1.9 and earlier...

So we're waiting for someone to give us a plugin that puts a Turbo button or slide on the Main page and the Dashboard, and clearly shows which mode it's currently in.

 

That might be a bit of a issue/challenge!  What I was thinking of was a plugin with an icon on the 'Settings' page called (say) "Write Speed" and when click-on would give a user the ability to select Normal or Turbo mode of writing to the array.  Granted, its usefulness  would probably be short IF LT gets version 6.2 out of the beta and rc stages in a timely manner. 

[cyberstyx]

I haven't noticed any write speed differences due to a large RAM.

 

When I was initially building my system, I started with 1 Parity at 6TB and one data disk at 6TB. File transfers kept constant at around ~105MB/s or more (between 100 and 120MB/s actually).

 

When I added another 4TB data disk, write speed was greatly reduced to 40 ~ 60 MB/s.

 

I have an i7 with 24GB of RAM and it doesn't feel like it's doing anything great to write speeds. There may be a speed "bump" initially (maybe due to RAM caching as previously mentioned), but that bonus is lost after a few seconds.

 

But I didn't know Turbo Write was already available through a command, it was one of the excellent new features I was waiting for from 6.2. I 'll give it a go later on today as I have 4TBs of data pending to be migrated.

[Frank1940]

I haven't noticed any write speed differences due to a large RAM.

 

When I was initially building my system, I started with 1 Parity at 6TB and one data disk at 6TB. File transfers kept constant at around ~105MB/s or more (between 100 and 120MB/s actually).

 

When I added another 4TB data disk, write speed was greatly reduced to 40 ~ 60 MB/s.

 

I have an i7 with 24GB of RAM and it doesn't feel like it's doing anything great to write speeds. There may be a speed "bump" initially (maybe due to RAM caching as previously mentioned), but that bonus is lost after a few seconds.

 

But I didn't know Turbo Write was already available through a command, it was one of the excellent new features I was waiting for from 6.2. I 'll give it a go later on today as I have 4TBs of data pending to be migrated.

 

With only a parity disk and a single data disk, the default writing mode is the Turbo Write mode.  As soon as you install the second disk, it become normal default mode.  (As a second piece of information, parity and single data disk is effectively a RAID 1 array and the parity disk data is exactly what is on the data disk.)