February 19, 201511 yr Write speeds of that magnitude are achievable in several ways ... => Using a cache drive => If there's no parity drive ... this should write at the target disks speed (unless limited by the network) => Writes that are fully buffered by UnRAID (i.e. smaller than the available RAM buffer space) => Writes to an array with only one data drive ... this will be written in a different way, which does not require the usual 4 disk I/O's (UnRAID will recognize that it's effectively a RAID-1 and will write it directly to the target disks without the need to pre-read the sector contents from the two involved disks. => "Turbo Write" mode is enabled ... i.e. all disks are spinning -- this will also allow writes at near disk speed, since there are no pre-reads of the target disk or parity disk. It will require one read from all disks except the target disk; and a single write to the target disk and parity disk. So only one I/O per disk in the array ... which is notably quicker than two I/O's to each of the two involved disks. But NORMAL UnRAID operation cannot achieve writes with speeds that high ... it's simply physically impossible with current drive speeds (unless, of course, you build an array of SSD drives).
February 19, 201511 yr Ah, so having one data drive and one parity drive is effectively RAID1? I was only using the two MC04's, the HGST drives were never added to the array.
February 19, 201511 yr Ah, so having one data drive and one parity drive is effectively RAID1? I was only using the two MC04's, the HGST drives were never added to the array. correct, and re: jphipps' post don't conflate single drive write speed with array speeds. No one said it wasn't possible to observe full-speed dd based speed tests to a single drive. You just can't can't do that when writing to the parity+data-disks array; with the exception of the "raid1" special case. Even turbo write will be slower than bare-metal single drive speeds because the write still requires more than just a single write operation on a single disk and with enough drives it is easy to saturate your bus. But you wouldn't know until you tested it.
February 19, 201511 yr Author Well that only took a page of back and forth to solve that mystery. Still i thought "turbo" mode was frowned appon since it required all the disks spun up, and it got less and less effective with each disk, i think the break even point was 5 disks.
February 19, 201511 yr "frowned upon" is a strong way to put it. It has its pros, cons, and limitations. I personally certainly wouldn't leave it on all the time because 1) I am using a cache drive and 2) my writes are sparse and not usually speed hungry. but there are times when I'm doing a lot of moving / organizing that I've decided to turn it on. But there are others on here who use it in a more production use case, who also might not be using a cache (since data isn't protected on cache) who I know actually schedule the setting of turbo-write based on times of the day given their need. There is nothing inherently Bad about having all the drives spun up when you are actively using the array, or even when you're not except for power consumption. As to the limitations ... I don't know that 5 has been settled with modern hardware, but maybe I missed some more recent testing. Either way, that means it IS useful when used on a smaller array
February 19, 201511 yr Forgot to mention, I do have turbo write turned on. That test was using a server with 18 data drives and parity with no cache drive in use.
February 19, 201511 yr Just ran another test without turbo write on: Jeffs-Mac-mini-2:~ jphipps$ dd if=/dev/zero of=/Volumes/Movies/test.out bs=4096 count=6000000 6000000+0 records in 6000000+0 records out 24576000000 bytes transferred in 413.319705 secs (59460025 bytes/sec) Only about 60MB/s.. I usually leae it turned on because I havent' really seen any downside to leaving it on..
February 19, 201511 yr All XFS data drives here (although I don't think it matters). The real key is lots of ECC RAM and x64. Both tests were conducted on WIN 8.1 desktop (gigabit). Reading from parity protected data drive Writing to parity protected data drive Love the fact that I can actually write to the array faster than I can read from it. John
February 19, 201511 yr ... Love the fact that I can actually write to the array faster than I can read from it. Does look nice ... but as you know, it's not REALLY writing that fast -- it's simply buffering the writes in your "... lots of ECC RAM." All XFS data drives here (although I don't think it matters). Agree that it doesn't matter -- especially since the writes are buffered. The actual writes likely also don't matter except when a disk is very full ... in that case RFS is notably slower than XFS.
February 19, 201511 yr As to the limitations ... I don't know that 5 has been settled with modern hardware, but maybe I missed some more recent testing. Either way, that means it IS useful when used on a smaller array As long as your controllers aren't causing bus limits (i.e. if you're using port expanders) there shouldn't really be any bus limits with modern systems and "only" 25 drives max. The performance tests jphipps ran clearly show that it's still helpful with 18 drives ==> 99Mb/s with turbo write; 60Mb/s without for a 24GB write. Both of those numbers are going to be a bit higher than the actual write speed (due to the cached writes that UnRAID hasn't completed when the transfer is "done" as far as the client knows), but it clearly shows that turbo write provides higher performance. ... I usually leae it turned on because I havent' really seen any downside to leaving it on.. I don't think there's any downside either. As I understand it, even if it's turned on, it's only used if all the drives are spinning. At least that was the way Tom had once indicated he was going to implement it. jphipps => can you confirm if that's the case? [very simple test of course: Just spin down all drives; then do a write directly to a disk share ... and see if only that disk & parity spun up or if all drives spun up.]
February 19, 201511 yr I've done that test, I can tell you that as implemented in b12 it spins up all the drives. I do remember tom's idea, liked it even as an option, but as best I can tell there has not been any development on the turbo-write front.
February 19, 201511 yr That's too bad -- for some reason I thought the auto mode had already been implemented. Simple enough to enable/disable ... but it would have been nicer if a simple click on "Spin Up" was all that was needed to enable turbo writes ... and they were automatically disabled again once the drives spun down. I suspect for most of us the standard write method is the better choice (and certainly if you have a cache drive) => but it really depends on just how you use your array.
February 19, 201511 yr I must admit that I fill my unRAID array first, then add the parity drive afterwards. Means quicker population from wherever the data started off.
February 19, 201511 yr Yeah, I have done the same test as jumperalex and is powering up all the drives. I have also monitored iostat while doing my writes and it is doing equal IO across all the drives. I found the spin up is not too bad because read a lot more than doing writes, so the drives stay spun down a lot of the time. I have also found a big killer in performance is the 2TB and smaller drives. My other server has a mix of 2 and 4 TB drives, and a parity check is dramatically faster once it passes the 2TB mark...
February 20, 201511 yr I must admit that I fill my unRAID array first, then add the parity drive afterwards. Means quicker population from wherever the data started off. Not at all surprising, given the write speeds you noted.
February 20, 201511 yr ... I found the spin up is not too bad because read a lot more than doing writes, so the drives stay spun down a lot of the time. Indeed, most UnRAID users do far more reads than writes, as perhaps the most common usage is as a media server. Whether or not it's worth spinning up all the drives for a write depends a lot on your typical write scenario ==> if you do your writes in "batches", where you're doing a fair amount of writing, then clearly it's worth it for the speed gain. If you only write a few files at once, I'd think the slightly slower writes isn't a big deal, and personally I'd rather not spin up all the other drives. Just a matter of personal preference. I have also found a big killer in performance is the 2TB and smaller drives. My other server has a mix of 2 and 4 TB drives, and a parity check is dramatically faster once it passes the 2TB mark... Technically it's not the SIZE of the drives that matters ... it's the performance, which is driven by two fundamental characteristics: areal density and rotation rate. Most of us have older 2TB drives that have areal densities in the 500 - 667 Gb/platter range, so there's a BIG jump in sustained transfer speed once those are out of the mix in a parity check and all the other drives are 3-4 TB drives with 1TB/platter areal density. The difference wouldn't be nearly as dramatic if you were using 2TB drives with 1TB platters -- such as WD Reds. There would, however, still be a slowdown as the parity check approached the 2TB point ... in this case due to the much slower transfers from the innermost cylinders. This is true for all drives -- so if you had a mix of 1TB, 2TB, 3TB, and 4TB drives that all had 1TB platters (e.g. WD Reds) the check would start off very fast, and would slow down as it approached 1TB, then speed up again, then slow down as it approach 2TB, then speed up again, etc.
February 20, 201511 yr I must admit that I fill my unRAID array first, then add the parity drive afterwards. Means quicker population from wherever the data started off. Not at all surprising, given the write speeds you noted. Yeah, but I still had a parity drive this time for testing. Although, as was said, it's only one data drive and one parity drive which enhances performance.
Archived
This topic is now archived and is closed to further replies.