I've got two UnRAID servers (main server and off-site backup), both of which have 6 x 3TB WD Reds in their arrays not including cache (I also have 2 x 3TB Hitachi Desktar 7k3000s laying around as cold spares).

 

I'm looking to upgrade the parity drives this summer to gain another 3TB in storage space and I'm wondering what would be a recommended drive size/brand/model for me to go with.  Don't want to spend more than $400ish for the two drives.

 

Recommendations?

I assume from your question that both systems are limited to 6 drives, so you can't simply add another 3TB drive to each to gain the additional 3TB.

 

Given that, I'd spring for a significant upgrade in the parity size, so you can use larger drives.

 

If you really want to limit the total cost to $400, you could go with a pair of 5TB drives [e.g. 5TB WD Reds are often on sale for < $200;  or you could buy 5TB WD50EZRX units even without a sale]  ... but I'd be more inclined to go with either 6TB drives [e.g. 6TB WD Reds]  or  8TB  [the Seagate Archive SMR units].    Both of the latter would break your $400 budget, however.

 

I presume you know that upgrading the parity drive will not add ANY storage to the array => it will simply allow you to upgrade the array drives to the new size.

 

I assume from your question that both systems are limited to 6 drives, so you can't simply add another 3TB drive to each to gain the additional 3TB.

 

Given that, I'd spring for a significant upgrade in the parity size, so you can use larger drives.

 

If you really want to limit the total cost to $400, you could go with a pair of 5TB drives [e.g. 5TB WD Reds are often on sale for < $200;  or you could buy 5TB WD50EZRX units even without a sale]  ... but I'd be more inclined to go with either 6TB drives [e.g. 6TB WD Reds]  or  8TB  [the Seagate Archive SMR units].    Both of the latter would break your $400 budget, however.

 

I presume you know that upgrading the parity drive will not add ANY storage to the array => it will simply allow you to upgrade the array drives to the new size.

 

I actually have room for 8 drives per server including parity (cache is separate because I have 9 bays in my main server and on my backup server which is 8 bays won't have a cache drive).  That's why I was saying that upgrading the parity drives would add another 3TB of storage space per server.  So basically I have two free drive bays per server.

I actually have room for 8 drives per server including parity (cache is separate because I have 9 bays in my main server and on my backup server which is 8 bays won't have a cache drive).  That's why I was saying that upgrading the parity drives would add another 3TB of storage space per server.

 

Of course -- I hadn't thought about simply moving the old parity drive to the array 

I was thinking that if you just needed to add 3TB to the array, you'd buy another 3TB drive -- and since you weren't doing that, you must be maxed out on drive space or SATA ports.

 

... but I suppose the reality is I just wasn't thinking    [Call it a Senior moment  :) ]

 

 

By the way, a few weeks ago I'd have NEVER suggested using one of the Seagate SMR drives for parity due to the architectural limitations of shingled drives.    But danioj did some extensive testing of these drives in an UnRAID array [3 of the drives with one as parity] and the mitigations Seagate has embedded in the drives work VERY well for typical UnRAID uses.    You can read the details in the last 3-4 pages of this thread (starting ~ post #360):  http://lime-technology.com/forum/index.php?topic=36749.msg365314#msg365314

 

I'm impressed enough that my next server will use these drives (either the 8TB units or the forthcoming 10TB units, depending on just when I build it).

 

I actually have room for 8 drives per server including parity (cache is separate because I have 9 bays in my main server and on my backup server which is 8 bays won't have a cache drive).  That's why I was saying that upgrading the parity drives would add another 3TB of storage space per server.

 

Of course -- I hadn't thought about simply moving the old parity drive to the array 

I was thinking that if you just needed to add 3TB to the array, you'd buy another 3TB drive -- and since you weren't doing that, you must be maxed out on drive space or SATA ports.

 

... but I suppose the reality is I just wasn't thinking    [Call it a Senior moment  :) ]

 

 

By the way, a few weeks ago I'd have NEVER suggested using one of the Seagate SMR drives for parity due to the architectural limitations of shingled drives.    But danioj did some extensive testing of these drives in an UnRAID array [3 of the drives with one as parity] and the mitigations Seagate has embedded in the drives work VERY well for typical UnRAID uses.    You can read the details in the last 3-4 pages of this thread (starting ~ post #360):  http://lime-technology.com/forum/index.php?topic=36749.msg365314#msg365314

 

I'm impressed enough that my next server will use these drives (either the 8TB units or the forthcoming 10TB units, depending on just when I build it).

 

I was just reading that thread this morning.  I just wasn't sure if it was exhaustively tested enough to trust it as a parity drive in an UnRAID server..  But I might be able to swing $500 for 8TB parity drives if they are reliable.

Reliability is another issue => we simply don't have enough data to really evaluate that.  I'm also waiting to see if the 2nd test that Daniel is doing (copying a bunch of smaller files) has results as good as the first set of large files.    If so, I'll be happy enough to use the drives in my next server as long as there's not a bunch of negative reliability data in the next few months (I'll probably build a new box at the end of this year).

 

As I assume you gleaned from my comments in that thread, as long as your usage of UnRAID is such that you generally copy large files -- or if smaller files never more than 25GB at once -- then the migitations Seagate's built into the drives [the 25GB persistent cache and firmware that recognizes full band writes so it can eliminate the need for band rewrites in those cases] work very well, and let the drive perform essentially as well as a standard PMR drive.    Of course if you happen to have a use case that results in "hitting the wall" of write performance, the write speeds will drop to abysmal levels (~ 5MB/s or even worse for a few minutes) while the drive empties its persistent cache and/or does all of the pending band rewrites.

 

For a typical media server, however, it's likely that you'll do a BIG initial write of your data, which will all be sequential so it'll perform just fine;  and subsequent writes are likely to never exceed the capacity of the persistent cache unless they're very large files, in which case they'll avoid the band rewrite problem.

 

If you're concerned about the potential shingled drive issues, just go with 6TB units 

Reliability is another issue => we simply don't have enough data to really evaluate that.  I'm also waiting to see if the 2nd test that Daniel is doing (copying a bunch of smaller files) has results as good as the first set of large files.    If so, I'll be happy enough to use the drives in my next server as long as there's not a bunch of negative reliability data in the next few months (I'll probably build a new box at the end of this year).

 

As I assume you gleaned from my comments in that thread, as long as your usage of UnRAID is such that you generally copy large files -- or if smaller files never more than 25GB at once -- then the migitations Seagate's built into the drives [the 25GB persistent cache and firmware that recognizes full band writes so it can eliminate the need for band rewrites in those cases] work very well, and let the drive perform essentially as well as a standard PMR drive.    Of course if you happen to have a use case that results in "hitting the wall" of write performance, the write speeds will drop to abysmal levels (~ 5MB/s or even worse for a few minutes) while the drive empties its persistent cache and/or does all of the pending band rewrites.

 

For a typical media server, however, it's likely that you'll do a BIG initial write of your data, which will all be sequential so it'll perform just fine;  and subsequent writes are likely to never exceed the capacity of the persistent cache unless they're very large files, in which case they'll avoid the band rewrite problem.

 

If you're concerned about the potential shingled drive issues, just go with 6TB units 

 

Most of my writes to my array are files between 500GB and 4GB.  However, I'm a little confused about the persistent cache of 25GB.  How is that not an issue on the initial copy of data between my main server and backup server?

Most of my writes to my array are files between 500GB and 4GB.  However, I'm a little confused about the persistent cache of 25GB.  How is that not an issue on the initial copy of data between my main server and backup server?

 

Realize that most of the "internals" of the shingled drives are closely held by Seagate, so what we know is based on a lot of testing my 3rd parties and the performance we've seen in various testing.  But I think the following is pretty accurate ...

 

=>  The drives have a memory cache large enough to hold more than a zone's worth of data, so if the firmware recognizes that you're writing enough data for a complete shingled zone, it will simply write the entire zone and avoid the need for a zone re-write that would be necessary if you'd only written a single sector (or fewer sectors than the zone contains).    So if you're writing a lot of sequential data -- regardless of the file sizes -- there will be very few zone rewrites (a major performance killer with these drives).

 

=>  If you write a single sector in a zone, the drive will instead write it to the "persistent cache" ... a non-shingled area on the drive that effectively "buffers" all sectors that would require zone rewrites ... these are later written to the correct sectors -- and the corresponding zone rewrites done then -- at a time when the drive is otherwise not busy.    What I call "hitting the wall" of performance is when the persistent cache gets full and you're still doing a bunch of random writes that don't avoid the zone rewrite requirement => this can cause a HUGE drop in performance.

 

So ... a few observations (supported by the testing Daniel did in the other thread I referenced) ...

 

=>  If you're writing a large amount of sequential data, you'll end up with very little use of the persistent cache, since the drives will recognize that you're writing all of the sectors in each of the shingled zones.  There may be a few cases where this isn't true - but those will be written to the persistent cache, and it's unlikely you'll ever fill it.

 

=>  If you're later writing a few random files, these will be written to the persistent cache, and as long as it isn't filled performance will remain very good.  And if some of those files are large (multiple GB), they'll simply be written directly to any zones where they fill the entire zone ... so the persistent cache is even less likely to get filled.

 

The initial copy of all of the data on the main server to the backup server will be almost entirely sequential, so there will be very little use of the persistent cache, since all of the zones will be completely written.  In this case, the drive will act very much like a standard PMR drive.    The performance Daniel saw in copying his initial 5TB share reflects the accuracy of this assumption (averaged 38MB/s for the entire copy).    I'd like to see his confirmation that the next set of data (another 5TB or so, but with files that were significantly smaller) continues this level of performance -- hopefully he'll post the results of that soon.  I'm fairly confident they will, but it's always nice to see confirmation [Note that since they'll be located in sectors not quite as far out on the disk as the initial data the rate may be somewhat slower, but should still be in the same ballpark (certainly > 30MB/s].

 

 

Most of my writes to my array are files between 500GB and 4GB.  However, I'm a little confused about the persistent cache of 25GB.  How is that not an issue on the initial copy of data between my main server and backup server?

 

Realize that most of the "internals" of the shingled drives are closely held by Seagate, so what we know is based on a lot of testing my 3rd parties and the performance we've seen in various testing.  But I think the following is pretty accurate ...

 

=>  The drives have a memory cache large enough to hold more than a zone's worth of data, so if the firmware recognizes that you're writing enough data for a complete shingled zone, it will simply write the entire zone and avoid the need for a zone re-write that would be necessary if you'd only written a single sector (or fewer sectors than the zone contains).    So if you're writing a lot of sequential data -- regardless of the file sizes -- there will be very few zone rewrites (a major performance killer with these drives).

 

=>  If you write a single sector in a zone, the drive will instead write it to the "persistent cache" ... a non-shingled area on the drive that effectively "buffers" all sectors that would require zone rewrites ... these are later written to the correct sectors -- and the corresponding zone rewrites done then -- at a time when the drive is otherwise not busy.    What I call "hitting the wall" of performance is when the persistent cache gets full and you're still doing a bunch of random writes that don't avoid the zone rewrite requirement => this can cause a HUGE drop in performance.

 

So ... a few observations (supported by the testing Daniel did in the other thread I referenced) ...

 

=>  If you're writing a large amount of sequential data, you'll end up with very little use of the persistent cache, since the drives will recognize that you're writing all of the sectors in each of the shingled zones.  There may be a few cases where this isn't true - but those will be written to the persistent cache, and it's unlikely you'll ever fill it.

 

=>  If you're later writing a few random files, these will be written to the persistent cache, and as long as it isn't filled performance will remain very good.  And if some of those files are large (multiple GB), they'll simply be written directly to any zones where they fill the entire zone ... so the persistent cache is even less likely to get filled.

 

The initial copy of all of the data on the main server to the backup server will be almost entirely sequential, so there will be very little use of the persistent cache, since all of the zones will be completely written.  In this case, the drive will act very much like a standard PMR drive.    The performance Daniel saw in copying his initial 5TB share reflects the accuracy of this assumption (averaged 38MB/s for the entire copy).    I'd like to see his confirmation that the next set of data (another 5TB or so, but with files that were significantly smaller) continues this level of performance -- hopefully he'll post the results of that soon.  I'm fairly confident they will, but it's always nice to see confirmation [Note that since they'll be located in sectors not quite as far out on the disk as the initial data the rate may be somewhat slower, but should still be in the same ballpark (certainly > 30MB/s].

 

 

That was a terrific explanation, thank you for that.  Since I probably won't be upgrading my parity for another 2-3 months, we should have some more data by then.  If all I'm doing is upgrading my parity drive (meaning no copying data just rebuilding parity), I should be good on that front performance wise as well correct?

... If all I'm doing is upgrading my parity drive (meaning no copying data just rebuilding parity), I should be good on that front performance wise as well correct?

 

Absolutely.  A parity sync is an entirely sequential write of the entire drive, so there should be NO zone rewrites and, for that matter, no use at all of the persistent cache.

 

The performance Daniel saw in copying his initial 5TB share reflects the accuracy of this assumption (averaged 38MB/s for the entire copy).    I'd like to see his confirmation that the next set of data (another 5TB or so, but with files that were significantly smaller) continues this level of performance -- hopefully he'll post the results of that soon.  I'm fairly confident they will, but it's always nice to see confirmation [Note that since they'll be located in sectors not quite as far out on the disk as the initial data the rate may be somewhat slower, but should still be in the same ballpark (certainly > 30MB/s].

 

I have now posted. 80% in at the time of this post and I am experiencing the same results as I did with the write to the Array of larger files (~40MB/s  ) Sorry - have been interstate working! 

 

See the thread for the actual data I posted:

 

http://lime-technology.com/forum/index.php?topic=36749.400

 

I've also got a pair of these drives running in UnRAID and I've had no problems at all so far.  Speed is just fine.

I started a "sticky" to try and collate most of the "lessons learned" about these shingled technology units.    I'll fill in what I can today, but am then going to be gone for the next 10 days.

 

Much of what I'll put in the first couple posts will be copies of what I've outlined in other threads, but all collated in a single spot so folks considering these drives can read just a few posts instead of following some long threads where we were largely conjecturing how they would perform -- but we now know (thanks to Storage Review and the testing done by Daniel & others) a lot more of the details, so many pages of posts can be summarized in a few posts.

 

For those interested:  http://lime-technology.com/forum/index.php?topic=39526.0

 

I started a "sticky" to try and collate most of the "lessons learned" about these shingled technology units.    I'll fill in what I can today, but am then going to be gone for the next 10 days.

 

Much of what I'll put in the first couple posts will be copies of what I've outlined in other threads, but all collated in a single spot so folks considering these drives can read just a few posts instead of following some long threads where we were largely conjecturing how they would perform -- but we now know (thanks to Storage Review and the testing done by Daniel & others) a lot more of the details, so many pages of posts can be summarized in a few posts.

 

For those interested:  http://lime-technology.com/forum/index.php?topic=39526.0

 

Awesome.  I have to say I'm pretty much convinced these drives are excellent choices for most UnRAID setups.  I think they are at the top of my list when I upgrade my parity drives this summer.

Awesome.  I have to say I'm pretty much convinced these drives are excellent choices for most UnRAID setups.  I think they are at the top of my list when I upgrade my parity drives this summer.

 

Agree.    When these first came out, I thought they'd be good data drives, but not a good choice for parity, due to the far more random nature of parity accesses -- and the fact that they're mostly writes (everything except parity checks & rebuilds).    But the reverse-engineering Storage Review did to expose the essence of Seagate's mitigation strategy; and the testing Daniel has done; have clearly shown that they work just fine for all drives in an array -- including parity.    There ARE some worst-case scenarios that I can conjure up that would likely force them to "hit the wall" where the persistent cache was full and additional writes were at abysmal speeds ... but these are simply NOT likely scenarios in UnRAID usage.

 

... Bottom line:  I'm definitely planning to use them for my next server.

 

Edit:  The ONLY thing that would likely deter me from doing so would be a sudden spate of negative reliability experiences from the current users ... but I do NOT anticipate at all.    As I've noted in another thread, I think any drive that's been thoroughly tested for "infant mortality" issues is likely to last for a LONG time.    Some of my oldest drives are the "terrible" old Seagate 1.5TB units that were trashed in many forums for being completely unreliable.    I've got a half dozen of them that have well over 50,000 hours on them and are all still working perfectly in my oldest array.