Seagate’s first shingled hard drives now shipping: 8TB for just $260

[garycase]

... the remedy is to use a RAID0 pair OR buy a large non-SMR drive for parity and move the SMR to a data slot.

 

Precisely what I've said several times in this thread 

[garycase]

A sequential write is writing a long series of consecutive sectors. Whether the sectors previously had data on them or not is not the drive's concern - only the file system knows or cares. Your notion that the first write matters seems false.

 

I am, of course, well aware of what a sequential write is 

 

According to the Carnegie Mellon article on this technology and the mitigation efforts that manufacturers are using to minimize the write penalty, the drives maintain a flag as to whether or not data has been written to each track.  I certainly agree (and have said this earlier) that only the file system knows if there's actual data in these tracks; but the drive DOES know if they've ever been written to ... and that is what determines whether or not there's a need to do a read/rewrite of subsequent tracks in the shingled band when a track is written.  If the overlayed tracks haven't previously been written to, then the read/rewrite operation isn't required (or done).  THAT is why the first write to the tracks in a shingled band matter -- subsequent writes to the same tracks will be notably slower due to the read/rewrite required for the overlayed shingles.

 

 

[BRiT]

Even if the drives do not go through a preclear, simply adding them to the array will have 0s written to every single sector because unraid needs to know the parity calculation iscorrect. There goes any flag being cleared on the drive for every single sector.

[SSD]

Yes, and furthermore the drive would need to maintain the zeros if it did a partial shingle band write. But I still believe that if you are doing a large sequential write and all of the sectors are being written that the read could be avoided. It's the way I'd code it!

[garycase]

Even if the drives do not go through a preclear, simply adding them to the array will have 0s written to every single sector because unraid needs to know the parity calculation is correct. There goes any flag being cleared on the drive for every single sector.

 

Good point.  I had thought about the issue with pre-clears, but you're absolutely right that if it's added to a protected array, it will still be zeroed, so the same thing's true there as well.    The only time it'll not be true is if you're populating the drive before you assign parity (as many do when building a new array).

 

 

Yes, and furthermore the drive would need to maintain the zeros if it did a partial shingle band write. But I still believe that if you are doing a large sequential write and all of the sectors are being written that the read could be avoided. It's the way I'd code it!

 

It's certainly the way I'd code it too -- IF the individual band controls were done by the computer.  But since they're not, any such logic has to be implemented within the drive's firmware, which is, of course, file system agnostic.    There are, however, cases where the band rewrites will likely be avoided by the drive's firmware even with rewrites as long as the firmware is written to correlate FPDMA native command queuing requests with the banded shingles.  If there are pending full-track writes to every shingle in a band, then the firmware could (and should) be smart enough to realize it can skip the read/re-write process.    This is one of the optimization techniques discussed in the Carnegie Mellon paper I mentioned earlier ... and I suspect it's in fact implemented in the firmware for the SMR drives.

 

Note, however, that even though you could avoid the read/re-write requirement, there will still be a read of the entire band by UnRAID so it can preserve parity.    Thinking a bit about that shows just how complex the interaction between NCQ and the band rewrite process needs to be => consider a typical NCQ optimization:  Suppose the following commands are queued:  (a) Read Track X, (b) Read Track X+1, © Write Track X, (d) Write Track X+1.    The seek-optimized way that NCQ would execute these commands would be to read track X, then write track X, and then do the operations on track X+1.    But note that on a shingled drive writing track X would destroy the data on track X+1 ... so the NCQ optimizations need to be disabled while operating within that band.

 

Note also that unless EVERY track in a band is being written, the read/re-write process still needs to be done to protect the data on the unwritten tracks.

 

 

 

 

 

 

 

 

[c3]

The seek-optimized way that NCQ would execute these commands would be to read track X, then write track X, and then do the operations on track X+1.

 

 

You might want to read about what NCQ is. https://www.smarthdd.com/en/ncq.htm

 

Since the drive is doing the NCQ optimization, it would be so ridiculous to do as you suggested. These drives fully understand that track is meaningless, and optimizations would be applied to band rather than track.

 

On the other hand, you could mess with the scheduler, in which case you should treat these drives like SSD, not using CFQ. Basically, if you need to optimize for reads, pick deadline, otherwise noop.

 

While these drives are called "archive", they are engineered to be completely rewritten 67 times in the 3 year warranty period (approximately 3.5TB per week).

[BRiT]

The seek-optimized way that NCQ would execute these commands would be to read track X, then write track X, and then do the operations on track X+1.

 

 

You might want to read about what NCQ is. https://www.smarthdd.com/en/ncq.htm

 

Since the drive is doing the NCQ optimization, it would be so ridiculous to do as you suggested. These drives fully understand that track is meaningless, and optimizations would be applied to band rather than track.

 

On the other hand, you could mess with the scheduler, in which case you should treat these drives like SSD, not using CFQ. Basically, if you need to optimize for reads, pick deadline, otherwise noop.

 

While these drives are called "archive", they are engineered to be completely rewritten 67 times in the 3 year warranty period (approximately 3.5TB per week).

 

So will it crap out or be outside of warranty if you rewrite a single sector 68 times (which would be the entire shingle for that band)?

[c3]

The media reliability should be similar to high capacity PMR drives. SMR does not change the media, hence this is a 25%/33% increase in capacity without new media technology.  Sector reallocations are still possible. But like most high capacity media, single sector reallocations are becoming rare. SMR increases the chance that a defect will span tracks as well as sectors, due to the higher track density achieved.

 

 

[garycase]

I'm very familiar with the NCQ command set, although I have not read any details on what enhancements may have been made to this with regard to shingled drives.  If the NCQ decisions are indeed based on bands with these drives (which makes sense), then this will certainly eliminate issues like I noted above ... I'm not at all surprised that this is apparently the case.

 

I presume the enhancements for SMR drives also include "look-ahead" logic to eliminate band rewrites when enough writes are queued to entirely rewrite a band, or to not write a track before doing reads on other tracks in the same band).    As I noted much earlier, I'm sure the manufacturers have done whatever they can to help mitigate the otherwise very slow write performance these drives would have;  but there's only so much you can do to overcome physical limitations (e.g. the time for the drive to rotate enough times to do a band rewrite).

 

[jtown]

So has anyone tried one of these drives yet or are we all just flapping gums?

[danioj]

Gum flapping for me ....

 

I'm going to try one when I can get one for array drives in my Backup server. Going to keep WD Red's in Production. At the moment though here (In Australia) they are no-where to be found!

 

I've seen them on State Side sides Amazon and bhphotovideo but both for pre order. Over there the price seems similar be as advertised in the initial press release which is good.

 

Over here there is only one site Techbuy which has them for pre-order but the price is so much higher 420AUD (~350USD). Wont be buying from them - I think the price is inflated as they seem to be the first to be selling them over here.

 

What I have found interesting is that Seagate are not selling them on their web site yet. Plus - the warranty at each site that "has" them so far seems to be listed as 1 year with re-seller. Makes me pause.

[jtown]

A few minutes after I posted that I saw that Amazon had dropped the price of the external a few dollars.  After confirming with Newegg's Seagate rep that the drive inside is a standard SATA drive, I ordered one from Amazon.  (Sorry, Newegg, but $15.79 is $15.79.)

[danioj]

What is the warranty on the external drive? Will Seagate honor 3 yr warranty on the drive inside it?

[jtown]

What is the warranty on the external drive? Will Seagate honor 3 yr warranty on the drive inside it?

 

I'm 99.44% certain that opening the external case voids the warranty.  That's been the price of shelling since the practice began.

[garycase]

... After confirming with Newegg's Seagate rep that the drive inside is a standard SATA drive ...

 

SATA is the interface to the drive -- it has nothing to do with the recording technology used on the platters.  A shingled (SMR) drive IS a "standard SATA drive" ... it's just doesn't use perpendicular recording (PMR).

 

Since Seagate only makes one 8TB drive (the new SMR Archive unit), it's almost certain that what's in their 8TB external is one of these new drives.    Note that there are other 8TB external units available, but they all have 2 drives in them (i.e. they're RAID-0 arrays with a pair of 4TB drives).

 

[jtown]

... After confirming with Newegg's Seagate rep that the drive inside is a standard SATA drive ...

 

SATA is the interface to the drive -- it has nothing to do with the recording technology used on the platters.  A shingled (SMR) drive IS a "standard SATA drive" ... it's just doesn't use perpendicular recording (PMR).

 

Since Seagate only makes one 8TB drive (the new SMR Archive unit), it's almost certain that what's in their 8TB external is one of these new drives.    Note that there are other 8TB external units available, but they all have 2 drives in them (i.e. they're RAID-0 arrays with a pair of 4TB drives).

 

Good Lord, I'm not a moron.  Can we get that accepted as a given?  Thanks.

 

I asked if it had a standard SATA interface because some newer portable drives have the USB interface soldered directly to the drive and have no SATA interface.  I wanted to make sure that wasn't the case with this drive.

[trurl]

...Good Lord, I'm not a moron.  Can we get that accepted as a given?  Thanks...

Nobody said you were. He was just adding some information to the discussion for the benefit of all.

 

Also, other than 4 posts today, you haven't posted in over a year so nobody knows what to assume about your knowledge without taking the trouble to look at your posting history. Better safe than sorry. There are certainly some people on this forum who need to be saved from themselves.

[jtown]

...Good Lord, I'm not a moron.  Can we get that accepted as a given?  Thanks...

Nobody said you were. He was just adding some information to the discussion for the benefit of all.

 

Also, other than 4 posts today, you haven't posted in over a year so nobody knows what to assume about your knowledge without taking the trouble to look at your posting history. Better safe than sorry. There are certainly some people on this forum who need to be saved from themselves.

 

If somebody gets this far in without realizing there are potential performance and longevity issues with this drive due to the new SMR technology, one more post isn't going to save them.

 

As for how long it's been since I've posted, what does that have to do with anything?  You can also see that I've been around here since 2010 and that my posts aren't frivolous or uninformed (unless I'm looking for information on a subject).

 

If you'd like to benefit from my experience with the new drive when it arrives, I'll be glad to post it.  If not, just let me know and I'll be on my way.

[garycase]

As trurl noted, I wasn't implying any such thing.    The context of your post made it seem you may have assumed the drives in the external cases weren't shingled ... so I just wanted to be sure you understood that they almost certainly are.

 

As trurl also noted, there are indeed a lot of folks who "... need to be saved from themselves ..."

 

[garycase]

If somebody gets this far in without realizing there are potential performance and longevity issues with this drive due to the new SMR technology, one more post isn't going to save them.

 

As long as they realize that they're buying one of the SMR drives -- but if they thought they were buying a PMR drive it might be useful for them to realize that they weren't 

 

As for "... performance and longevity issues ..." ==> It's not at all clear there will be any longevity issues, but there will almost certainly be write issues ... and until we see how well they've mitigated those, it's all speculative.

 

 

If you'd like to benefit from my experience with the new drive when it arrives, I'll be glad to post it.  If not, just let me know and I'll be on my way.

 

I'm sure we'd all like to see the performance details when you get the drive -- preferably "outside" of UnRAID, so the performance isn't throttled by the parity calculations and/or network speed.    Obviously the most interesting results will be for random writes.

 

[jtown]

Oh, there's longevity issues.  A month ago someone said the drives are engineered to be completely overwritten 67 times.  If true, a few preclear cycles will put a big dent in that number if.  Then the initial parity build.  And it will get written to every time I store something on any drive.

 

First one's going in as a parity drive which should hammer on every pressure point of SMR.

[c3]

Oh, there's longevity issues.  A month ago someone said the drives are engineered to be completely overwritten 67 times.  If true, a few preclear cycles will put a big dent in that number if.  Then the initial parity build.  And it will get written to every time I store something on any drive.

 

First one's going in as a parity drive which should hammer on every pressure point of SMR.

 

I am the source of the 67x capacity written. This is the stated spec of the drive. This is to make it clear these are not meant to be written once and read mostly, which is how some were characterizing the SMR drives. Clearly by the specifications, these drives are engineered to handle a lifetime of writes, and should not be limited to a "fill and read" characterization.

 

As noted previously in this thread, the media is the same as PMR drives, so media longevity is not a concern.

[trurl]

...As for how long it's been since I've posted, what does that have to do with anything?  You can also see that I've been around here since 2010 and that my posts aren't frivolous or uninformed (unless I'm looking for information on a subject).

...

My point was that unless someone goes to the trouble of looking at your post history as I did on this occassion, we don't know anything about you.

[garycase]

... I am the source of the 67x capacity written.  This is the stated spec of the drive.

 

Curious where you found this specification?    It's not on Seagate's site, nor in the Data Sheet for the drive.  I assume you have mistakenly calculated this based on the 180TB/yr Workload Rate Limit (for 3 years this would be 540TB, which divided by the size of the drive is 67.5).    The Workload Rate Limit is NOT, however, based on writes to the drive.  Seagate determines this by adding the lifetime reads and lifetime writes, then annualizing this.    In fact, the 180TB spec is fairly low ... designed for "nearline light" drive usage.    Their nearline products typically have a 550TB/yr specification. 

 

The Workload Rate Limit is one of the key specs in Seagate's studies on drive reliability -- they note in their papers on this that "... Keeping HDDs at a low temperature within the specified range and within their usage time and workload specifications are necessary conditions for long-term drive reliability."  The 180TB/yr spec for the Archive Drives is simply the guideline for the total reads + writes for these drives during the 8760 hrs/yr they designed to operate (24/7).

 

I do not, however, think there's any issue with doing as many writes as you want with these drives -- but they're simply not designed for applications where write performance is a key requirement.  There's a good reason they're called "Archive" disks    Seagate's specification sheet notes the following list of "Best-Fit Applications":

(Note the emphasis on "archive" and "cold storage")

 

• Cost-effective online archiving

• Object storage

• Big Data cold storage

• Cloud active archive

• Web-scale archiving

 

 

As for the physical media ==> I certainly agree that ...

 

... media longevity is not a concern.

 

[jtown]

Just got my tracking number.  Should have the drive by end of day tomorrow and I'll run some benchmarks while it's in the case.  In addition to that, I'll copy various sets of files from RAM to the drive and back (provided I can find a good RAMdisk program).  A 10 gig file, ten 1 gig files, 10 gigs of ~5 meg files, 10 gigs of tiny files.  Then a full diagnostic before I pop it out of the case.  Then I can see what it gets on a fast machine connected directly to a SATA controller with the same benchmarks and file sets.

 

Any requests for other tests should be made before I start preclearing.