Seagate’s first shingled hard drives now shipping: 8TB for just $260

[danioj]

Arrrrggghhh. Crashed again. It seems like using a VM from my iMac to do this from within Windows is not going to work. I'll have to look at a different solution again when I get home!!

 

Edit: I am just going to use Rsync. I was concerned about using Rsync before as I didn't know it did post copy transfer verification. However just read this from the man page.

 

Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking a whole-file checksum that is gener? ated as the file is transferred, but that automatic after-the-transfer verification has nothing to do with this option’s before-the-transfer "Does this file need to be updated?" check.

 

I'll try and set this command up tonight and have it spit out the progress. I guess I could run it in screen. I've only used Rsync locally before not remotely so need to read up.

[garycase]

The good news is it's the Mac that's crashing (or at least the Mac VM) -- NOT either of the UnRAID boxes 

 

Just do it with RSync ... that's fine.    If you include --progress on the command line, it will show the % done and the speed as the copy progresses.

 

[danioj]

Agreed! It would be much worse if it was Unraid.

 

I went into the data centre at work and asked the tech in there and he might have solved it for me. He said that terracopy has a memory leak issue that he has experienced which means it keeps consuming until it crashes the OS when copying large volumes of data. He said that the issue was identified by the terracopy team and addressed in their latest v3 alpha. Suggested I try that (I downloaded latest "stable"). I'll give it one more go with this suggestion before I switch to Rsync.

 

The gods don't want me to do this do they!??

[garycase]

Interesting ... I've never encountered that, although I think the largest copy I've done was about 2TB of data.

 

Just looked at mine, and I'm still on v2.27 => never upgraded to v2.3 (latest)    Not sure if the leak was introduced in v2.3, or if I've just been lucky and not encountered the bug.

 

Certainly won't hurt to try v3 alpha 2

 

[danioj]

Further Update.

 

It looks like I have been able to keep a successful copy. Added VM Guest Additions to VM, disabled a few pieces of software running in the background (on both guest and host) and changed the share from DNS name to ip address (Still using latest Terracopy Stable).

 

Anyway here is your update. To build the suspense ...... 

 

 

25% after 1TB of data the copy is still smashing along at ~38MB/s.

 

Hope it all holds steady!

 

EDIT: For those who need more evidence here is a screenshot of my Backup Server Unraid GUI.

 

[garycase]

Looking good.  Your drive temps are excellent, and it's a very good sign that the speed is holding.  I think it's very likely it'll stay that way, since your files are clearly primarily large media files ... you've only got 1472 files to copy, so that's an average of over 3GB/file.    Looking forward to further updates as the copy progresses to confirm that 

[danioj]

Looking good.  Your drive temps are excellent, and it's a very good sign that the speed is holding.  I think it's very likely it'll stay that way, since your files are clearly primarily large media files ... you've only got 1472 files to copy, so that's an average of over 3GB/file.    Looking forward to further updates as the copy progresses to confirm that 

 

Spot on. I think it will hold too, but time will tell.

 

As for temps, those are with the new Noctuas I installed. I managed to shave another 2-3C off each drive in the DS380. I'll post an update in the DS380 thread.

[danioj]

Further Update:

 

Copy is still going Strong!! Now at 58 percent (2.7 TB) and maintaining 38 - 40MB/s 

 

No noticeable slowdown at all so far!

 

 

[garycase]

Looking great.  I'm confident it will sustain full speed all the way to the finish.   

 

The results would likely not be quite so good if, instead of a lot of large media files you had tens of thousands of MP3's, pictures, etc. that were dramatically smaller files (a few MB average size instead the over 3GB average your data has).    Even then it's not clear just how "smart" the drive's firmware is r.e. mitigation of this => since the writes are all very close together in time (even though the "most full" setting causes the location of the writes on the parity drive to be in different locations, there are still a lot of writes very close together that MAY be buffered in the drive's RAM for a bit and then recognized as full band writes.    But even if that's not the case, and the writes slow dramatically down due to a full persistent cache,  that could be avoided by simply not using the "most full" allocation setting -- so the files would be written sequentially.

 

Bottom line:  This is very clearly showing that these drives work well even as parity for typical UnRAID usage 

I guess my next server will be packed full of shingled drives !!  [8TB or larger, depending on when I build it  ]

 

 

[danioj]

Looking great.  I'm confident it will sustain full speed all the way to the finish.   

 

I hope so!

 

The results would likely not be quite so good if, instead of a lot of large media files you had tens of thousands of MP3's, pictures, etc. that were dramatically smaller files (a few MB average size instead the over 3GB average your data has).    Even then it's not clear just how "smart" the drive's firmware is r.e. mitigation of this => since the writes are all very close together in time (even though the "most full" setting causes the location of the writes on the parity drive to be in different locations, there are still a lot of writes very close together that MAY be buffered in the drive's RAM for a bit and then recognized as full band writes.    But even if that's not the case, and the writes slow dramatically down due to a full persistent cache,  that could be avoided by simply not using the "most full" allocation setting -- so the files would be written sequentially.

 

Well, after this run I have another share to backup which is of similar size but the files are between 5 and 10 times smaller. Ill post those too if you'd like to see that data.

 

Bottom line:  This is very clearly showing that these drives work well even as parity for typical UnRAID usage 

I guess my next server will be packed full of shingled drives !!  [8TB or larger, depending on when I build it  ]

 

[garycase]

Well, after this run I have another share to backup which is of similar size but the files are between 5 and 10 times smaller. Ill post those too if you'd like to see that data.

 

Yes, please do.  Those are still fairly large files, but will likely have a LOT more "fragments" that end up on the persistent cache.  If that also runs to completion without any slowdowns, that will be even more confirmation of how well these drives work for UnRAID.

   

[HellDiverUK]

Total lack of stock of the 8TB Archive in the UK at the moment.  3 suppliers listing them, all back ordered. 

[danioj]

Last Update for this test:

 

Copy is Complete (It's now doing the testing of the files which I guess you don't need to see). To me "on average" speed was maintained throughout at ~38MB/s.

 

 

 

 

I did sit in front of it for a couple of periods (maybe 20 minutes each time) just to add some observations. I DID notice that every now and then the speed would dip. In my first observation (at around 50%) it dipped in the copying of a file to about 18MB/s but only for a second and went back up to 38-39MB/s. It did this a couple of times.

 

Similarly I just sat in front of it at the end and I observed that the speed dipped this time observed to as low as 2MB/s but again only for a second and then went back to 38-39MB/s. Again it only did this a couple of times in 20 minutes.

 

I have to go to work but Ill let you all draw whatever conclusions you want (and Ill read later). I'll set the copy of the smaller files going tonight and report.

[garycase]

Thanks for the detailed updates.    As you noted, there's no reason to provide play-by-play updates on the verifications ... they're just reads, so the shingling has no impact on that.

 

The occasional brief (second or two) drops in speed aren't something I'd worry about.    That's not uncommon ... it could simply be a seek delay on one of your source disks;  a delay while UnRAID empties its own buffers to disk; or some directory writes that were buffered.    I don't think (but don't know for sure of course) that it's a shingled disk band rewrite delay, as I'd expect that to be a bit longer than just 1 or 2 seconds.    It COULD be that ... but even if it is, if this only lasts for a small number of seconds, that's excellent news !!

 

[danioj]

Some further supportive data from an Aussie forum I'm a member of but not active in.

 

http://forums.overclockers.com.au/showpost.php?p=16643176&postcount=139

[garycase]

An interesting set of additional data.    It seems very clear that the mitigations that Seagate incorporated in the firmware of these units work exceptionally well => especially for typical UnRAID usage patterns.

 

And even if a user isn't "typical", as long as there are never writes that fill the persistent cache (25GB) it's unlikely the "wall" of write performance will ever be hit.

 

[garycase]

Daniel,

 

How did the copy of the smaller set of files go??

 

[danioj]

Daniel,

 

How did the copy of the smaller set of files go??

 

 

I've had to go away on Business since the 15th in ASP, NT. I didn't set it running because I wouldn't be able to observe. I fly back into MEL on Thursday night AEST and will run it then. Standby.

[garycase]

Thanks for the update ... just wondered what had happened since the status posts suddenly went "silent".  Just wanted to be sure you didn't have any system crashes; failed validations; etc.    Bad news is NOT allowed  :)

... these Archive drives are doing really well !!

 

[HellDiverUK]

I just got a pair of 8TB Archive drives.  I'll get then stuffed in to my UnRAID box this evening hopefully.

[HellDiverUK]

They're in. I've blown away my old array and started again with the two 8TB drives.  Parity is building at 189MB/s, expected to finish in just under 12 hours. 

[pras1011]

Where did you buy them from and how much?

[garycase]

They're in. I've blown away my old array and started again with the two 8TB drives.  Parity is building at 189MB/s, expected to finish in just under 12 hours.

 

Are these the only two drives in your new array?

 

 

[HellDiverUK]

pras1011, I got them from Lambdatek, they were about £230 each.  Similar price to a 6TB Red.

 

http://www.lambda-tek.com/Seagate-ST8000AS0002~sh/B1999368

[HellDiverUK]

They're in. I've blown away my old array and started again with the two 8TB drives.  Parity is building at 189MB/s, expected to finish in just under 12 hours.

 

Are these the only two drives in your new array?

 

Correct.  For now, anyway.  The two 4TB drives will go in later.