Seagate 8TB Shingled Drives in UnRAID

[falconexe]

Another great tutorial on this topic if you just want a straight forward encrypted cloud sync.

 

 

[falconexe]

Here is another way of performing block based backups (true backups) not syncing. You can use the same cloud services, or even UNRAID to UNRAID.

 

 

[falconexe]

9 hours ago, BRiT said:

 

You should check out this thread on how Google Drives and Rclone can provide unlimited storage for around $12 a month. They don't actually require 5 users and they don't limit storage to 1TB when less than 5 users. Some users have well over 200 TBs.

 

I was using BackBlaze via a fleet of external USB3 drives and a Windows 10 client with SyncFolders. Needless to say, it was a bit much ha ha. I've been looking for something quick, secure and native to UNRAID for a LONG time. I just setup GSuite tonight and have my ENTIRE server syncing to the cloud and uploading at a staggering 250Mbit/s (Peaks at 35 Megabytes/s!). I have fiber with 1Gig Up though... One note, in order to setup GSuite, YOU WILL NEED A DOMAIN. That was a bit surprising, but I already had one so just hooked it up through GoDaddy authentication pass-through (GSuite prompts for this).

 

Thanks for the suggestion. Instead of using RClone, I ended up using the Duplicati docker with encryption. So far, so good!

[falconexe]

7 hours ago, falconexe said:

I just setup GSuite tonight and have my ENTIRE server syncing to the cloud and uploading at a staggering 250Mbit/s (Peaks at 35 Megabytes/s!). I have fiber with 1Gig Up though...So far, so good!

 

Welp, there are drawbacks from having fiber internet. I already hit the limit of GSuite (Google Drive) in just a few hours at those sustained upload speeds. So apparently, you can only upload 750GB in a single 24 hour time-frame (shared across all GSuite products).

 

I hit exactly 725.4GB before Duplicati started throwing server side errors. I have since throttled my uploads to 8 MB/s to keep it under this ceiling. (Math works out to 691.2 GB/Day [8MB * 86,400 Sec / 1000MB = 691.2GB/Day]. 9 MB/s puts it over, and the parameter has to be a whole number). This should keep Duplicati happy and support uninterrupted backups during my initial upload set. This would never be a problem once all files are initially backed up, but it is an interesting facet of the this solution's workflow.

 

 

[BRiT]

11 minutes ago, falconexe said:

 

Welp, there are drawbacks from having fiber internet. I already hit the limit of GSuite (Google Drive) in just a few hours at those sustained upload speeds. So apparently, you can only upload 750GB in a single 24 hour time-frame (shared across all GSuite products).

 

I hit exactly 725.4GB before Duplicati started throwing server side errors. I have since throttled my uploads to 8 MB/s to keep it under this ceiling. (Math works out to 691.2 GB/Day [8MB * 86,400 Sec / 1000MB = 691.2GB/Day]. 9 MB/s puts it over, and the parameter has to be a whole number). This should keep Duplicati happy and support uninterrupted backups during my initial upload set. This would never be a problem once all files are initially backed up, but it is an interesting facet of the this solution's workflow.

 

 

 

Thats why they suggest using Service Accounts in the rclone thread. They have the means to automatically cycle through them to remove these limits when using rclone.

[falconexe]

8 minutes ago, BRiT said:

 

Thats why they suggest using Service Accounts in the rclone thread. They have the means to automatically cycle through them to remove these limits when using rclone.

Good to know. Is there anything like that for use with Duplicati? I saw that with RClone, you can get an API ID.

[JorgeB]

Please stay on topic and start another another thread if you want to continue that discussion. 

[falconexe]

2 minutes ago, johnnie.black said:

Please stay on topic and start another another thread if you want to continue that discussion. 

Yeah... no worries.

[Paul_Ber]

I have the 6TB WD60EFAX that WD never said what type it was until the spring 2020, well these are SMR.

 

Model numbers of the drives and SMR or CMR: 

2TB    WD20EFAX    SMR
2TB    WD20EFRX    CMR
3TB    WD30EFAX    SMR
3TB    WD30EFRX    CMR
4TB    WD40EFAX    SMR
4TB    WD40EFRX    CMR
6TB    WD60EFAX    SMR
6TB    WD60EFRX    CMR

 

I contacted WD and they are doing an advanced RMA without giving them my credit card number.  They are sending me a WD60EFRX 1st.  Then once I have done all the preclear, swap, rebuild, wipe old drive it should be a few days I guess.

 

I already have:

WD60EFRX parity

WD30EFRX

WD30EFRX

WD60EFAX the drive i will be exchanging 

WD50EFRX not added to array yet

 

When do increase the size of the array I usually buy a bigger parity drive and the old parity drive replaces a smaller drive.  Limited to 4 drives at the moment, and 2 cache.

 

WD has a weird naming convention because the have a WD80EFAX that is CMR. 

 

Edited December 15, 2020 by Paul_Ber

[Hoopster]

8 minutes ago, Paul_Ber said:

WD has a weird naming convention because the have a WD80EFAX that is CMR

I think all models 8TB and above are CMR only so they don't worry about model number distinctions as much, but, it's still confusing given the naming conventions on the lower capacity models.

[Lolight]

On 12/1/2018 at 8:47 PM, John_M said:

Five pre-clear passes though! That's 5 x 3 x 8 = 120 TB of reads/writes, which is more than twice the annual workload in less than 10 days. That's my main objection to these Barracuda Compute drives - they have a very low annual workload rating (55 TB/year) which, on drives of such capacity, you'll exceed just doing a monthly parity check. However, in their favour, they do have an unrecoverable read error rating of one per 10^15 bits read, which is an order of magnitude better than a WD Red.

Am I being wrong to conclude that Seagate 8TB shingled drives work perfectly fine in the typical Unraid environment (write once/read many)?

[JorgeB]

1 hour ago, Lolight said:

Am I being wrong to conclude that Seagate 8TB shingled drives work perfectly fine in the typical Unraid environment (write once/read many)?

Most of them yes, old Archive drives work great, some newer models like the ST8000DM004 appear to be more hit and miss, at least some of them. 

[Lolight]

4 hours ago, JorgeB said:

Most of them yes, old Archive drives work great, some newer models like the ST8000DM004 appear to be more hit and miss, at least some of them. 

Over the last few years I have acquired 4 Seagate 8TB drives, which are still in their USB enclosures.

All drives are full.

I consider transferring all their data to my brand new Unraid machine and then adding them to the array.

Here's the list:

 

ST8000AS0002 - green SMART

ST8000DM004 - green SMART

ST8000DM004 - green SMART

ST8000DM004 - yellow SMART:

C5 Current Pending Sector Count - 8

C6 Uncorrectable Sector Count - 8

 

How serious is the SMART reporting result in the last one?

I assume I shouldn't even consider adding that one to the array?

Thanks!

Edited May 6 by Lolight

[JorgeB]

49 minutes ago, Lolight said:

How serious is the SMART reporting result in the last one?

It's serious unless they are false positives, you can confirm by running an extended SMART test.

[ChatNoir]

47 minutes ago, Lolight said:

ST8000DM004 - yellow SMART:

C5 Current Pending Sector Count - 8

C6 Uncorrectable Sector Count - 8

 

How serious is the SMART reporting result in the last one?

I assume I shouldn't even consider adding that one to the array?

 

You have to consider that the Parity protection is dependent of the parity drives and all drives in the Array. If a drive is not reliable, you are putting your data at risk.

[Lolight]

On 5/6/2021 at 7:39 PM, JorgeB said:

It's serious unless they are false positives, you can confirm by running an extended SMART test.

Scanned the drive with StableBit scanner.

Upon completion it has warned me of problematic 8 sectors (as in the prior to the scan SMART report) and then offered me to recover damaged files.

I agreed.

StableBit seemed to have no problem reading those 8 presumably damaged sectors and recovered the file.

And now SMART shows green.

Reallocated Sectors, Current pending sector and Uncorrectable sector counts all show 0.

 

1) Does it mean that the initial SMART was false positive?

2) What could be possible reasons for SMART throwing false positives?

 

Thanks.

Edited May 8 by Lolight

[xerces8]

Those sectors were corrupted. Now they are overwritten with error-free content.

 

A quite normal scenario.

[PeteAron]

I'm late to the party on this one

 

I have one ST8000DM004 that I bought because it was cheap.  It failed in february with a bunch of udmacrc errors that were not affected by swapping out the cable - last count was 1491 and that was generated after a preclear when i tried to put it into my array and write to it - unraid wrote a few GB and then hundreds of crc errors popped and i had to rebuild my array with the disk it was replacing.

 

I was able to RMA that drive and i now have a refurbished ST8000DM004 with another year of warranty.  I really would like to use it in my array, and from reading here it sounds like I can get the life out of it by rebuilding my smaller disk onto it, then moving data from other disks onto the DM0004 so that no more writes occur to it, and just let it sit and serve data.  IIRC i used the original DM004 that failed as my parity disk for a while.  

 

Does anyone like or dislike my strategy?  I hate to throw away $250 replacing what should be a perfectly serviceable drive.  I'd like to minimize my risk with it.

 

I am running dual parity, so not that worried about a failed drive if that's what happens.  

 

Thanks for your suggestions on this.

[jbartlett]

1 hour ago, PeteAron said:

moving data from other disks onto the DM0004 so that no more writes occur to it, and just let it sit and serve data

I did that with mine. But in your case, I'd copy (not move) files onto it and then do a binary/crc compare on the files to ensure they're identical. Beyond Compare by Scooter Software can help with this. Then once a match is verified, remove the source.

[PeteAron]

Thanks jb.  I think you are implying to then remove the old drive and use the new config to rebuild parity after swapping the disks?  Probably a bit less work too.

 

thanks!

[jbartlett]

2 hours ago, PeteAron said:

Thanks jb.  I think you are implying to then remove the old drive and use the new config to rebuild parity after swapping the disks?  Probably a bit less work too.

 

thanks!

I think I follow what you're saying. The parity rebuild should function the same as forcing a write on each sector and then verifying that write - but you'll be at a greater risk of a party rebuild failing since all sectors must report OK.

[Lolight]

I have a questions related to one of my spare 4TB Seagate USB drives (by Samsung) that I have uncovered in a closet.

It's in a third party USB enclosure.

A complete surface scan with Stablebit hasn't found any errors.

Except for a very strange one: the drive doesn't output any SMART info other than the basic attributes: model number, serial number, buffer size, speed and drive letter (checked with a couple of tools).

Tried another USB enclosure - no difference.

I've searched and couldn't find anything on what could be a problem.

 

 

Edited May 14 by Lolight

[xerces8]

It is a common problem with many USB enclosures. Do other hard drives return more information in the same enclosure?

What software did you try?  CrystalDiskInfo for example returns SMART data when HDTune does not. (yes, HDTune is old, just an example)

[Lolight]

17 hours ago, xerces8 said:

It is a common problem with many USB enclosures. Do other hard drives return more information in the same enclosure?

What software did you try?  CrystalDiskInfo for example returns SMART data when HDTune does not. (yes, HDTune is old, just an example)

I've tried a different enclosure with the same result - no SMART output.

I've tried CrystalDiskInfo and Stablebit - no output from that drive.

All my other external drives report SMART as expected.