garycase

Yes, I definitely mean outside of the UnRAID server. Simply storing data on a different set of drives in the same server is NOT a good backup -- a major event that wipes out your server would also wipe out your backups. What your individual backup strategy is depends on how YOU want to set it up. Clearly how much backup space you need depends on how much data you have that you need/want to back up. I personally backup everything -- I figure if it's important enough to store on a fault tolerant server; it's important enough to backup. So I have a dedicated backup server that backups up both of my other UnRAID servers as well as some of my personal systems. But if you only have a few TB of "important" data to backup, you may very well be able to do your backups on, for example, a single external USB v3 drive => you can get pretty high capacity external drives these days for very reasonable prices ... 5TB for ~ $125; 8TB for ~ $210. You could also use a set of bare drives that you store in drive cases and access via docking stations. This strategy lets you use old, lower capacity drives that you've retired from active service in your PC's as backups ... these can even be drives that you don't want to use in an active system because of high reallocated sectors counts; excessive hours; etc. -- but are fine for the very occasional access they'd receive in this use case. The important thing isn't HOW you do your backups; it's that you DO them

Anyone who follows my comments over the years r.e. mini-ITX builds knows I ALWAYS recommend using nothing but SFX power supplies for these builds They simply provide more working room; better airflow; and, as you've just documented well; more room for cable routing

Dual parity doesn't hold any more data than single parity. The parity drive (or drives) don't hold ANY data. They simply provide the technical ability to protect against failure of any of the other drives in the array. If you do not have, 'nor plan to get, a set of backup drives, then I would DEFINITELY go with dual parity. So what you need in your system is TWO 4TB parity drives; plus as many data drives as you want (up to the limits of (a) your UnRAID license; and (b) the capacity of your system ... not only in physical terms, but also based on the number of SATA ports you have available (you can always add more via expansion cards if needed). For example, using the list of drives you provided ... you could have 2 4TB parity disks; plus data drives of 4TB, 3TB, 2TB, and 1TB. This would give you 10TB of protected storage => i.e. if any 2 drives failed you wouldn't lose any data; although if a drive DOES fail you should immediately replace it and let UnRAID rebuild your data onto it -- what dual parity does is provide another layer of protection so that if a 2nd drive was to fail during that rebuild, the rebuild would still be successful and you wouldn't lose any data. Notwithstanding the excellent protection a dual parity system provides, you should still absolutely have a backup copy of the data you consider important do not want to take a chance on losing => e.g. your home movies & photos

That shows both the "Linux way" and how to do it with HDAT2. Personally, I find the simple "SetMax" command in HDAT2 the easiest method to use, but you can do whichever you find more comfortable.

Agree with just about all the points already noted. To summarize ... => Clearly dual parity is a major improvement over single parity. Think of it as extra insurance that very significantly decreases the likelihood of ever losing data during a disk rebuild. But you CAN start with a single parity drive, if that's more economically suitable to you. => The parity drive sets a limit on how large your data drives can be. On the other hand, it's a waste of money to buy a parity drive (or drives) that is larger than the size of the data drives you plan to buy. So if you buy 8TB drives for parity; then buy 8TB drives for data. If you're only going to use 4TB drives for data; buy 4TB drives for parity. etc. This applies to FUTURE purchases -- not necessarily the size of drives you have now. e.g. if you currently have some 4TB drives you can use; but plan to buy 8TB drives in the future; then clearly you should buy 8TB drives for parity. => Absolutely agree that RAID in any form -- regardless of the degree of fault tolerance -- is NOT a substitute for a good backup plan. => As also noted earlier, the # of drives in your array really doesn't change the benefits of dual parity -- no matter how many drives you have, it will protect any disk rebuild against a 2nd drive failure. Clearly the more drives you have, the more likely it is that you'll encounter that issue -- but Murphy (of Murphy's Law fame) tends to guarantee that number doesn't really matter

You might also be able to restore the max capacity by removing what is likely an HPA using HDAT2 ( http://www.hdat2.com/ ), unless the extra space is hidden via some other mechanism. There's also a "Linux way" to remove the HPA, but I don't recall the specifics -- someone else may post the details.

Seems John edited his post to show the actual number while I was posting. But yes, you can certainly use the drive as a data drive.

The standard byte count for a 4TB drive is 4,000,787,030,016

Yes, that's an excellent board for a very power-efficient system. You may, however, want to consider the even-better C2750D4I, which has 8 cores and has about 70% more CPU "horsepower". The 2750 is a much better choice if you plan to do any Plex transcoding. Note that while these Atom-based boards are excellent for a basic NAS with perhaps a single VM and a few Dockers; if you plan on doing more extensive virtualization and/or a lot of simultaneous Plex transcodes, you should look at higher-performance boards that use a Xeon processor.

Agree with Frank that the risk of only using single parity goes up as the # of drives increases. However, the benefit of dual parity is the same regardless of how many data drives you have => it still provides a 2nd layer of fault tolerance that will allow a disk rebuild to successfully complete even if a 2nd drive fails during the rebuild. This is true whether you have 2 data drives or 20 data drives. Clearly the likelihood of a 2nd failure increases as the number of drives increases, but it CAN happen with any number of drives. Another factor I'd consider is whether or not you're adequately backed up. Remember that RAID -- no matter how fault-tolerant -- is NOT a substituted for backups. If you have current backups, then if a drive fails, and you can't successfully rebuild it due to a 2nd drive failure during the process; then you can always just copy all of the data that was on that drive back to your array from your backups. If you don't have backups, that 2nd failure is clearly a FAR more consequential event -- a good reason to have a 2nd parity drive to protect against it, regardless of the number of data drives in the array.

Yes, I'd say that's working just fine

Good question. My understanding is that the parity drives do NOT have to be the same size, as long as both meet the condition of being >= any of the data drives. ... so I'm fairly sure you could simply replace one of them with a 10TB drive and wait for the rebuild to complete; and then replace the other and repeat the process. You could, of course, simply replace both at the same time, but then your system would be "at risk" until the new parity syncs completed.

I haven't done any USB-based pre-clearing; but my understanding is the newest version supports USB v3, so as long as your computer has USB3 ports, pre-clearing via USB should be just about as fast as a direct SATA connection -- and has the advantage that you'll know the drive is good before opening the case to remove it (thus voiding the warranty).

Not exactly surprising -- with five times the CPU power and four times as much memory

Remember these are called "shingled" drives => the name is derived from the fact that the writes overlap each other, as c3 noted above -- think of the shingles on a roof (this is why they named the technique "shingled"). The drives perform very well for reads (no different than standard PMR drives); but to write anything on a specific track requires re-writing the entire track as well as every track after it up until the next "buffer" track (where there's no shingling). This could result in TERRIBLE write performance, except there is an area reserved for "buffering" all of these random writes, known as the "persistent cache", which buffers writes to a non-shingled area, and then moves them to the shingled sections later (during idle time). Seagate has done a very good job of mitigating these issues, as I've outlined in this thread: http://lime-technology.com/forum/index.php?topic=39526.0

That's the right sector count for an 8TB drive, so you're good to go. In fact, you don't even need to bother removing the old partitions => UnRAID will re-initialize the drive anyway.

Thanks for correcting that -- not sure where my mind was :)

That 134MB appears to be a 2nd partition, NOT an HPA (which wouldn't show up on the display) If you attach this drive to a PC, and go to Disk Management, you should be able to simply delete both of the existing partitions ... then have another look and see what it shows. Run a /dev/sdX (where X is the assigned letter for that drive) to see exactly what the sector count it. Then you'll know if there's an HPA or not.

Thanks for the update Justin => too bad it wasn't a nice, neat "Hey, I finally figured out that THIS fixes it" But good to know you're no longer seeing the same mysterious "errors" every time.

There's no real reason to change previous disks to XFS ... especially if they're essentially static (i.e. full of media files and rarely written to or modified) As for a standby drive -- not a bad idea; BUT if you don't have dual parity, I'd upgrade to that first. You can always overnight a new drive when one fails; and with dual parity your array will still be protected while you're waiting for it. [Of course a spare "on the shelf" is even better, since you can start the rebuild process immediately]

The mathematics of the 2nd parity calculation are more complex than the simple longitudinal parity used for the first parity calculation (which can be - and is - done via simple XOR instructions). But the math is really irrelevant -- what matters is that the 2nd parity drive provides a 2nd layer of fault tolerance => i.e. ANY two drives can fail and you won't lose any data. That does NOT, of course, mean you should ignore a drive failure and wait for a 2nd failure before replacing your failed drives !! What it means is that if a drive fails, and you get a replacement drive and do a rebuild, that a 2nd drive failure during that rebuild won't cause a problem with the rebuild -- it will still complete successfully. The simple fact is that with the very large sizes of today's drives, the bit error rates are such that it's moderately likely that you could indeed have a bit failure during a rebuild. Another factor is that many folks buy their drives in "bunches" -- so it's likely you have several drives of the same age ... and when one starts failing, another may not be far behind. Being able to sustain two failures without data loss is a MAJOR improvement in the reliability of the system, as long as you don't "push it" and wait for the 2nd failure before doing anything about a failed drive. This is the reason major data centers no longer use RAID-5 => they use RAID-6 instead, which is dual fault tolerant. As I noted earlier, regardless of the degree of fault tolerance your RAID array has, RAID is NOT a substitute for having your data backed up. But that's another topic ...

When you say "not recognized", do you mean it doesn't show in Windows Explorer (File Explorer in '10) ... or that it isn't shown in Disk Management? It's normal that it wouldn't show in Explorer; but if you go to Disk Management and it shows, then all you have to do is delete the existing (Linux) partition; and create a new volume for Windows. Just right-click on Computer (This PC in '10); select Manage; then click on Disk Management.

I interpret this to mean you do NOT have backups -- right? Without re-hashing why that's a bad idea, I will note that if you're not going to keep your data backed up, you should, as a minimum, upgrade the system to dual parity.

Another thing to note about BackBlaze => they basically buy the least expensive drives they can ... generally consumer grade units. In fact, after the flooding in Thailand several years ago they paid a "bounty" for folks to send in drives extracted from external units (which are often less expensive than bare drive), just to minimize their cost. When you have thousands of drives; a robust RAID setup; and don't mind the hassle of always having a few drives being rebuilt, that's not necessarily a bad business model. But I think if there was good reliability data for the drives targeted to more robust operations -- NAS and Enterprise units -- you'd find that these are much better choices for most of us.

... and I now carry a 128GB USB flash drive in my pocket (on my key ring) => that's 4,923 times as much storage as that first hard drive had; and it cost me about 1/100th as much