Adding pre cleared & formatted disk to an array

[Prashant]

Hello all, 

I'm first time user. With the help of all you guys & especially spaceinvader videos, I have successfully set up my unraid from scratch. Presently using trial key but will be purchasing plus key today as my unraid is successful. 

My configuration is as follows; 

Intel i7 8700 8th generation CPU

Asus rog strix H370 F mobo

Corsair vengeance 16GB 2400 mHz RAM single module 

750 watts 80plus bronze modular PSU

cooler Master K380 case

Zotac GTX 1060 GPU

KINGSTON A400 SSD 240GB dedicated for windows 10 Pro (dual boot application) 

KINGSTON A400 SSD 240 GB for cache

3 X 2TB IRONWOLF for array (1 parity + 2 data disks) 

SanDisk ultra fit 3.1 USB Flash 32 GB for OS

It's my DIY project & only few days with system. 

My first question or rather observation in forum is as follows: Yesterday I added one 80GB Hitachi HDD, using Krusader, transferred useful data to newly created share, using unassigned device plugin removed the old partions, pre cleared it & formatted in XFS file system. I had read somewhere that such drive can be added to array without needing to clear it format again. But when I added this disk to array, it was put to clearing operation & post that was available as unmountable needing to be formatted again to be part of array. Without much thinking I completed the process. Something went wrong in my procedure? Is it normal? Repeating of clear & format function. 

Now I'm struggling with my first VM. I have ordered Ziyituod SATA3.0 Card, 6 Port pcie card for expansion. I understand that I can seamlessly shift my array drives to this card as unraid remembers the configuration by disk serial numbers and I need not do any additional steps, Please confirm my understanding. Also is it right choice for this card? 

Thank you all in advance & for my successful unraid built. 

[JorgeB]

1 hour ago, Prashant said:

I had read somewhere that such drive can be added to array without needing to clear it format again.

It still needs to be formatted, it doesn't need to be cleared.

 

If the drive was cleared it means the preclear didn't work correctly, and if you want you can post about it on the appropriate support thread, preclear plugin, preclear docker, etc.

[Prashant]

Thanks for clarifying & guidance. 

[Frank1940]

4 hours ago, Prashant said:

Yesterday I added one 80GB Hitachi HDD, using Krusader, transferred useful data to newly created share, using unassigned device plugin removed the old partions, pre cleared it & formatted in XFS file system.

When you did this, you eliminated the preclear status.  I.e., a precleared disk has only zeros written to every byte on the data area of the disk except for a special precleared signature.  Formatting a disk writes a lot of data to a disk.

 

Let's realize what happens when you add a new disk (straight out of the packing) to Unraid.  It first writes zeros to every byte on the disk.  (This is Unraid's clearing process.)  This assures that Parity1 is valid.  (At this point, I am not sure how Parity2 become valid.)  It now adds the disk to the array and the disk must next be formatted which adds a fair amount of data to the disk to built the structure of the file system.  As it writes to the formatting information to the disk, parity is updated for each and every byte written to that disk as it will be for every byte written thereafter. What happens is that if you take a disk straight from the preclear process and add it to the array, it looks for and finds the preclear signature.  It will now skip its clearing process, adds the disk to the array and begins the formatting process, writing the parity information as it does so.

 

Why a preclear script?  Well, way back in olden times, Unraid took the entire array off-line while it did the clearing process.  This wasn't long for an 80GB HD but as hard drives got larger, it became a real issue.  Hence, the first preclear script appeared back in 2008.  But the script did much more than just clear the disk.   It also tested every byte on the disk.  Back then, hard disks were not as reliable as they are today.   Early failures were more of a problem.  Folks started wanting to extend testing for 50 to 100 hours which would catch most disks that were going to fail early in their lives.  (I can remember back in the 1990's every computer manufacturer burned in their systems for 72 hours before shipping to catch this type of failure!)  The preclear script was perfect for this type of burn-in and testing.  The preclear element of the preclear scripts is now a non-issue (as the array is not taken down while it runs) but the testing part is still with us.

[JorgeB]

Just now, Frank1940 said:

When you did this, you eliminated the preclear status.  I.e., a precleared disk has only zeros written to every byte on the data area of the disk except for a special precleared signature.  Formatting a disk writes a lot of data to a disk.

Good catch, I missed that, that explains it.

[trurl]

46 minutes ago, Frank1940 said:

(At this point, I am not sure how Parity2 become valid.) 

A clear disk also has no impact on parity2. Same as parity1. All zeros doesn't affect the parity2 result either.

[trurl]

6 hours ago, Prashant said:

added one 80GB Hitachi HDD

Do you mean you are putting this tiny (and I assume very old) drive in your array? Why?

[Prashant]

6 minutes ago, trurl said:

Do you mean you are putting this tiny (and I assume very old) drive in your array? Why?

Basically being new user, I am just exploring things & get hands on experience of the features. I'll use it for storing data which perhaps is not important. I used Krusader, unassigned device, pre clear etc. On this hdd, till date which I had only read about it. Also with this disk I'll explore replacing disk etc. feature. 

[trurl]

1 minute ago, Prashant said:

I'll use it for storing data which perhaps is not important.

OK temporarily for testing and for experimenting, but not OK for storing data which isn't important.

 

Every disk in the array is important regardless of what data is on it. In order to reliably rebuild every bit of a disk, every bit of parity PLUS every bit of ALL other disks must be reliably read. So even if you don't care about the data on a disk, it is still important to the redundancy of your array.

[Prashant]

@Frank1940, Now it's "clear" to me. If have to add disk to array then I'll just pre clear it for obvious reasons you mentioned above. Any if I plan to use it as unassigned device then only format it, post pre clear. 

[trurl]

1 minute ago, trurl said:

OK temporarily for testing and for experimenting, but not OK for storing data which isn't important.

Even if the disk passed preclear and SMART is good I wouldn't use it as a permanent member of the array. You should only install as many disks as you need for capacity. Each additional disk is an additional point of failure. That disk doesn't have enough capacity to waste an extra slot and extra port on.

[Prashant]

2 minutes ago, trurl said:

OK temporarily for testing and for experimenting, but not OK for storing data which isn't important.

 

Every disk in the array is important regardless of what data is on it. In order to reliably rebuild every bit of a disk, every bit of parity PLUS every bit of ALL other disks must be reliably read. So even if you don't care about the data on a disk, it is still important to the redundancy of your array.

Thanks for valuable tip, well noted.  Since it is not having data, can remove it from array & is there way to tell config just to forget & rebuild parity without it? 

[trurl]

2 hours ago, Prashant said:

is there way to tell config just to forget & rebuild parity without it? 

Tools - New Config will let you assign disks however you want and optionally (by default) rebuild parity.

 

Keep all assignments then unassign that disk and start the array. You must let it rebuild parity since an empty disk is not a clear disk as you have learned, so parity will not be valid without that disk until it is rebuilt.