Should I Build my Array Now Before my SATA Controller Comes?

[JorgeB]

With turbo write enable writes will be as fast as the slowest disk as long as there are no controller bottlenecks, with only 5 disks, and if they are all similar characteristics, write speed would be practically the same as writing to a single disk without parity.

 

Cache is not useful for the initial transfer, as it would fill up quickly and you'd need to run the mover.

[Synaptix]

With turbo write enable writes will be as fast as the slowest disk as long as there are no controller bottlenecks, with only 5 disks, and if they are all similar characteristics, write speed would be practically the same as writing to a single disk without parity.

 

Cache is not useful for the initial transfer, as it would fill up quickly and you'd need to run the mover.

 

I'd still have to wait for Parity Sync to complete before turning on turbo write and copying the data over correct? I assume that with parity syncing/building, all the drives will be spinning anyway.

[JorgeB]

Yes, transfering data during a parity sync/check is a bad idea as it will slow down both operations considerably.

[Synaptix]

Such a great community here. Thank you everyone for all the help!

[Synaptix]

Out of curiosity, what happens if I build the array with parity, stop the array, shutdown the server, remove the parity drive, then boot up again without the parity drive?

 

Also, same scenario but instead of removing a parity drive, I remove a data drive from the array and boot the system without it?

 

[JorgeB]

Array won't start automatically (if autostart is enable) with a missing disk, if you start without parity current parity disk will be "forgot" and re-synced once re-assigned, starting without a data disk will result on that disk's contents being emulated by parity plus all other data disks, you can still access all its data (and even write although not recommended), disk will be rebuilt when the old disk (or a new one) is assigned to its slot.

[SSD]

Out of curiosity, what happens if I build the array with parity, stop the array, shutdown the server, remove the parity drive, then boot up again without the parity drive?

 

Also, same scenario but instead of removing a parity drive, I remove a data drive from the array and boot the system without it?

 

If you remove the parity drive, the parity drive gets marked with a red mark.

 

Your array is no longer protected, although you could continue to write to your individual disks with no protection.

 

You'd have to rebuild parity if you wrote to any disk - or use the trust parity to shoehorn the disk back in. If you shoehorn it back in, you'd need to run a parity check.

 

If you remove a disk, unRAID will see the disk as failed. unRAID would then simulate the disk - all of its data would appear valid and correct. But in the background, your parity and all data drives would be scrambling to gather the data to do the simulation, so you might notice a slowdown (buy many people don't). You'd need to rebuild that disk. While the disk is simulated, if any other disk fails, the simulation ends and both the disk you removed and the failed disk contents would be gone. The array would not even start.

[Synaptix]

I'm considering going dual parity since with all my hot swap bays, I will have space for 15 drives. I currently do not have 15 drives, but overtime, it will be slowly built to that. I think with that many drives, one parity disk is a bit too risky because the rebuild may take quite awhile (drives in array will range from 3-8TB) and a second drive failure would hurt.

 

The problem I'm seeing though is that if I decide to upgrade my parity drives (currently 6TB), I'm going to need to double-up and upgrade both which will be costly.

 

If I wanted to this route later, would I be able to just assign a drive later to be parity?

 

Decisions, decisions...

 

[JorgeB]

Yes, parity2 can be added at any time. 

[SSD]

I'm considering going dual parity since with all my hot swap bays, I will have space for 15 drives. I currently do not have 15 drives, but overtime, it will be slowly built to that. I think with that many drives, one parity disk is a bit too risky because the rebuild may take quite awhile (drives in array will range from 3-8TB) and a second drive failure would hurt.

 

The problem I'm seeing though is that if I decide to upgrade my parity drives (currently 6TB), I'm going to need to double-up and upgrade both which will be costly.

 

If I wanted to this route later, would I be able to just assign a drive later to be parity?

 

Decisions, decisions...

 

I am confused by this question, so let me try to explain and maybe you can reask if this doesn't hit it.

 

Parity needs to be equal or greater than the size of your largest data drive.

 

If your largest data drive is 4T, then you'd need one 4T drive for parity, and if you wanted dual parity, you'd need another 4T disk for that.

 

So let's say you have 5 4T drives. 3 would be data, 2 would be parity.

 

You need more space and you'd like to be able to move to 6T drives. So you buy a 6T drive and upsize one parity and have that 4T drive for data (4 4T drives data, 6T and 4T parity). And then you need more space, so you buy a second 6T drive and it replaces the second 4T parity, and you'd use that for data (5 4T drives data, 2 6T drives parity). Then you need more space and buy your third 6T drive. It is added as a data drive. (5 4T drives and 6T drive for data, and 2 6T drives parity).

 

8T is a pretty good deal right now. You can buy an 8T Seagate Archive drive for about $210. They work very well for unRAID and almost if not less than many 6T drives. $26/T is pretty good. I watched this forum a long time before I bought any, but just bought 2. Both precleared nicely and are in my array.

 

BTW, if at any time you decide - "you know what, I don't need dual parity", you can always remove the second parity and add as a data disk. But I think every new user should have dual parity, as it may save you from a self-inflicted wound (which are the most common here). I shot myself in the foot plenty of times learning - so that's not meant to be critical, its just the way it is. Takes time to learn the ropes. After your first crisis you'll be in a better position to decide if the extra parity cost is worth it.

[Synaptix]

I'm considering going dual parity since with all my hot swap bays, I will have space for 15 drives. I currently do not have 15 drives, but overtime, it will be slowly built to that. I think with that many drives, one parity disk is a bit too risky because the rebuild may take quite awhile (drives in array will range from 3-8TB) and a second drive failure would hurt.

 

The problem I'm seeing though is that if I decide to upgrade my parity drives (currently 6TB), I'm going to need to double-up and upgrade both which will be costly.

 

If I wanted to this route later, would I be able to just assign a drive later to be parity?

 

Decisions, decisions...

 

I am confused by this question, so let me try to explain and maybe you can reask if this doesn't hit it.

 

Parity needs to be equal or greater than the size of your largest data drive.

 

If your largest data drive is 4T, then you'd need one 4T drive for parity, and if you wanted dual parity, you'd need another 4T disk for that.

 

So let's say you have 5 4T drives. 3 would be data, 2 would be parity.

 

You need more space and you'd like to be able to move to 6T drives. So you buy a 6T drive and upsize one parity and have that 4T drive for data (4 4T drives data, 6T and 4T parity). And then you need more space, so you buy a second 6T drive and it replaces the second 4T parity, and you'd use that for data (5 4T drives data, 2 6T drives parity). Then you need more space and buy your third 6T drive. It is added as a data drive. (5 4T drives and 6T drive for data, and 2 6T drives parity).

 

8T is a pretty good deal right now. You can buy an 8T Seagate Archive drive for about $210. They work very well for unRAID and almost if not less than many 6T drives. $26/T is pretty good. I watched this forum a long time before I bought any, but just bought 2. Both precleared nicely and are in my array.

 

BTW, if at any time you decide - "you know what, I don't need dual parity", you can always remove the second parity and add as a data disk. But I think every new user should have dual parity, as it may save you from a self-inflicted wound (which are the most common here). I shot myself in the foot plenty of times learning - so that's not meant to be critical, its just the way it is. Takes time to learn the ropes. After your first crisis you'll be in a better position to decide if the extra parity cost is worth it.

 

You pretty much nailed it. So to clarify, if I'm using dual parity and decide to to upgrade my parity drives, I don't have to upgrade both at the same time. I can upgrade one parity drive, and then later on, upgrade my second parity drive if I wish?

 

Also, I'm ready to build my first array with dual parity and when clicking start, it warns me with "Dual parity valid requires ALL disks in their original slots". Since this is my first array, technically the state of the current drives are the "original" right? Keep in mind once I receive my controller card and hot swap bays, the ports they're connected to will likely change.

[SSD]

You pretty much nailed it. So to clarify, if I'm using dual parity and decide to to upgrade my parity drives, I don't have to upgrade both at the same time. I can upgrade one parity drive, and then later on, upgrade my second parity drive if I wish?

 

Also, I'm ready to build my first array with dual parity and when clicking start, it warns me with "Dual parity requires all drives be placed in their original slots". Since this is my first array, technically the state of the current drives are the "original" right? Keep in mind once I receive my controller card and hot swap bays, the ports they're connected to will likely change.

 

Did you complete preclears? Do you want to post the smart reports so I can double check the drives are good?

 

unRAID requires you assign disks to slots. Your disks are NOT identified by controller and port - they are assigned by manufacturer and serial number. So moving disks to different ports and controllers has zero impact on unRAID. Once, long ago, this was different and more complicated. I and others requested that drives be assigned by manu and serial number, and Tom agreed and made the enhancement sometime in the unRAID 3.1/3.2 days I think.

 

What unRAID is telling you - is that once you assign a disk to an unRAID slot, with dual parity, you can't move it to another slot. Some people (me included), like to keep drives separated by content. For example, drives for TV Shows and drives for Movies. Say I have 1 drive for TV Shows (disk1),  and 2 drives for Movies (2,3). And then I fill up my TV show drive, I will want to add another TV show drive. Maybe my OCD says I want that to be disk2. So I'd like to move the Movie disks to slots 3 and 4, and put the new disk in slot2. (Does it matter one bit for user shares? No. But OCD doesn't have to make sense.  ) You can do that (with a little effort) with unRAID, but NOT with dual parity. You'd have to make the new disk slot4, leaving the disks in slots 1-3 alone. Unless you are willing to rebuild parity, that is. So you might say - I'd like to leave a gap for a second or even thrd TV Show disk, and therefore put my TV show disk as 1, leave 2 and 3 blank, and then have movies in 4 and 5. I do not have dual parity - so don't know if that is allowed or not. But if it is, that is an option.

[Synaptix]

You pretty much nailed it. So to clarify, if I'm using dual parity and decide to to upgrade my parity drives, I don't have to upgrade both at the same time. I can upgrade one parity drive, and then later on, upgrade my second parity drive if I wish?

 

Also, I'm ready to build my first array with dual parity and when clicking start, it warns me with "Dual parity requires all drives be placed in their original slots". Since this is my first array, technically the state of the current drives are the "original" right? Keep in mind once I receive my controller card and hot swap bays, the ports they're connected to will likely change.

 

Did you complete preclears? Do you want to post the smart reports so I can double check the drives are good?

 

unRAID requires you assign disks to slots. Your disks are NOT identified by controller and port - they are assigned by manufacturer and serial number. So moving disks to different ports and controllers has zero impact on unRAID. Once, long ago, this was different and more complicated. I and others requested that drives be assigned by manu and serial number, and Tom agreed and made the enhancement sometime in the unRAID 3.1/3.2 days I think.

 

What unRAID is telling you - is that once you assign a disk to an unRAID slot, with dual parity, you can't move it to another slot. Some people (me included), like to keep drives separated by content. For example, drives for TV Shows and drives for Movies. Say I have 1 drive for TV Shows (disk1),  and 2 drives for Movies (2,3). And then I fill up my TV show drive, I will want to add another TV show drive. Maybe my OCD says I want that to be disk2. So I'd like to move the Movie disks to slots 3 and 4, and put the new disk in slot2. (Does it matter one bit for user shares? No. But OCD doesn't have to make sense.  ) You can do that (with a little effort) with unRAID, but NOT with dual parity. You'd have to make the new disk slot4, leaving the disks in slots 1-3 alone. Unless you are willing to rebuild parity, that is. So you might say - I'd like to leave a gap for a second or even thrd TV Show disk, and therefore put my TV show disk as 1, leave 2 and 3 blank, and then have movies in 4 and 5. I do not have dual parity - so don't know if that is allowed or not. But if it is, that is an option.

 

Preclear looks good, are you referring to the preclear report from the preclear plugin? All of them look identical with the exception of the "194-Temperature_Celsius" and "9-Power_On_Hours" values in the "S.M.A.R.T. Status default" section. All other values in that section show a 0. And above that reads "SUCCESS" for Pre-Reading, Zeroing, Writing preclear signature, verifying preclear signature, and post-read verification.

 

If I understand correctly then, with dual parity, once I build this array with a specific drive assigned to a specific slot (for example, serial# AAAA in the "Disk 1" slot), that #AAAA hard disk can only be assigned to "Disk 1" and nothing else. If I wanted to stop the array, unassign #AAAA from Disk 1 and move #AAAA to Disk 5, I wouldn't be able to do that (unless I rebuild parity).

[garycase]

...  once I receive my controller card and hot swap bays, the ports they're connected to will likely change.

 

The physical ports you connect the drives to don't matter -- but you can't change the SLOT the drive is assigned to (e.g. Disk 1, Disk 2, etc.).

 

[Synaptix]

Looks like I ran into my first issue, after checking "Parity is already valid" and clicking "Start" to start the array, nothing happens. The status bar of the browser just says "Waiting for tower" and nothing else happens. I was running two preclears but they're not part of the array and I didn't think they would affect starting the array.

 

I tried reloading the browser as well as opening the web GUI in incognito mode and nothing is bringing it up. I am able to SSH into the server though. At this point, I'm a bit afraid to reboot it. I was watching the Getting Started videos and as soon as he clicked "Start" it refreshed quickly and it gave him the "Format" button.

 

Any suggestions?

[JorgeB]

You'll have to wait for the preclears to finish or kill them to get the webGUI back.

[Synaptix]

You'll have to wait for the preclears to finish or kill them to get the webGUI back.

 

No kidding, is this a bug with the preclear plugin? I can just killed it by going into the tmux sessions and ctrl+C them right?

[JorgeB]

There's was a time the plugin was optimized to allow array start and stop during a preclear, but something changed that it doesn't work anymore, you can kill a running preclear with:

 

tmux kill-session -t "preclear_disk_sdX"

[Synaptix]

There's was a time the plugin was optimized to allow array start and stop during a preclear, but something changed that it doesn't work anymore, you can kill a running preclear with:

 

tmux kill-session -t "preclear_disk_sdX"

 

That worked. Thanks!

[SSD]

Post the preclear before and afters. They should be in a folder on the flash - Preclear_reports I think. There should be several files for each precleared drive.

 

The deltas do not tell the whole story.

[Synaptix]

Post the preclear before and afters. They should be in a folder on the flash - Preclear_reports I think. There should be several files for each precleared drive.

 

The deltas do not tell the whole story.

 

Just checked \\tower\flash\preclear_reports and the only .txt files in there are the delta preclear reports. No before and afters. I'm using the preclear plugin.

 

Also, I just finished preclearing two drives and I'm ready to add them to the array. When I stop the array and assign the drives to new slots, can I check that the Parity is already valid since I'm inserting two zeroed drives?

 

I'm primarily asking because if a sync is required, I might as well wait since I'll be receiving my controller and hot swap cages today. And it'll allow me to fully complete my build, and then perform the sync.

[JorgeB]

Also, I just finished preclearing two drives and I'm ready to add them to the array. When I stop the array and assign the drives to new slots, can I check that the Parity is already valid since I'm inserting two zeroed drives?

 

Parity remains synced when any precleared disk can is added to the array.

[Synaptix]

Also, I just finished preclearing two drives and I'm ready to add them to the array. When I stop the array and assign the drives to new slots, can I check that the Parity is already valid since I'm inserting two zeroed drives?

 

Parity remains synced when any precleared disk can is added to the array.

 

Great, thanks!

[Paul_Ber]

Post the preclear before and afters. They should be in a folder on the flash - Preclear_reports I think. There should be several files for each precleared drive.

 

The deltas do not tell the whole story.

My SATA ports are all full.  I want to preclear a larger drive but all I have is an USB 3 SATA drive dock.  Running the preclear plugin but because it is not really a direct SATA port, there is no smart Data.

 

Am I wasting my time preclearing with the USB 3 SATA HD dock?

############################################################################################################################ # # # unRAID Server Preclear of disk /dev/sdh # # Cycle 1 of 1, partition start on sector 64. # # # # # # Step 1 of 6 - Pre-read verification: [9:48:50 @ 141 MB/s] SUCCESS # # Step 2 of 6 - Erasing in progress: (12% Done) # # # # # # # # # # # # # # # # # # ** Time elapsed: 0:57:30 | Write speed: 179 MB/s | Average speed: 178 MB/s # # # ############################################################################################################################ # Cycle elapsed time: 10:46:25 | Total elapsed time: 10:46:26 # ############################################################################################################################ ############################################################################################################################ # # # S.M.A.R.T. Status default # # # # # # ATTRIBUTE INITIAL STATUS # # 5-Reallocated_Sector_Ct 0 - # # 9-Power_On_Hours 7391 - # # 194-Temperature_Celsius 32 - # # 196-Reallocated_Event_Count 0 - # # 197-Current_Pending_Sector 0 - # # 198-Offline_Uncorrectable 0 - # # 199-UDMA_CRC_Error_Count 0 - # # # # # # # # # # # ############################################################################################################################ # SMART Status not supported: Incomplete response, ATA output registers missing # ############################################################################################################################

 

 

Sent from my Pixel XL using Tapatalk

 

[SSD]

Post the preclear before and afters. They should be in a folder on the flash - Preclear_reports I think. There should be several files for each precleared drive.

 

The deltas do not tell the whole story.

My SATA ports are all full.  I want to preclear a larger drive but all I have is an USB 3 SATA drive dock.  Running the preclear plugin but because it is not really a direct SATA port, there is no smart Data.

 

Am I wasting my time preclearing with the USB 3 SATA HD dock?

############################################################################################################################ # # # unRAID Server Preclear of disk /dev/sdh # # Cycle 1 of 1, partition start on sector 64. # # # # # # Step 1 of 6 - Pre-read verification: [9:48:50 @ 141 MB/s] SUCCESS # # Step 2 of 6 - Erasing in progress: (12% Done) # # # # # # # # # # # # # # # # # # ** Time elapsed: 0:57:30 | Write speed: 179 MB/s | Average speed: 178 MB/s # # # ############################################################################################################################ # Cycle elapsed time: 10:46:25 | Total elapsed time: 10:46:26 # ############################################################################################################################ ############################################################################################################################ # # # S.M.A.R.T. Status default # # # # # # ATTRIBUTE INITIAL STATUS # # 5-Reallocated_Sector_Ct 0 - # # 9-Power_On_Hours 7391 - # # 194-Temperature_Celsius 32 - # # 196-Reallocated_Event_Count 0 - # # 197-Current_Pending_Sector 0 - # # 198-Offline_Uncorrectable 0 - # # 199-UDMA_CRC_Error_Count 0 - # # # # # # # # # # # ############################################################################################################################ # SMART Status not supported: Incomplete response, ATA output registers missing # ############################################################################################################################

 

 

Sent from my Pixel XL using Tapatalk

 

Not sure. Just because the smart reports don't work, doesn't mean that the preclear will not work. Smartctl has a bunch of command line options including support for some usb chipsets. I've mucked with it and gotten smart output from certain USB drives. If the USB does not muck with the drive's geometry, it should work. If you can put a desktop drive from your computer  in there and it could read it, I'd bet it'll work. UPDATE - only fear is that the script itself may, due to the incomplete SMART reports, fail the drive. I doubt it, but if that happened the drive would not have a valid preclear signature (but the disk would nonetheless be zeroed). If that happens, I have a solution to try to avoid having to preclear it again.