How to recover or re-do drive ordering after creating a new config and replacing cache drives

[chansearrington]

Hoping someone can confirm my assumptions or point me in the right direction:
 

• System setup
  • Array
    • 2x Parity Drives (each 6TB)
    • 6x Drives (each 6TB)
  • Cache
    • 2x Cache Drives
       
• Background
  • I needed to upgrade my cache drives
  • To do so, I:
    • turned off docker
    • turned off VMs
    • Moved everything off of cache drives to arrary drives by setting all cache settings to "yes"
    • Stopped the array
    • Created a new config
      • Preserved assignments for Arrary Drives
      • did NOT preserve cache drives assignments
    • New config correctly showed new all original array devices in the correct order
    • New confirg showed old cache drives as being un-assigned
    • turned off the machine and installed the new cache drives
    • Turned on the machine
    • Config now shows ALL devices being unassigned
    • Parity drives show as "unmountable" and unassigned
    • Putting them back into the array shows that "All existing data on this device will be OVERWRITTEN when array is started"
    • I DID NOT screenshot my device order (noob mistake I won't make again)
       
• Questions
  • How do I put all these drives back the way they were?
    • Does it even matter?
  • Do I care that my parity drives will be overwritten?
  • I know two of the drives are parity (the show as greyed out / unmountable based on other forum posts), but I don't know which one is parity 1 vs parity 2

Devices before I assign them:


 

Devices after I assign them:

[JonathanM]

Do you have a flash backup from before you set the new config with all your current drives?

[itimpi]

1 hour ago, chansearrington said:

• New config correctly showed new all original array devices in the correct order
• New confirg showed old cache drives as being un-assigned
• turned off the machine and installed the new cache drives

You missed an important step.  You should have started the array after the assignments are as you wanted them.   After doing a new Config all drives are deemed uncommitted until the array is started.  That is required to commit them to the flash drive.  

 

1 hour ago, chansearrington said:

How do I put all these drives back the way they were?

If you do not have a record of how they were set up then unless you can tell from their contents you are just guessing at the drive order.

 

1 hour ago, chansearrington said:

• Do I care that my parity drives will be overwritten?
• I know two of the drives are parity (the show as greyed out / unmountable based on other forum posts), but I don't know which one is parity 1 vs parity 2

Since parity1 and parity2 use different algorithms so they are not interchangeable, and getting even one drive on the wrong order would invalidate parity2 anyway there is not going to be much alternative other than to rebuild both parity drives.   This will be fine as long as no array drive fails before parity is rebuilt as you are not protected until it has completed.

 

 

 

 

[chansearrington]

26 minutes ago, JonathanM said:

Do you have a flash backup from before you set the new config with all your current drives?

@JonathanM No, I don't. 😞

 

[chansearrington]

3 minutes ago, itimpi said:

You missed an important step.  You should have started the array after the assignments are as you wanted them.   After doing a new Config all drives are deemed uncommitted until the array is started.  That is required to commit them to the flash drive.  

Ah, ok. Got it.

 

3 minutes ago, itimpi said:

Since parity1 and parity2 use different algorithms so they are not interchangeable, and getting even one drive on the wrong order would invalidate parity2 anyway there is not going to be much alternative other than to rebuild both parity drives.   This will be fine as long as no array drive fails before parity is rebuilt as you are not protected until it has completed.

@itimpi Just to make sure I understand...

The data in the parity drives isn't essential to keep all the data in the other drives.

So put the drives in any order and rebuild both parity drives

All the data in the other non-parity drives will be fine as long as none of them fail during the parity rebuild?
 

[itimpi]

8 minutes ago, chansearrington said:

The data in the parity drives isn't essential to keep all the data in the other drives

Yes - parity is only important after a drive fails to reconstruct its contents.

 

8 minutes ago, chansearrington said:

So put the drives in any order and rebuild both parity drives

All the data in the other non-parity drives will be fine as long as none of them fail during the parity rebuild?

Yes.  
 

The key thing is to not have any data drive accidentally assigned as a parity drive as that will end up losing its contents.

 

if any share have includes/excludes for specific drives these will need to be checked (and if needed corrected). 
 

[chansearrington]

@itimpi Got it! Thank you!

[chansearrington]

Just following up here. Everything turned out fine with this. All data was still there and the arrays are done rebuilding.