Oops during parity swap, please advise

[Timbiotic]

I got done with the parity swap procedure but wanted to format the old parity drive so i unassigned it before starting the array.  Now the array wont start because of missing disks, but if i put the disk back in, it wants to run the copy again.  Even though it is complete.  Is there a way I can force it to start and not copy again?

[trurl]

I know you had already marked your other thread solved, but it would have been better to mark it unsolved and just continued there. For anyone following along here is a link:

https://lime-technology.com/forums/topic/69516-solved-hung-gui-during-parity-swapupgrade/

 

I already commented in that other thread that you didn't need to use the parity swap anyway, but if you are going to do so, why did you think you should try to format it in the middle of the process?

 

You should be very careful in the future when the idea of formatting a disk occurs to you. Many people have lost data that way simply because they didn't understand what the word "format" means. In every operating system you have ever used, "format" means "write an empty filesystem to this disk". When you are rebuilding that is never appropriate. In the usual rebuilding scenario, if you format a data disk, unRAID will treat it the same way it treats all writes to the array, by updating parity so that it agrees that the disk you are rebuilding has an empty filesystem on it.

 

Now your scenario is slightly different, but format is still inappropriate. Maybe we should just start from scratch and decide whether there is a more straightforward way to accomplish your ends.

 

In your other thread, I believe you said that you had already moved all the data off the original data disk. So, there is really nothing to be gained by keeping your array configuration, copying parity to a new disk, and rebuilding the (empty) data disk to the old parity.

 

If this is accurate, then the simple thing to do would be to set a New Config, assign the new parity disk to the parity slot, assign the old parity disk to the data slot you want to replace, sync parity, and format the (former parity) data disk.

 

Please respond before you actually do anything so we can make sure we both understand the situation.

[itimpi]

Why did you want to format it?    

[Timbiotic]

I am glad you responded cause I have my finger on the new config using "preserve all disks" hoping it would make it so it can start again and use the newly copied 4tb as parity.

 

I wanted to format the old parity with no data because I want to convert from reiser to xfs. I thought just unassigning would allow me to do this.  I should have started the array first i guess.

 

I wanted to do the parity swap so my new parity doesn't need to be rebuilt from scratch

Edited February 19, 2018 by Timbiotic

[itimpi]

To change the file system you simply stop the array, click on the disk to change the file system type and restart the array.  At that point you are offered the option to format the disk.   Doing it that way preserves parity.

[trurl]

1 minute ago, Timbiotic said:

I am glad you responded cause I have my finger on the new config using "preserve all disks" hoping it would make it so it can start again and use the newly copied 4tb as parity.

 

I DO want to format the old parity with no data because I want to convert from reiser to xfs. I thought just unassigning would allow me to do this.  I should have started the array first i guess.

 

I DO want to do the parity swap so my new parity doesn't need to be rebuilt from scratch

In order to do the parity swap, you are going to have to start over with the process, let it copy parity, let it rebuild the (empty) reiser filesystem to the data slot you replaced with the old parity, then format that empty disk as xfs.

 

There is not really any good reason to do parity swap unless you already have a disabled disk. It is a lot more trouble, which means more opportunities for mistakes (as we have seen), and doesn't really save any time or effort for you or unRAID.

[Timbiotic]

So doing a new config wouldn't save time? 

Is copy faster than rebuilding parity? 

 

I also have two other 4tb disk, and a 3tb disk,  i want to swap another three 1tb disks with.  I also responded in the other thread i did have a disk i removed.  I am in process of swapping out my 1tb drives.  I just wanted as little array downtime as possible.  Hence unbalance.  It lets me clear disks and keep the array up.  Am I going about this wrong?

 

Basically i want little downtime, I also want to upgrade 4 x 1tb drives (already pulled one in the parity swap procedure).

 

My plan was to add, unbalance, add, unbalance, etc... But when i unbalance i never move the data to the 1tb drives. Is there a more efficient way of doing this?

Edited February 19, 2018 by Timbiotic

[trurl]

Just now, Timbiotic said:

thanks so what should i do now?  New config preserve all disks?  Or is there no way to make the parity copy i did overnight keep without a rebuild?

At this point I wouldn't trust what you have already done. I can maybe think of ways to keep the parity copy, but I hesitate to advise since I think it may be better to go with a simpler approach. Anything I suggested would be off the top of my head based on my understanding and not based on anything I have actually done. And it probably wouldn't really save time or effort.

 

I am going to respond to your newer post separately. Wait for it.

[JonathanM]

Do you have enough room currently on your array disks to empty all your 1TB drives at once? If so, the fastest way is to remove all the 1TB drives, put all the drives in that you want to use in the final layout, and set a new config with that layout. When you start the array, it will build parity and allow you to format the blank drives. The array will be online and available the entire time, and protected after parity is done.

[trurl]

jonathanm has a good plan for you. Let us know if it fits your scenario, and if you have any questions along the way.

[Timbiotic]

2 minutes ago, jonathanm said:

Do you have enough room currently on your array disks to empty all your 1TB drives at once? If so, the fastest way is to remove all the 1TB drives, put all the drives in that you want to use in the final layout, and set a new config with that layout. When you start the array, it will build parity and allow you to format the blank drives. The array will be online and available the entire time, and protected after parity is done.

 

This sounds good, but i cannot wrap my mind around it.

 

Are you saying unbalance all of the 1tb drives now, then remove them?    Currently my array will not start unless i run the parity copy again, or set a new config. 

 

So in your scenario:

 

1. Run New config and choose "preserve all drives"

2. Start Array

3. Unbalance all 1tb drives

4. stop array and shut down

5. replace 1tb drives with new drives

6. Start array and do a new config? 

 

Would it make sense to disable parity completely when unbalancing for speed since it will need to be rebuilt?

[JonathanM]

BTW, all this moving data around and one disk at a time stuff is MUCH riskier to your data than running without valid parity for a day or two. It's too easy to fat finger a command or click something you shouldn't vs. the probability of a drive failing.

 

Also, instead of moving the data off of the 1TB drives, you would be much better served to copy it, and leave the originals intact. That way if there is an oops, you can recover from the original drives.

 

When in doubt, copy data and delete after confirmation of a good copy, or keep the data intact for a backup.

[JonathanM]

2 minutes ago, Timbiotic said:

Are you saying unbalance all of the 1tb drives now, then remove them?    Currently my array will not start unless i run the parity copy again, or set a new config. 

No, ideally you would copy the contents then remove them.

 

Are all your disks healthy? I haven't noticed whether you posted diagnostics or not. The SMART health status of your disks is important to determine whether we need to get you back protected before proceeding. If all your disks are in perfect shape I would be ok running without parity for the copy operations.

[trurl]

2 minutes ago, jonathanm said:

No, ideally you would copy the contents then remove them.

 

Are all your disks healthy? I haven't noticed whether you posted diagnostics or not. The SMART health status of your disks is important to determine whether we need to get you back protected before proceeding. If all your disks are in perfect shape I would be ok running without parity for the copy operations.

Agree it would be a good idea to get diagnostics just in case he is unaware of disk issues.

 

While it might be OK to delay building parity, I don't see any good reason to do so. It is going to have to be rebuilt in any case.

[trurl]

Instead of using unBalance to get the data off the 1TB disks, you could just resync with all the new disks in place, then mount the 1TB disks with Unassigned Devices and copy the data from them that way. I don't think unBalance will work with Unassigned Devices but there are lots of other ways of copying the data.

[Timbiotic]

Quote

Instead of using unBalance to get the data off the 1TB disks, you could just resync with all the new disks in place, then mount the 1TB disks with Unassigned Devices and copy the data from them that way. I don't think unBalance will work with Unassigned Devices but there are lots of other ways of copying the data.

 

 

I cannot fit all the disk in the case.  So I am replacing disks. I guess i could rsync to the current other 3tb disks first, but unbalance seems easier and checks permissions etc.. for potential issues. 

 

29 minutes ago, jonathanm said:

BTW, all this moving data around and one disk at a time stuff is MUCH riskier to your data than running without valid parity for a day or two. It's too easy to fat finger a command or click something you shouldn't vs. the probability of a drive failing.

 

Also, instead of moving the data off of the 1TB drives, you would be much better served to copy it, and leave the originals intact. That way if there is an oops, you can recover from the original drives.

 

When in doubt, copy data and delete after confirmation of a good copy, or keep the data intact for a backup.

 

I have backups of my critical stuff, but if I am curious about this.  If i copy using unbalance how does that work with the shares?  With duplication on disks?  I mean I will remove the old disks eventually but while there is a copy wont that cause issues?

 

I have uploaded diagnostics.  The only issues i know of is the data disk i already pulled had 3 CRC errors.  Hence me getting rid of it first.  Also since upgrading to 6.4.1 and using the new ip docker feature i have had call traces and kernel panics.  It seems a few others have had the same issue.  I have opened a bug report on it.

lillis.69.mu-diagnostics-20180219-1039.zip

Edited February 19, 2018 by Timbiotic

[JorgeB]

16 minutes ago, Timbiotic said:

The only issues i know of is the data disk i already pulled had 3 CRC errors.  Hence me getting rid of it first. 

CRC errors are a connection issue, not a problem with the disk, usually a bad SATA cable.

[Timbiotic]

1 minute ago, johnnie.black said:

CRC errors are a connection issue, not a problem with the disk, usually a bad SATA cable.

Good to know! Do they stick with the disk smart history permanently? Because I havent seen any new ones in the life of that disk.  Either way i replaced a SATA with a SAS so all new cables.

[JorgeB]

Just now, Timbiotic said:

Do they stick with the disk smart history permanently?

Yes, it never resets, as long as it doesn't increase the problem was fixed.

[Timbiotic]

I got array started using new config and preserve all disks.

I then unassigned the parity

I then used itimpi's suggestion to format the old parity with XFS

I am now copying all of the remaining 1tb drives to the formatted old parity drive.

When this is finished i will shut down and swap out the 1tb for the larger drives

 

Before i add back the parity  drive and resync, is there something i should do first to make sure the copied data is good?  Or does unbalance include this?  I will have the old drives on a shelf in case I guess.

 

Thanks for all of the assistance, i LOVE this community.