Help: Array stuck in "copy" mode, after using swap method to add a larger parity drive

[carlosliu]

Hello,

 

I'm following the instruction of "The parity swap procedure" to add a larger new parity drive, and move the old parity drive to a data drive. https://wiki.unraid.net/The_parity_swap_procedure#:~:text=This procedure is strictly for,building parity will immediately begin.

 

After finishing the 27 hour "copy" process, the array status show "Stopped. Upgrading disk/swapping parity" which is expected according to the document.

 

However, it does not show the "START" button. Instead, it is still presenting the "COPY" button with copy information/checkbox.

 

Screenshot below

 

 

Quickly checked the system log. Nothing out of ordinary.

 

I'm a bit stuck now, not sure what to do next. Any help would be appreciated, thanks.

 

--

Carlos

[trurl]

Attach Diagnostics to your NEXT post in this thread 

[carlosliu]

Thanks. Please see diagnostics in attachment

bongo-diagnostics-20230510-1749.zip

[carlosliu]

I accidentally reboot the server.

 

Now, the array is reverted to the same state before copying, since no data were wrote to any data drive and old parity drive.

 

I guess I will simply do two-steps upgrade, first upgrade the parity drive, then data drive.

 

Thanks for helping anyway.

[trurl]

3 hours ago, carlosliu said:

two-steps upgrade, first upgrade the parity drive, then data drive

That is the best way anyway. Parity swap is really for those situations where you already have a data disk that needs replacement, but you want to buy a disk larger than parity.

[chansearrington]

@trurl I just experienced the same.



I added a new drive (sdx) to be the new parity drive and assigned me old parity (SDC) to a failed drive spot.

I then waited for 2 days for the copy parity to complete but the start button doesn't show like it says in the docs, just as @carlosliu highlighted


Here is my diagnostic file.
cronos-diagnostics-20231216-2344.zip

[JorgeB]

According to the screenshot copy was not done or it was interrupted, note that after the copy is done you cannot do anything else with the GUI other than start the rebuild, or it will interrupt the parity swap.

[trurl]

Problems reading both disks 4 and 5, both look like disk problems not connection problems, but SMART for 4 is the worst of the two. Disk4 also unmountable.

 

I didn't notice anything in syslog to indicate parity copy ever started, but I probably just missed it.

 

Screenshot seems to be waiting for you to check the box to enable the copy button.

[chansearrington]

@JorgeB it for sure was done. This is what the screen showed after it hit 100%

[chansearrington]

57 minutes ago, trurl said:

Problems reading both disks 4 and 5, both look like disk problems not connection problems, but SMART for 4 is the worst of the two. Disk4 also unmountable.

 

I didn't notice anything in syslog to indicate parity copy ever started, but I probably just missed it.

 

Screenshot seems to be waiting for you to check the box to enable the copy button.

@trurl / @JorgeB So Sorry. I uploaded the diagnostics of the wrong Unraid Server.

Here's the correct one. 

the-ark-diagnostics-20231217-0745.zip

[trurl]

1 hour ago, chansearrington said:

uploaded the diagnostics of the wrong Unraid Server

You mean your other Unraid server has all those problems I mentioned?

2 hours ago, trurl said:

Problems reading both disks 4 and 5, both look like disk problems not connection problems, but SMART for 4 is the worst of the two. Disk4 also unmountable.

 

[trurl]

Do you have Notifications configured to alert you immediately by email or other agent as soon as a problem is detected?

8 minutes ago, trurl said:

You mean your other Unraid server has all those problems I mentioned?

Don't let one unnoticed problem become multiple problems and data loss.

 

Do any disks on either server show SMART warnings on the Dashboard page? Disks 4 and 5 on the other server definitely should, I haven't examined SMART for the large number of disks on the server that is the topic of this thread.

[chansearrington]

55 minutes ago, trurl said:

You mean your other Unraid server has all those problems I mentioned?

 

Yes, that’s another server (Cronos). I’m working on fixing that one as well.
 

But the server that I experienced this bug on is “the Ark” ignore “Cronos” for this bug thread.

 

on “the Ark”, I  followed the docs for parity swap to add a larger new parity drive, and move the old parity drive to replace another failing drive.
 

https://docs.unraid.net/legacy/FAQ/parity-swap-procedure/

 

After a ~48 hour "copy" process, I watched the copy process go from 98 to 100%. Then a couple minutes later the array updated to show "Stopped. Upgrading disk/swapping parity" which is expected according to the document.

 

However, it does not show the "START" button. Instead, it is still presenting the "COPY" button with copy information/checkbox. (See screenshot above)

[trurl]

During the copy, I see lots of "critical medium errors" on multiple disks.

 

Let me ask my unanswered question another way.

 

Which of the many disks on your ark server have SMART warnings on the Dashboard page?

[chansearrington]

Only one.

[trurl]

New Disk 18, sdc, is just CRC (connection) error. I usually just acknowledge the occasional CRC error, maybe reseat the cable, investigate further if they increase rapidly. Many of your drives have CRC errors that you must have already acknowledged and they haven't increased since.

 

Is that unassigned Dev1 sds? Looks like the drive that was originally sdt, serial ending 1413, when you booted disconnected and reconnected as sds since it now has that serial and there is no sdt connected. It was showing critical medium errors all thru syslog including during copy. That one has pending sectors.

 

The other drive that was throwing critical medium errors was sdo but not immediately clear which drive that was at the time since sdo is in syslog with different serial numbers at different times. sdo started out as disk18 with serial ending 7227, but sdo was unassigned with serial ending 2P9T when the diagnostics were taken.

 

Obviously original disk18 was part of the parity swap, but wouldn't have been read during parity copy nor used during disk18 rebuild.

 

Doesn't look like original disk18 is still connected but it was when you booted.

 

Have you been doing "hotswap" during any of this?

 

Were any of these other disks I mention involved in the parity swap?

 

 

 

 

[JorgeB]

Parity copy looks to have finished successfully, but then parity was showing as wrong, suggesting the capacity changed, I would reboot first and try again.

[chansearrington]

@JorgeB Thanks. I restarted and I'm currently running the job again. I'll post back when it's finished. Currently at 57%