Data drive failure while rebuilding parity

[eggsbenedict]

I'm in the process of upgrading my array of 3TB drives to 12TB drives. I started by shutting the system down and removing the old 3TB parity disk, then installing the 12TB disk in it's place. I booted back up and assigned the new 12TB disk as the parity drive, and started the parity sync. I then left the house. An hour or so into the parity sync, a data drive started spitting out errors, and the parity sync was cancelled. I came home to find the data disk showing "disk is disabled, contents emulated" and the parity drive having a yellow triangle "parity sync invalid". I've stopped the array, but haven't touched anything else. Wondering what my next move should be?

 

I still have the old 3TB parity drive sitting untouched since removing it this morning.

 

I believe at some point after this happened, but before I came home, the mover script ran and put some files from the cache disk onto a data drive. Not sure if that affect anything. The data that was moved just now is entirely uncritical.

 

I'm using version 6.5.0, and I've got the diagnostics zip file attached.

 

Thanks for any help anyone can offer, trying not to panic!

coop-diagnostics-20210423-1511.zip

[eggsbenedict]

I've removed the disabled data drive and plugged it into my Windows desktop. I've backed up my critical data from this drive. Everything seems to be fine, with the exception of the handful of irrelevant media files that Unraid attempted to write to this disk during the parity sync. When removing the drive from the server, the SATA cable rally didn't seem like it had much of a "grip" on the drive, it might have been nudged loose while swapping the parity drives earlier.

 

I'm thinking in order to safeguard all other data ASAP, I will keep this drive out of the server and do a "New Config", which will rebuild parity and protect my other 4 data drives? Then I can add this "failed" data disk back to the array as a new data disk, and move as many files as possible onto other drives, and then trash the disk.

 

 

[itimpi]

1 hour ago, eggsbenedict said:

Then I can add this "failed" data disk back to the array as a new data disk, and move as many files as possible onto other drives, and then trash the disk.

Do NOT add it as a new data drive as that would cause unRaid to ‘Clear’ it (by writing zeroes to every sector) before adding it to the array and thus erasing its contents.   From your description this is NOT what you want to happen?    It would also then leave you with the problem of how to remove the drive from the array once you had copied files off it.
 

 Instead simply mount the drive using Unassigned Devices and that will make it available for copying off the files onto the disks you will be keeping in the array.  After copying the files you can then simply unmount it without affecting the array contents and remove it.   I would recommend checking the drive can be mounted using UD before attempting the New Config + parity rebuild you mention justo make sure it will mount OK just in case there is some additional action (e.g a file system repair) required to get UD to mount it successfully.

 

it is also worth mentioning that there may be nothing much wrong with the disabled disk if it was simply a cable problem that caused it to be disabled by triggering a write error  as that is by far the commonest reason disks get disabled.

[eggsbenedict]

3 minutes ago, itimpi said:

Do NOT add it as a new data drive as that would cause unRaid to ‘Clear’ it (by writing zeroes to every sector) before adding it to the array and thus erasing its contents.   From your description this is NOT what you want to happen?    It would also then leave you with the problem of how to remove the drive from the array once you had copied files off it.
 

 Instead simply mount the drive using Unassigned Devices and that will make it available for copying off the files onto the disks you will be keeping in the array.  After copying the files you can then simply unmount it without affecting the array contents and remove it.   I would recommend checking the drive can be mounted using UD before attempting the New Config + parity rebuild you mention justo make sure it will mount OK just in case there is some additional action (e.g a file system repair) required to get UD to mount it successfully.

 

it is also worth mentioning that there may be nothing much wrong with the disabled disk if it was simply a cable problem that caused it to be disabled by triggering a write error  as that is by far the commonest reason disks get disabled.

Ok, thanks.

 

Is there any risk to my other data if I just hook the drive back up right away, and do a "new config", to see if it can rebuild parity without any failure? 

[itimpi]

Just now, eggsbenedict said:

Is there any risk to my other data if I just hook the drive back up right away, and do a "new config", to see if it can rebuild parity without any failure? 

No other than the fact of any stress put on the data drives. Building parity is a read-only operation as far as the data drives are concerned.

 

You probably first want to check the problem drive is mountable via UD as if a file system repair is going to be required then now is the time to do it before the New Config + parity build.