Parity Sync continues with read errors from a data drive.

[Sokoal117]

Background: So I am hoping I didn't mess up. I will provide a little background before I explain my question. I wanted to upgrade an old drive in my array with a new 8tb one. I pre-cleared the drive and everything was good, mounted it and started the data rebuild. After about 8-10h I got ~5000 errors and the drive dismounted. I looked it up and it looked like I just didn't seat the drive right and it came loose. So I tried the rebuild again and it was successful. After running the array for 5h with this new drive I decided to upgrade my Parity drive to a new 14tb one. Well after ~12h of the parity sync this new 8tb drive started spewing more errors. After another 7h it started having smart errors about bad sectors and relocated data.

 

Questions: Is the Parity Sync Valid once it completes if a data drive had errors during the sync? And if I do and XFS Repair and errors are found does that mean I need to redo the sync? If there are no error to correct in the XFS Repair does that mean it is good and I can replace the failing drive?

 

Facts:

1. Drive is Failing SMART status.
2. Drive showed Error during Parity Sync.
3. Parity Drive is new and Failing data drive has 5h of new data.
4. No XFS repair has been run yet. See Update.

 

Update: Parity Sync Completed without errors though I believe that is a lie. I have now run an XFS repair and there were no errors. I am getting bad sectors on the drive but due to its minimal utilization sector relocation is working. I have restarted the parity sync with the OS in safe mode and maintenance mode for the array. I am hoping that this sync will complete without errors and then I can rebuild that failing drive with a known good one.

Edited July 10, 2022 by Sokoal117

[itimpi]

The parity will not be valid if any data drive had read errors while building the parity.

 

In terms of getting back to a sensible state:

• Were  there any changes to the data on other drives during the latest attempt at parity build?   If not do you have the old parity dtive intact.
• do you still have the data drive that you replaced with its data intact?

It might be worth posting your system’s diagnostics zip file so we can see exactly what the current state of affairs is.

 

[Sokoal117]

@itimpi I have attached the current diagnostics file. The original parity sync finished with the 5485 errors. I decided (for better or for worse) to try another sync while in safe mode and maintenance mode. My thought process was hoping that the integrity of the failing drive may hold long enough to complete another sync and correct those errors. So far this sync has corrected 6624 errors and only has 78 errors (so better?). 

 

On 7/10/2022 at 12:40 AM, itimpi said:

• Were  there any changes to the data on other drives during the latest attempt at parity build?   If not do you have the old parity dtive intact.
• do you still have the data drive that you replaced with its data intact?

 

• Yes there were changes when an unwanted docker restarted and added ~120gb to the array, the failing drive in particular too. I have the old parity drive but it would be invalid.
• I do have that data drive as well but since data has been added to the array I can't swap back any drives. I have to work with what I have in the array now.

 

Question: Would running another parity sync after this to correct the new 78 errors that popped up be even worth it? Or should I put a new drive in as a replacement for the failing one after this current parity sync and rebuild the data drive from the parity that has 78 errors now on it?

skynetserver-diagnostics-20220711-0011.zip

Edited July 11, 2022 by Sokoal117
Clarity and info

[JorgeB]

Disk10 appears to be failing, you can run an extended SMART test to confirm.

[Sokoal117]

@JorgeB I am running an extended test now due to the size being 8TB will take close to a day. I will report back when it is completed.

11 hours ago, Sokoal117 said:

Question: Would running another parity sync after this to correct the new 78 errors that popped up be even worth it? Or should I put a new drive in as a replacement for the failing one after this current parity sync and rebuild the data drive from the parity that has 78 errors now on it?

 

[JorgeB]

8 minutes ago, Sokoal117 said:

Question: Would running another parity sync after this to correct the new 78 errors that popped up be even worth it?

Wait for the extended test, if the disk is failing you might get more errors.

 

Do you still have the old disk, any new data since the disk was upgraded?

[Sokoal117]

@JorgeB Thanks for getting back to me. I guess I am wondering if rebuilding a data drive off of a party that has been synced with 78 error will poss a big problem. If all it will cause is a corrupted movie or something, and all I have to do is redo the remux of the faulty file, when I find one and resync parity after that. Then this is a minor inconvenience but if it causes more then I will need to be more cautious.

11 hours ago, Sokoal117 said:

• Yes there were changes when an unwanted docker restarted and added ~120gb to the array, the failing drive in particular too. I have the old parity drive but it would be invalid.
• I do have that data drive as well but since data has been added to the array I can't swap back any drives. I have to work with what I have in the array now.

Edited July 11, 2022 by Sokoal117
Rewording

[JorgeB]

9 minutes ago, Sokoal117 said:

I guess I am wondering if rebuilding a data drive off of a party that has been synced with 78 error will poss a big problem.

Those errors can cause some data corruption in the rebuilt disk, if they coincide with sectors use by data, but so few errors should not be anything major, could be for example just a small glitch during a movie.