Hello,

I have a 2 Parity set up on my server. Close to 2 weeks ago I had two disks get the infamous red X's at the same time. I re-added one of them in it's original spot and it seemed to rebuild without issue (I'm not sure exactly which disk that was) and it has remained green for about a week. The second disk (Disk 11) didn't succeed so I ordered a new drive, and have been emulating it since. The replacement drive is bigger than the current parity drives, so the plan was to do some shuffling of drives. The drives arrived so today (about 6 hours ago maybe?) I did the following:

1. Shut down the server and added the new drive, then rebooted

2. Stopped the array and swapped out the newly added drive with Parity 1

3. Started the array so it would rebuild Parity 1

4. I just noticed two things:

   1. Parity sync is still stuck at .5%

   2. I also noticed Disk 2 is not showing a temperature and has a growing list of read errors

   3. I checked the logs and there are in fact tons of Disk 2 read errors

5. So now I have the following:

   1. Disk 11 seems to be bad after failing the first attempt to rebuild

   2. Disk 2 (which I now suspect is the one I "successfully" rebuilt) seems to actually be bad despite showing green

   3. Parity 1 rebuild seems to not be stalled

Have a screwed my parity and lost tons of my data? It seems disk 2 is actually bad along with disk 11 and I compounded the problem by attempting to rebuild Parity 1 . . .

I would love your advice. Parity Rebuild is still running but I downloaded the logs and attached them here.

Thank you

calculon-diagnostics-20250712-1836.zip

Jul 12 13:04:49 Calculon kernel: mpt2sas_cm0 fault info from func: mpt3sas_base_make_ioc_ready

Jul 12 13:04:49 Calculon kernel: mpt2sas_cm0: fault_state(0x7e23)!

Jul 12 13:04:49 Calculon kernel: mpt2sas_cm0: sending diag reset !!

Jul 12 13:04:50 Calculon kernel: mpt2sas_cm0: diag reset: SUCCESS

Problem with the HBA, make sure it's well seated and sufficiently cooled. You can also try a different PCIe slot if possible.

calculon-diagnostics-20250713-1135.zipThank you. I opened the case to double checked all the connections and there was definitely one drive that didn't seem well connected (Disk 11 I'm pretty sure). I wasn't able to feel an issue with the HBA connection, but regardless everything seems to be proceeding properly now. The parity-sync is currently at 8% without a single read error so I think that may have (fingers crossed, knock on wood) fixed the issue. I have included the current logs if you're curious, but for now I think we're good. Thanks again.

Edited July 13, 2025Jul 13 by beatmurph

Hello,

So I did as you asked and parity successfully rebuilt without issue. After that completed I set out to swap out the disabled Disk 11:

1. Stopped the array

2. Removed the current drive from the Drive 11 slot

3. Added the previously issue free Parity drive to the Drive 11 slot

4. The system started a process - "data rebuild" I believe?

After completing that process Drive 11 is listed as "Unmountable: wrong or no file system" despite the file system showing as xfs. Also when I hover over the green dot next to Drive 11 I get the message "Normal operation, device is active". Did I do something wrong in trying to replace Disk 11?

Diagnostics attached if needed.

calculon-diagnostics-20250717-1003.zip

Thanks again

When a disk is unmountable, we recommend fixing the filesystem before rebuilding, but do it now. Check filesystem for disk11

OK, I followed the steps and everything worked great. Thanks again, yet another lesson learned