Not sure what is going on with my system, but I had a scheduled bi-quarterly parity check that kicked off a few days ago that disabled 2nd parity disk and my disk 19.



in an effort to resolve this I shutdown re-seated every cable and my HBA and most of my disks came back to normal and were no longer disable but the one disk 19. Parity sync / rebuild kicked off and currently while I type is at 84% complete with ~5hrs left to go. I'm really hoping this disk comes back to life with the almost 9tb of data that it has but if not, I'm okay with losing some data as none of it is precious to me and I believe I can get most of it back. I just want to have my server back into a healthy state, any help or direction is welcome. Additionally I have a new set of Mini SAS HD SFF-8643 to SFF-8087 cables showing up tomorrow to replace my current ones as a precautionary measure.
Below is my current state

superplex-diagnostics-20250408-1200.zip

Uninstall Mover Tuning plugin at least until everything is working well again. Not that it is the cause of these problems, but it is often problematic, and clutters up the logs. Many who think they want that are probably overthinking how cache should be used.

 

55 minutes ago, Lean_ said:

rebuild kicked off

You are rebuilding an unmountable filesystem on disk19. Usually we try to repair before rebuilding on top of the same disk, that gives more options for recovering data. I guess we will just have to hope for the best when we repair the rebuilt filesystem.

 

But...

problems continuing with accessing multiple disks, 3, 4, 6. It can't disable anymore since you already have two disabled. It must be able to accurately read every bit of all other disks to rebuild the disabled disks. Entirely possible (actually, probable) that disk19 can't be accurately emulated since some other disks can't be read.

 

Looks like all HDD are on the same controller. Either controller or power would be main suspects.

 

 

 

 

In fact, you should just stop it all now and try to fix those problems, maybe wait on your new cables.

You are correct, they are all on the same controller.

I just uninstalled mover tuning plugin. Generally want files to live on the cache for as long as possible to minimize the spinup of other disks. Saves me about $700 a year iirc.

Additionally, I did have a weird uptime reset last week with no actual power outage. Also, for more context, I'm using a LSI 9300-8i controller and a supermicro chassis with a sas3 backplane and SQ PSU's.

Thank you for the response, I will update with a new post as soon as the rebuild is complete.

Oh? Stop the parity sync/rebuild?

Rebuild can't be going well with all those other disks not being read. Disk19 rebuild is unmountable anyway, and parity rebuild is unlikely to be any better.

Does it change anything if those errors occurred after I passed the 4TB mark? 

No, rebuild is rebuilding the entire parity2 and the entire disk19, but it doesn't have enough data coming from the other disks to expect that to be any good.

 

https://docs.unraid.net/unraid-os/overview/nas/#parity-protected-array

 

It might even be that if all the other problems were fixed, emulated disk19 would be mountable, since it could get the correct data for the emulation from reading all other disks. Then you could rebuild a mountable filesystem instead of the unmountable filesystem you are currently rebuilding.

 

 

Okay thank you! I've just cancelled it, would you mind outlining for me what the potential next-steps would look like?

You need to get all other disks working well again. Maybe the new cables will take care of it.

 

17 minutes ago, Lean_ said:

I did have a weird uptime reset last week with no actual power outage

Or maybe some power problem.

I've since swapped to a spare PSU so hopefully that doesn't happen again.

So what you're saying is, there's a chance that when I plug in the new cables, everything 'just works' ?

The disabled disks will still have to be rebuilt, but hopefully emulated disk 19 will be mounted when you rebuild. 

Cable has been swapped, drive 19 is still unmountable Parity 2 also still needs to be resynced, how should I proceed?

Edited April 9, 20251 yr by Lean_

Post new diagnostics

18 minutes ago, trurl said:

Post new diagnostics

 

superplex-diagnostics-20250409-1311.zip

1 hour ago, Lean_ said:

how should I proceed?

Looks like you already did, rebuilding an unmountable filesystem to disk19 along with parity2 rebuild.

That started automatically with the array, but I cancelled it like you suggested yesterday.

If you reassigned either disk, then starting the array is going to start rebuild.

 

Check filesystem on disk19 from the webUI, post the output.

On 4/9/2025 at 2:09 PM, trurl said:

If you reassigned either disk, then starting the array is going to start rebuild.

 

Check filesystem on disk19 from the webUI, post the output.

 

 

Edited April 12, 20251 yr by Lean_

I feel like I should also mention I have spare disks just in case that makes any of this easier.

Did you get a FIX button after that?

I did not, just CHECK, and "No file system corruption detected."

After all that, with many entries saying what it "would" do if it weren't -n nomodify.

 

Let's try it from the command line then.

 

xfs_repair -v /dev/md19p1 > /boot/xfs_repair_d19.txt

Then get xfs_repair_d19.txt from the flash drive and post it. You can get that file at MAIN - Boot Device and browse into flash, then Download it from the '+' ACTION popup.

 

Unfortunately the file is empty and this is what was returned in console.

Thank you for all of your help by the way, I am very appreciative of your time.

Edited April 10, 20251 yr by Lean_