beatmurph

I swore I ran extended on both, but I'll do so for the one you said. Thank you again.

Hello, I want to thank you all again for your time and help. Even though I wound up where I started with the data, I did learn and feel better about some of the process now. The parity rebuild is almost done and I'll be moving forward with rebuilding the data for Drive 5 after. I do have another question, but please let me know if I should create a new post. I have 2 drives from early on when I was super new to all this that had previously developed red X's. I removed them from the server at that time, but I decided to go ahead and do a full SMART test on both of them the other day. SMART tests are challenging to understand for a novice and I was hoping you might be able to help me determine if they seem reliable to return to the array or not? From what I can tell it seems one of them is fine and the other is not. I have attached the SMART reports. calculon-smart-20250106-1959.zip calculon-smart-20250117-1359.zip

OK, so I'm starting this today and as I reread your comment I realize I may have misunderstood it. When you say "we're giving up on Disk 5" Are you saying that the data on that disk is lost and there is no hope of recovery? If I'm understanding what I'm reading in the link you set me, this will completely reset parity using the all the data disks I have with the exception of disk 5?

Thank you. I'll work on this and let you know how it goes.

Forgive me, but I want to make sure I'm precise as I'm nervous I might loose what is in the array as it stands and/or what is on the disk that isn't in the array but still has the data from before the failure. I'm not familiar with how to do a "new config" is there a specific set of instructions?

I did some looking for a new parity sync and haven't seen anything. Would you be able to point me to the instruction page, or help me through it? I'm assuming it's not just replacing the assigned drive.

OK, so just to clarify, I'm not positive if parity was finish rebuilding for Parity 2 before both disks 5 & 6 developed the red X, so its possible the red Xs developed while Parity 2 was rebuilding. You still think a resync is the is the best path forward in this case given the original disk 6 seems to be mountable with all the original data? I have tried looking for a guide on parity resync. Is there one I should follow?

So I have still not tried the file system check for the current Disk 6 (which is the former Disk 5). Would it make sense to try that first. Maybe without -n as originally suggested?

Happy to. Here you go calculon-diagnostics-20250112-1645.zip

I couldn't find if there was a way to export this so here is a copy/paste: "Phase 1 - find and verify superblock... couldn't verify primary superblock - not enough secondary superblocks with matching geometry !!! attempting to find secondary superblock... .found candidate secondary superblock... unable to verify superblock, continuing... ........found candidate secondary superblock... unable to verify superblock, continuing... unable to verify superblock, continuing... . . ..Sorry, could not find valid secondary superblock Exiting now." There might be something missing where the space is, but the amount of dots was crashing everything I tried to copy it over to. I get the sense that doesn't matter.

I completed the filesystem check for Disk 5. I executed with the -n as I was a little nervous jumping straight into repair, though I was unaware it would take a full day. I was going to proceed with your original recommendation to do the check and repair without -n, but given the time it takes I figured I better pause any action for now and await your recommendation. I have not formatted the disks. They still show as unmountable in the array. I have attached the latest diagnostics after performing a Check Filesystem on Disk 5. calculon-diagnostics-20250112-1022.zip

Actually, I might be in worse shape. After the data-rebuild of Disks 5 & 6 there was a warning presented which stated something to the affect. "Warning, when the array is started all data on these disks will be deleted" because of that I had been avoiding starting the array. I'm not sure if in some brain dead state I eventually started the array for some reason, or if maybe I did a system restart and that automatically started the array. Either way, I'm just now looking and registering that the array has been started. I just stopped it and that warning no longer exists for disks 5 & 6.

The Disks currently in the array: current Disk 5 - formerly Parity 1 (underwent data rebuild in current configuration) current Disk 6 - formerly Disk 5 (underwent data rebuild in current configuration) These are for some reason showing as "Unmountable: Unsupported or no file system". This is the part that confused me the most. They are both disks that were in the array before I started the re-build and are still listed as xfs which is the same as the rest of the array. Because of that I don't know of a way to see what is on them.

Correct, Disk 6 was only moved from the array to unassigned disk devices. I can mount it and see all its original contents and have not done anything with it but a SMART check. This was Disk 6

No, Disc 5 was moved to Disk 6 in the array and after that I initiated the data-rebuild.

Can I assume by "old drives" you mean the ones specifically that were moved around? In this case it is the following: current Disk 5 - formerly Parity 1 (underwent data rebuild in current configuration) current Disk 6 - formerly Disk 5 (underwent data rebuild in current configuration) former Disk 6 - no in unassigned disk pool Currently both parity drives are from the new batch. All other drives unchanged with no known issues

Hello, and thank you for the quick response. This is my first time posting, so sorry if that was something I should have done by default. I have also turned on notifications so I should see these responses right away. Diagnostics are now attached, but I have to let you know that I have done a reboot since completing the rebuild. calculon-diagnostics-20250110-1004.zip

Hello, I recently acted without caution and believe I have destroyed parity for my array. I was preparing to start the process of manually rebuilding the lost data (which would be a long and tedious process), but after discovering one of the drives still has the original data when mounting as an unassigned disk device I thought I should reach out to the experts here to see if there is any hope of recovering the data without that amount of work. I recently looked to expand the number of drives in my array which included replacing both parity drives with larger capacity drives. Here is what happened: I powered down my server and added the new larger capacity drives, then started the server back up. At this time all drives (new & current array) showed in the UI without issues I stopped the array and reassigned Parity 1 to one of the new larger drives. Then initiated a parity rebuild At some point During the parity rebuild of Parity 1, Parity 2 showed the Red X next to it. I allowed rebuild to finish since I was replacing Parity 2 next anyway After Parity rebuild completed for Parity 1, I began the same process for Parity 2 by reassigning another one of the new larger drives. At some point during this parity rebuild, one of the data drives developed a Red X also (grrr). I continued to monitor the rebuild every few hours and everything seemed fine. When I returned to check on the server after it should have completed I found it had completed the parity build but now a second data drive was showing the Red X. This is when I probably should have come here, but I did not. I did the following. I then reassigned one of the original (now replaced) parity drives as well as one of the two data drives that had the Red X to the two data drive slots in the array with Red Xs and initiated a data-rebuild for them. They rebuild did finally complete and now they are listed as "Unmountable: Unsupported or no file system". I'll admit that I'm confused by that since they were just part of the array prior and are listed as xfs like all the other drives in the array. Maybe that's to be expected? So then I mounted the data disks that developed a Red X during the Parity 2 rebuild and can see that it has the data I would expect to see on it. If I had taken my time and thought to do this for the other Red X data drive instead of using it for a data-rebuild in the array then I might of found the same, but now I'll never know it seems. Is there any chance of saving my data or did I damage it beyond repair already? Thank you.