Tried to replace and rebuild a failed disk, errors in a second disk began... now fearful of losing everything

[trurl]

4 hours ago, egeis said:

I'm also going to follow the rebuild by cloning the parity drive to a new and larger disk, then move the parity disk in to replace the suspicious disk2. 

There is a special procedure for this called parity swap. I don't know if that is what you had in mind.

[egeis]

So the format was set on "auto" so there was no "Check Filesystem" dialog. I changed from "auto" to "xfs". The check button appeared with a "-n" flag. I ran that process... there is a LOT of output. It finished with statements about "No modify flag set, skipping filesystem flush and exiting".  

 

I can't find anything in the manual/storage management page that makes sense except adding the flag "-L". But I'm not sure I'm looking to do a "filesystem flush"...

 

Appreciative of any guidance. 

[egeis]

17 minutes ago, trurl said:

Formatting it outside the array before using it to rebuild would have been pointless since that empty filesystem would be completely overwritten by rebuild.

When I initially stated my problem, I mentioned that I attempted the first data rebuild on this disk for disk2. Then disk1 was showing failures during that rebuild; It was only done with 60GB (2.6%) 12 hours later. That's when I asked for help. The filesystem probably wasn't empty. I don't think a format would've been pointless.

[trurl]

16 minutes ago, egeis said:

The filesystem probably wasn't empty. I don't think a format would've been pointless.

Rebuild completely overwrites every bit of the disk with the results of the parity calculation. Doesn't matter at all what was already on the disk. Doesn't matter if there was an empty filesystem, a filesystem with a lot of files, or no filesystem at all. Format before rebuild is pointless.

 

19 minutes ago, egeis said:

"-n" flag. I ran that process... there is a LOT of output. It finished with statements about "No modify flag set, skipping filesystem flush and exiting".  

-n is the "No modify flag". You have to run it without that or it won't actually fix anything.

 

Did you capture the output?

 

 

 

[egeis]

Here's the output. Lots  of xfs_repair_output.txt  text!

 

I'll rerun with a flag assuming "filesystem flush" is a good thing!

Edited November 16, 2021 by egeis

[trurl]

That looks pretty bad but you will have to go ahead without the -n flag and hope for the best. Maybe something can be read from the original disk1 if necessary.

[egeis]

The original disk1 is not readable. I saved a ton of what I needed before I rebuilt the data. Running xfs_repair now.

[egeis]

It mounted! I may have lost a few things, but I'm super happy I made it through this nightmare. Thanks to you guys!

 

[trurl]

Post new diagnostics.

[trurl]

Or just check your lost+found share to see how much repair had to dump there.

[trurl]

18 hours ago, egeis said:

follow the rebuild by cloning the parity drive to a new and larger disk, then move the parity disk in to replace the suspicious disk2. 

12 hours ago, trurl said:

There is a special procedure for this called parity swap. I don't know if that is what you had in mind.

Don't try to do this any other way.

[trurl]

Just now, trurl said:

Don't try to do this any other way.

Unless you just want to rebuild parity to a new larger disk, then follow that by rebuilding disk2 to that former parity disk.

[egeis]

In the process of trying to rebuild parity using a bigger drive..., it disabled the new parity disk about an hour in. This has now happened 2 times. I changed the connection and it happened again (third time). I think this is a mobo - controller problem. I'm attaching diagnostics in case anyone is interested... but I'm pretty sure I'm just gonna buy a new motherboard. The motherboard I built this with probably wasn't a good choice for Unraid (Asus - X99 TaiChi). I didn't really understand too much about the long-term consequences when I went down this road initially. It's been running since 2017, so I guess it gave me a good 4 years... but these disk problems definitely aren't worth it.

 

 unraidtower-diagnostics-20211123-1548.zip

[trurl]

Nov 23 09:49:15 UnRaidTower kernel: ata2.00: ATA-10: WDC WD40EFAX-68JH4N1,      WD-WX22D619H7RT, 83.00A83, max UDMA/133
Nov 23 09:50:06 UnRaidTower kernel: md: import disk0: (sdb) WDC_WD40EFAX-68JH4N1_WD-WX22D619H7RT size: 3907018532 
Nov 23 09:50:06 UnRaidTower kernel: md: import_slot: 0 replaced
Nov 23 10:59:33 UnRaidTower kernel: ata2: SATA link down (SStatus 0 SControl 300)
Nov 23 10:59:33 UnRaidTower kernel: ata2.00: link offline, clearing class 1 to NONE
Nov 23 10:59:34 UnRaidTower kernel: ata2: hard resetting link
Nov 23 10:59:39 UnRaidTower kernel: md: disk0 write error, sector=1251653984

I still suspect a connection problem, maybe SATA, maybe power or splitter.

[egeis]

Things were running well for more than 2 weeks. Parity was disabled last night. 

 

Any last suggestions before I buy a new Mobo?

unraidtower-diagnostics-20211201-2125.zip

[trurl]

Do you have any spare SATA ports you could try?

[egeis]

Actually, I'm gonna buy a SAS HBA card and move all the disks to those ports

I've already shifted around some SATA ports. Definitely worried that it's the mobo's native controllers causing issues

Edited December 2, 2021 by egeis

[egeis]

I inserted the SAS HBA and moved all 4 disks to the card. Everything is recognized, parity is being rebuilt. Speeds look good, disks look good.