Failed Parity Rebuild after Parity Swap

[JoshHolme]

Hello everyone. I had a dead drive and bought a replacement. I want to move my drives to 6TB eventually, so I bought a 6TB to replace my 4TB and followed the parity swap procedure to do so. The parity drive rebuilt fine, and then I was trying to rebuild the array, but it keeps failing on my old parity drive that is now my disk 1. Any help or pointers?

apollo-diagnostics-20220125-1638.zip

[JorgeB]

Jan 25 16:22:03 Apollo kernel: ata5: softreset failed (1st FIS failed)
Jan 25 16:22:03 Apollo kernel: ata5: hard resetting link
Jan 25 16:22:03 Apollo kernel: ata9: softreset failed (1st FIS failed)
Jan 25 16:22:03 Apollo kernel: ata9: hard resetting link
Jan 25 16:22:03 Apollo kernel: ata1: softreset failed (1st FIS failed)
Jan 25 16:22:03 Apollo kernel: ata1: hard resetting link
Jan 25 16:22:03 Apollo kernel: ata10: softreset failed (1st FIS failed)
Jan 25 16:22:03 Apollo kernel: ata10: hard resetting link
Jan 25 16:22:03 Apollo kernel: ata2: softreset failed (1st FIS failed)
Jan 25 16:22:03 Apollo kernel: ata2: hard resetting link
Jan 25 16:22:09 Apollo kernel: ata10: hard resetting link
Jan 25 16:22:09 Apollo kernel: ata1: SATA link down (SStatus 0 SControl 300)
Jan 25 16:22:14 Apollo kernel: ata1: hard resetting link
Jan 25 16:22:14 Apollo kernel: ata2: SATA link down (SStatus 0 SControl 300)
Jan 25 16:22:19 Apollo kernel: ata2: hard resetting link
Jan 25 16:22:19 Apollo kernel: ata9: SATA link down (SStatus 0 SControl 300)
Jan 25 16:22:24 Apollo kernel: ata9: hard resetting link
Jan 25 16:22:24 Apollo kernel: ata5: SATA link down (SStatus 0 SControl 300)
Jan 25 16:22:29 Apollo kernel: ata5: hard resetting link
Jan 25 16:22:30 Apollo kernel: ata5: SATA link down (SStatus 0 SControl 300)
Jan 25 16:22:35 Apollo kernel: ata5: hard resetting link
Jan 25 16:22:35 Apollo kernel: ata9: softreset failed (1st FIS failed)
Jan 25 16:22:35 Apollo kernel: ata9: hard resetting link

 

Multiple disk issues, could be a controller, power or connection problem.

[JoshHolme]

Weird. It’s been running fine for months before this. I’ll have to check the connectors

[JoshHolme]

@JorgeB I'm hesitant to think that it could be an issue with that since I've done parity rebuilds with this same controller, sata cables and other hardware before. The disk that it's trying to rebuild used to be my parity drive, but now is being used as Disk 1 in my system. It starts out fine but then always fails after a few minutes. Does this extra info help out at all for other ideas you think it might be?

[JorgeB]

Like mentioned you have errors on multiple disks, disk1 isn't the problem, at least not for now

 

Jan 25 16:21:48 Apollo kernel: md: disk1 write error, sector=273781944
..
Jan 25 16:22:35 Apollo kernel: md: disk3 read error, sector=3558475008
..
Jan 25 16:23:02 Apollo kernel: md: disk6 read error, sector=273783216
..
Jan 25 16:23:26 Apollo kernel: md: disk2 read error, sector=273782224
..
Jan 25 16:23:41 Apollo kernel: md: disk4 read error, sector=273786064
..
Jan 25 16:24:27 Apollo kernel: md: disk5 read error, sector=273785856

 

[JoshHolme]

My SATA cables are fairly old. I just ordered all new cables and hopefully that fixes the issue. They should be here Monday

[trurl]

Do you have power splitters to your disks? That would be a likely suspect for multiple disk problems.

[JoshHolme]

The drives were powered by a cable with many sata power connectors on it. I reseated all sata data and power cables. Sata data cables must be at least 10 years old, passed on from another old server. I have new cables to replace those coming Monday, but for now I was able to get the rebuild to finish, and I think we’re in the clear. Thank you everyone!