I've just updated a server to Unraid 6.6.5 and I was going through the syslog when I found

Nov  9 09:51:31 Lapulapu kernel: ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov  9 09:51:31 Lapulapu kernel: ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov  9 09:51:31 Lapulapu kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov  9 09:51:31 Lapulapu kernel: ata9.00: ATA-9: TOSHIBA THNSNH256GMCT,         X3ES100STOMY, HTGAN102, max UDMA/100
Nov  9 09:51:31 Lapulapu kernel: ata9.00: 500118192 sectors, multi 16: LBA48 NCQ (depth 32), AA
Nov  9 09:51:31 Lapulapu kernel: ata9.00: configured for UDMA/100
Nov  9 09:51:31 Lapulapu kernel: sdb: sdb1
Nov  9 09:51:31 Lapulapu kernel: Alternate GPT is invalid, using primary GPT.
Nov  9 09:51:31 Lapulapu kernel: sdd: sdd1
Nov  9 09:51:31 Lapulapu kernel: sdc: sdc1
Nov  9 09:51:31 Lapulapu kernel: sde: sde1

It isn't at all apparent which disk is being referred to. How can I find out and how can I fix it?

lapulapu-diagnostics-20181109-1008.zip

 

Try using sgdisk, with the array stopped, it should identify the problem disk and I believe it can also fix it.

Thanks for that @johnnie.black. I ran it on each array disk in turn until I got this (the others, including two parities) are all ok:

root@Lapulapu:~# sgdisk -v /dev/sdd
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************

No problems found. 30 free sectors (15.0 KiB) available in 1
segments, the largest of which is 30 (15.0 KiB) in size.

Running it a second time produced exactly the same output so I feel I need to follow the recommendation about disk verification and recovery. The question is, how best to proceed?

 

If my understanding is correct, the GPT is in that small area of the disk that is not involved in the parity calculation - hence I'm working on the raw disk device, not the corresponding md device - which, of course, doesn't exist when the array is stopped.

 

So I guess I could either try to repair the GPT in situ or just rebuild the disk from parity onto a replacement. Do I have an option to rebuild onto the same disk? Presumably I'd have to wipe the existing partition table first (would the preclear script help there?). I think I'll have lunch and do a bit of reading before I do anything else as sgdisk looks like a powerful and dangerous tool. I do have backups but it would be educational and possibly useful for someone else in the same position in the future if I can repair this without losing the data.

 

Incidentally, I think I know what caused this problem. I had what I believed was a failing Dell H310 HBA, which I replaced. I didn't throw out the old one though and tested it in a different server. On the hottest day of the year I ended up removing the heatsink and replacing the TIM, which had dried out. I haven't been able to make it go wrong since.

50 minutes ago, John_M said:

If my understanding is correct, the GPT is in that small area of the disk that is not involved in the parity calculation

correct

 

50 minutes ago, John_M said:

So I guess I could either try to repair the GPT in situ or just rebuild the disk from parity onto a replacement. Do I have an option to rebuild onto the same disk?

Both should work, but I never needed to repair a GPT, so not 100% sure repair would work, rebuild should work without issues, and preclear can clear the GPT, though a normal rebuild should repartition the disk and also fix it.

34 minutes ago, johnnie.black said:

a normal rebuild should repartition the disk

I thought that would probably be the case but I didn't know for sure. What I did, in case anyone else finds themselves in the same situation, was to unassign the disk (Disk5) and start the array. Then I used Unassigned Devices to delete the partition on the disk. Then I used UD's Format option. Then I stopped the array and ran

sgdisk -v /dev/sdd

again, which confirmed that the partition table is now good. Then I reassigned the disk to Disk5 and started the array. It's now rebuilding. Thanks, j.b.