Last week I awoke to a yellow triangle next to my parity drive which appears to mean parity is being emulated due to a disk problem. There were no other obvious errors with the array, and the SMART report of the parity disk appeared clean.. After some reading on the forum I figured that rebuilding parity was the way to go. I shut down the array, removed the parity disk (well inactivated it), restarted the array, stopped it again, chose the parity disk again and restarted the array. Parity started rebuilding. I went away for the weekend.

 

On return I see the array is up and running, "parity is valid", all disks are green, and there is a data disk (#2) with thousands of read errors. I'm not really sure what to do now. I have attached diagnostics to help. I do still have a diagnostic report from right before the parity rebuild if that would help. I checked a bunch of random checksums from that disk and have not found any errors, yet. Any help would be greatly appreciated.

tower-diagnostics-20170905-1802.zip

Disk2 needs to be replaced:
 

197 Current_Pending_Sector  0x0012   080   080   000    Old_age   Always       -       3408
198 Offline_Uncorrectable   0x0010   080   080   000    Old_age   Offline      -       3408

Unfortunately, the bad sectors were already there during the parity sync:

 

Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531216
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531216
Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531224
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531224
Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531232
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531232
Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531240
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531240
Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531248
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531248
Aug 30 01:42:05 Tower kernel: md: disk2 read error, sector=5855531256
Aug 30 01:42:05 Tower kernel: md: recovery thread: multiple disk errors, sector=5855531256

You can still replace disk2 with a new disk but some files on the rebuilt disk will be corrupt, if you have checksums you can check which ones are affected and replace them with backups if available.

 

You should consider dual parity for such a large array.

I had something similar happen to me a few weeks ago during a parity check. The parity check revealed a ton of errors on one of my data drives. I read people saying that I needed to replace the drive, but I acknowledged the little triangle icon and just ignored it. I was hoping the disk would fail and I could replace it, but no such luck. Its still chugging away. The disk marked those sectors as bad and moved on. I do have a spare disk on standby though, hoping it will fail soon.

1 minute ago, RonUSMC said:

I had something similar happen to me a few weeks ago during a parity check

 

Parity check is different from parity sync, when running a parity check, parity should already be valid, if there's a read error from a data disk unRAID uses parity and the remaining disks to write those sector(s) back, if successful the disk remains enable and the bad sectors may be remapped (though it can have new ones on the next check), as for the OP issue, there where read errors during the parity sync, so parity is not 100% correct and the disk still has the same bad sectors.

That makes sense.

Well, I was trying to get all the data off the failing drive, unfortunately it went belly up and is currently unmountable. Oh, and the parity drive was disabled again, during the copy, so there's that. I do have drives to replace the failed disk, and what I now assume is a failing parity drive, too. The new parity needs to be pre cleared which takes forever to do three cycles. So I'm hoping to replace the dead data drive (as a new empty drive), resync parity, and get the new parity drive pre clearing to replace the potentially failing one, while crossing my fingers it doesn't go belly up, too  Fun times.