Oct 21 03:51:05 UnRAID logger: mover finished

Oct 21 09:44:47 UnRAID kernel: ata8.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen

Oct 21 09:44:47 UnRAID kernel: ata8.00: failed command: READ DMA EXT

Oct 21 09:44:47 UnRAID kernel: ata8.00: cmd 25/00:08:c8:71:32/00:00:29:00:00/e0 tag 0 dma 4096 in

Oct 21 09:44:47 UnRAID kernel:          res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)

Oct 21 09:44:47 UnRAID kernel: ata8.00: status: { DRDY }

Oct 21 09:44:47 UnRAID kernel: ata8: hard resetting link

Oct 21 09:44:52 UnRAID kernel: ata8: link is slow to respond, please be patient (ready=0)

Oct 21 09:44:57 UnRAID kernel: ata8: SRST failed (errno=-16)

Oct 21 09:44:57 UnRAID kernel: ata8: hard resetting link

Oct 21 09:45:03 UnRAID kernel: ata8: link is slow to respond, please be patient (ready=0)

Oct 21 09:45:07 UnRAID kernel: ata8: SRST failed (errno=-16)

Oct 21 09:45:07 UnRAID kernel: ata8: hard resetting link

Oct 21 09:45:13 UnRAID kernel: ata8: link is slow to respond, please be patient (ready=0)

Oct 21 09:45:42 UnRAID kernel: ata8: SRST failed (errno=-16)

Oct 21 09:45:42 UnRAID kernel: ata8: limiting SATA link speed to 1.5 Gbps

Oct 21 09:45:42 UnRAID kernel: ata8: hard resetting link

Oct 21 09:45:47 UnRAID kernel: ata8: SRST failed (errno=-16)

Oct 21 09:45:47 UnRAID kernel: ata8: reset failed, giving up

Oct 21 09:45:47 UnRAID kernel: ata8.00: disabled

Oct 21 09:45:47 UnRAID kernel: ata8: EH complete

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x28: 28 00 29 32 71 c8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691171784

Oct 21 09:45:47 UnRAID kernel: md: disk9 read error, sector=691171720

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x28: 28 00 df 09 08 f8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 3741911288

Oct 21 09:45:47 UnRAID kernel: md: disk9 read error, sector=3741911224

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x2a: 2a 00 29 32 71 c8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691171784

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691171784

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: md: disk9 write error, sector=691171720

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: md: recovery thread woken up ...

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x28: 28 00 29 33 6e e8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691236584

Oct 21 09:45:47 UnRAID kernel: md: disk9 read error, sector=691236520

Oct 21 09:45:47 UnRAID kernel: md: recovery thread has nothing to resync

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x2a: 2a 00 df 09 08 f8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 3741911288

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 3741911288

Oct 21 09:45:47 UnRAID kernel: md: disk9 write error, sector=3741911224

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] Unhandled error code

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] 

Oct 21 09:45:47 UnRAID kernel: Result: hostbyte=0x04 driverbyte=0x00

Oct 21 09:45:47 UnRAID kernel: sd 10:0:0:0: [sdf] CDB:

Oct 21 09:45:47 UnRAID kernel: cdb[0]=0x2a: 2a 00 29 33 6e e8 00 00 08 00

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691236584

Oct 21 09:45:47 UnRAID kernel: end_request: I/O error, dev sdf, sector 691236584

Oct 21 09:45:47 UnRAID kernel: md: disk9 write error, sector=691236520

Oct 22 03:40:01 UnRAID logger: mover started

 

 

I would say disk9 (sdf) is getting ready to fail due to the read/write errors, OR, it could be a sata cable.  Try a sata cable first but I would make sure you have good backups and a replacement drive ready.  Is disk9 on a controller that other drives are on?  No issue with those drives I take it?  I'm assuming nothing has changed?

Only disk 9 is red balled, no other errors. does the errors show an actual sector error or is it more likely an actual onboard hard drive error?

You have timeout errors and a limiting to a certain speed error as well as read and write to sectors errors.  This could be a lot of things such as cable or a physical hard drive issue.  Since it's red balled already, I would replace the drive and cable just to be sure, no sense in replacing just the drive if it was the cable that caused the issue.  Once you have replaced them and parity is back in place, you can then run preclears on that drive to see if it has any issues.

Figured it out...it was the onboard controller, removed the drive from the array, plugged it into an external on my workstation and heard a slight clunk clunk...nice part is that the drive was under hitachi's 5 year warranty...thanks for the help.