Maybe this should go in the support forum but I figured I would post it here first.
I have been seeing a lot of drive errors in my Norco 4220 lately. Two of my recent drives start showing errors as soon as I put them in multiple slots. This last drive has failed at exactly 99% of the preclear disk pre-read. The first time my server froze up and was unresponsive and I had to restart it. It is currently in slot 9 if you count from left to right from top to bottom. I also put it in slot 17 with the same result. I tried to preclear it last night and didn't notice anything wrong but woke up this morning to see this in the syslog followed by 2 gigs of drive error logs.
Nov 30 05:30:33 Tower kernel: sas: command 0xecd99540, task 0xceea1540, timed out: BLK_EH_NOT_HANDLED
Nov 30 05:30:33 Tower kernel: sas: Enter sas_scsi_recover_host
Nov 30 05:30:33 Tower kernel: sas: trying to find task 0xceea1540
Nov 30 05:30:33 Tower kernel: sas: sas_scsi_find_task: aborting task 0xceea1540
Nov 30 05:30:33 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1701:mvs_abort_task:rc= 5
Nov 30 05:30:33 Tower kernel: sas: sas_scsi_find_task: querying task 0xceea1540
Nov 30 05:30:33 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1645:mvs_query_task:rc= 5
Nov 30 05:30:33 Tower kernel: sas: sas_scsi_find_task: task 0xceea1540 failed to abort
Nov 30 05:30:33 Tower kernel: sas: task 0xceea1540 is not at LU: I_T recover
Nov 30 05:30:33 Tower kernel: sas: I_T nexus reset for dev 0700000000000000
Nov 30 05:30:33 Tower kernel: sas: I_T 0700000000000000 recovered
Nov 30 05:30:33 Tower kernel: sas: --- Exit sas_scsi_recover_host
Nov 30 05:31:04 Tower kernel: sas: command 0xecd99540, task 0xd0c6d900, timed out: BLK_EH_NOT_HANDLED
Nov 30 05:31:04 Tower kernel: sas: Enter sas_scsi_recover_host
Nov 30 05:31:04 Tower kernel: sas: trying to find task 0xd0c6d900
Nov 30 05:31:04 Tower kernel: sas: sas_scsi_find_task: aborting task 0xd0c6d900
Nov 30 05:31:04 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1701:mvs_abort_task:rc= 5
Nov 30 05:31:04 Tower kernel: sas: sas_scsi_find_task: querying task 0xd0c6d900
Nov 30 05:31:04 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1645:mvs_query_task:rc= 5
Nov 30 05:31:04 Tower kernel: sas: sas_scsi_find_task: task 0xd0c6d900 failed to abort
Nov 30 05:31:04 Tower kernel: sas: task 0xd0c6d900 is not at LU: I_T recover
Nov 30 05:31:04 Tower kernel: sas: I_T nexus reset for dev 0700000000000000
Nov 30 05:31:04 Tower kernel: sas: I_T 0700000000000000 recovered
Nov 30 05:31:04 Tower kernel: sas: --- Exit sas_scsi_recover_host
Nov 30 05:31:35 Tower kernel: sas: command 0xecd99540, task 0xd0c6d900, timed out: BLK_EH_NOT_HANDLED
Nov 30 05:31:35 Tower kernel: sas: Enter sas_scsi_recover_host
Nov 30 05:31:35 Tower kernel: sas: trying to find task 0xd0c6d900
Nov 30 05:31:35 Tower kernel: sas: sas_scsi_find_task: aborting task 0xd0c6d900
Nov 30 05:31:35 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1701:mvs_abort_task:rc= 5
Nov 30 05:31:35 Tower kernel: sas: sas_scsi_find_task: querying task 0xd0c6d900
Nov 30 05:31:35 Tower kernel: /usr/src/sas/trunk/mvsas_tgt/mv_sas.c 1645:mvs_query_task:rc= 5
Nov 30 05:31:35 Tower kernel: sas: sas_scsi_find_task: task 0xd0c6d900 failed to abort
Nov 30 05:31:35 Tower kernel: sas: task 0xd0c6d900 is not at LU: I_T recover
Nov 30 05:31:35 Tower kernel: sas: I_T nexus reset for dev 0700000000000000
Nov 30 05:31:35 Tower kernel: sas: I_T 0700000000000000 recovered
Nov 30 05:31:35 Tower kernel: sas: --- Exit sas_scsi_recover_host
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5.00: device reported invalid CHS sector 0
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x08
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] Sense Key : 0xb [current] [descriptor]
Nov 30 05:31:58 Tower kernel: Descriptor sense data with sense descriptors (in hex):
Nov 30 05:31:58 Tower kernel: 72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00
Nov 30 05:31:58 Tower kernel: 00 00 00 f6
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] ASC=0x0 ASCQ=0x0
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] CDB: cdb[0]=0x28: 28 00 e7 30 fe f8 00 02 00 00
Nov 30 05:31:58 Tower kernel: end_request: I/O error, dev sdf, sector 3878747896
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843487
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843488
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843489
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843490
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843491
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843492
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843493
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843494
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843495
Nov 30 05:31:58 Tower kernel: Buffer I/O error on device sdf, logical block 484843496
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x08
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] Sense Key : 0xb [current] [descriptor]
Nov 30 05:31:58 Tower kernel: Descriptor sense data with sense descriptors (in hex):
Nov 30 05:31:58 Tower kernel: 72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00
Nov 30 05:31:58 Tower kernel: 00 00 00 f6
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] ASC=0x0 ASCQ=0x0
Nov 30 05:31:58 Tower kernel: sd 2:0:4:0: [sdf] CDB: cdb[0]=0x28: 28 00 e7 30 fe f8 00 00 08 00
Nov 30 05:31:58 Tower kernel: end_request: I/O error, dev sdf, sector 3878747896
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }
Nov 30 05:31:58 Tower kernel: ata5: translated ATA stat/err 0x41/04 to SCSI SK/ASC/ASCQ 0xb/00/00
Nov 30 05:31:58 Tower kernel: ata5: status=0x41 { DriveReady Error }
Nov 30 05:31:58 Tower kernel: ata5: error=0x04 { DriveStatusError }