March 6, 201214 yr I found a CORSAIR Gaming Series GS600 at work that I can borrow for a couple of days. Right now it $82.48 with a $5MIR till tomorrow... I'll give this a try tonight. What a pain! I'm not looking forward to swapping out the PSU! But I guess I really shouldn't be cheap on the PSU (or any component) for a system that I need to have running 24/7 that has things I don't really want to lose! Perhaps just take an old power supply and use it to power some of your disks. (and have two power supplies connected for the test) Joe L.
March 6, 201214 yr Author Perhaps just take an old power supply and use it to power some of your disks. (and have two power supplies connected for the test) Maybe I'll try that... It crashed again last night. Probably when some on my nightly backups were trying to go or when the mover script kicked in It's at 31% now. I won't really access the array until its done... Or at least until It's past some of the smaller drives... The 1TB should be done soon then the 1.5s. Maybe I'll experiment then. Jim
March 6, 201214 yr Author Damn!! It crashed again.. This time I was somewhat watching... and I was able to get part of the syslog. There are some errors in there.. and before it crashed, it started to slow down (the MB/s that is) Now I can't get to it at all. What do these errors mean? Is this adapter related or drive related? ATA10 is Mar 6 08:07:52 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1388:found dev[3:5] is gone. (System) Mar 6 08:07:52 Tower kernel: sas: sas_ata_phy_reset: Found ATA device. (Drive related) Mar 6 08:07:52 Tower kernel: ata10.00: ATA-7: SAMSUNG HD154UI, 1AG01118, max UDMA7 (Drive related) Mar 6 08:07:52 Tower kernel: ata10.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32) (Drive related) Mar 6 08:07:52 Tower kernel: sdh: sdh1 (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:2:0: [sdh] Attached SCSI disk (Drive related) Mar 6 08:07:52 Tower kernel: ata10.00: configured for UDMA/133 (Drive related) Mar 6 08:07:52 Tower kernel: scsi 0:0:3:0: Direct-Access ATA SAMSUNG HD154UI 1AG0 PQ: 0 ANSI: 5 (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:3:0: [sdi] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB) (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:3:0: [sdi] Write Protect is off (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:3:0: [sdi] Mode Sense: 00 3a 00 00 (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:3:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA (Drive related) Mar 6 08:07:52 Tower kernel: sas: DONE DISCOVERY on port 3, pid:822, result:0 (Drive related) Mar 6 08:07:52 Tower kernel: sdi: sdi1 (Drive related) Mar 6 08:07:52 Tower kernel: sd 0:0:3:0: [sdi] Attached SCSI disk (Drive related) There errors are here: Mar 6 08:24:29 Tower emhttp: shcmd (66): /usr/sbin/hdparm -y /dev/sdf $stuff$> /dev/null (Drive related) Mar 6 11:53:48 Tower emhttp: shcmd (67): /usr/sbin/hdparm -y /dev/sdf $stuff$> /dev/null (Drive related) Mar 6 12:50:18 Tower kernel: sas: command 0xf1980600, task 0xf19223c0, timed out: BLK_EH_NOT_HANDLED (Drive related) Mar 6 12:50:18 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related) Mar 6 12:50:18 Tower kernel: sas: trying to find task 0xf19223c0 (Drive related) Mar 6 12:50:18 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf19223c0 (Drive related) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1703:<7>mv_abort_task() mvi=f7400000 task=f19223c0 slot=f74115d4 slot_idx=x0 (System) Mar 6 12:50:18 Tower kernel: sas: sas_scsi_find_task: querying task 0xf19223c0 (Drive related) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1632:mvs_query_task:rc= 5 (System) Mar 6 12:50:18 Tower kernel: sas: sas_scsi_find_task: task 0xf19223c0 failed to abort (Minor Issues) Mar 6 12:50:18 Tower kernel: sas: task 0xf19223c0 is not at LU: I_T recover (Drive related) Mar 6 12:50:18 Tower kernel: sas: I_T nexus reset for dev 0300000000000000 (Drive related) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x89800. (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001 (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2111:phy3 Unplug Notice (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1081 (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x10000 (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2138:notify plug in on phy[3] (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1224:port 3 attach dev info is 0 (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1226:port 3 attach sas addr is 3 (System) Mar 6 12:50:18 Tower kernel: drivers/scsi/mvsas/mv_sas.c 378:phy 3 byte dmaded. (System) Mar 6 12:50:18 Tower kernel: sas: sas_form_port: phy3 belongs to port3 already(1)! (Drive related) Mar 6 12:50:20 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1586:mvs_I_T_nexus_reset for device[3]:rc= 0 (System) Mar 6 12:50:20 Tower kernel: sas: I_T 0300000000000000 recovered (Drive related) Mar 6 12:50:20 Tower kernel: sas: sas_ata_task_done: SAS error 8d (Errors) Mar 6 12:50:20 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:50:20 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:50:20 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:50:20 Tower kernel: sas: --- Exit sas_scsi_recover_host (Drive related) Mar 6 12:50:20 Tower kernel: sas: sas_to_ata_err: Saw error 2. What to do? (Errors) Mar 6 12:50:20 Tower kernel: sas: sas_ata_task_done: SAS error 2 (Errors) Mar 6 12:50:20 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:50:20 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:50:20 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:50:51 Tower kernel: sas: command 0xf1980600, task 0xf19223c0, timed out: BLK_EH_NOT_HANDLED (Drive related) Mar 6 12:50:51 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related) Mar 6 12:50:51 Tower kernel: sas: trying to find task 0xf19223c0 (Drive related) Mar 6 12:50:51 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf19223c0 (Drive related) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1703:<7>mv_abort_task() mvi=f7400000 task=f19223c0 slot=f74115d4 slot_idx=x0 (System) Mar 6 12:50:51 Tower kernel: sas: sas_scsi_find_task: querying task 0xf19223c0 (Drive related) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1632:mvs_query_task:rc= 5 (System) Mar 6 12:50:51 Tower kernel: sas: sas_scsi_find_task: task 0xf19223c0 failed to abort (Minor Issues) Mar 6 12:50:51 Tower kernel: sas: task 0xf19223c0 is not at LU: I_T recover (Drive related) Mar 6 12:50:51 Tower kernel: sas: I_T nexus reset for dev 0300000000000000 (Drive related) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x89800. (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001001 (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2111:phy3 Unplug Notice (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001081 (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x10000 (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2138:notify plug in on phy[3] (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1224:port 3 attach dev info is 0 (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1226:port 3 attach sas addr is 3 (System) Mar 6 12:50:51 Tower kernel: drivers/scsi/mvsas/mv_sas.c 378:phy 3 byte dmaded. (System) Mar 6 12:50:51 Tower kernel: sas: sas_form_port: phy3 belongs to port3 already(1)! (Drive related) Mar 6 12:50:53 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1586:mvs_I_T_nexus_reset for device[3]:rc= 0 (System) Mar 6 12:50:53 Tower kernel: sas: I_T 0300000000000000 recovered (Drive related) Mar 6 12:50:53 Tower kernel: sas: sas_ata_task_done: SAS error 8d (Errors) Mar 6 12:50:53 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:50:53 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:50:53 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:50:53 Tower kernel: sas: --- Exit sas_scsi_recover_host (Drive related) Mar 6 12:50:53 Tower kernel: sas: sas_to_ata_err: Saw error 2. What to do? (Errors) Mar 6 12:50:53 Tower kernel: sas: sas_ata_task_done: SAS error 2 (Errors) Mar 6 12:50:53 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:50:53 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:50:53 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:51:24 Tower kernel: sas: command 0xf1980600, task 0xf19223c0, timed out: BLK_EH_NOT_HANDLED (Drive related) Mar 6 12:51:24 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related) Mar 6 12:51:24 Tower kernel: sas: trying to find task 0xf19223c0 (Drive related) Mar 6 12:51:24 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf19223c0 (Drive related) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1703:<7>mv_abort_task() mvi=f7400000 task=f19223c0 slot=f74115d4 slot_idx=x0 (System) Mar 6 12:51:24 Tower kernel: sas: sas_scsi_find_task: querying task 0xf19223c0 (Drive related) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1632:mvs_query_task:rc= 5 (System) Mar 6 12:51:24 Tower kernel: sas: sas_scsi_find_task: task 0xf19223c0 failed to abort (Minor Issues) Mar 6 12:51:24 Tower kernel: sas: task 0xf19223c0 is not at LU: I_T recover (Drive related) Mar 6 12:51:24 Tower kernel: sas: I_T nexus reset for dev 0300000000000000 (Drive related) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x89800. (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001 (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2111:phy3 Unplug Notice (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1081 (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x10000 (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2138:notify plug in on phy[3] (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1224:port 3 attach dev info is 0 (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1226:port 3 attach sas addr is 3 (System) Mar 6 12:51:24 Tower kernel: drivers/scsi/mvsas/mv_sas.c 378:phy 3 byte dmaded. (System) Mar 6 12:51:24 Tower kernel: sas: sas_form_port: phy3 belongs to port3 already(1)! (Drive related) Mar 6 12:51:26 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1586:mvs_I_T_nexus_reset for device[3]:rc= 0 (System) Mar 6 12:51:26 Tower kernel: sas: I_T 0300000000000000 recovered (Drive related) Mar 6 12:51:26 Tower kernel: sas: sas_ata_task_done: SAS error 8d (Errors) Mar 6 12:51:26 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:51:26 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:51:26 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:51:26 Tower kernel: sas: --- Exit sas_scsi_recover_host (Drive related) Mar 6 12:51:26 Tower kernel: sas: sas_to_ata_err: Saw error 2. What to do? (Errors) Mar 6 12:51:26 Tower kernel: sas: sas_ata_task_done: SAS error 2 (Errors) Mar 6 12:51:26 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:51:26 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:51:26 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:52:03 Tower kernel: sas: command 0xf3e56240, task 0xf19223c0, timed out: BLK_EH_NOT_HANDLED (Drive related) Mar 6 12:52:03 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related) Mar 6 12:52:03 Tower kernel: sas: trying to find task 0xf19223c0 (Drive related) Mar 6 12:52:03 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf19223c0 (Drive related) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1703:<7>mv_abort_task() mvi=f7400000 task=f19223c0 slot=f74115d4 slot_idx=x0 (System) Mar 6 12:52:03 Tower kernel: sas: sas_scsi_find_task: querying task 0xf19223c0 (Drive related) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1632:mvs_query_task:rc= 5 (System) Mar 6 12:52:03 Tower kernel: sas: sas_scsi_find_task: task 0xf19223c0 failed to abort (Minor Issues) Mar 6 12:52:03 Tower kernel: sas: task 0xf19223c0 is not at LU: I_T recover (Drive related) Mar 6 12:52:03 Tower kernel: sas: I_T nexus reset for dev 0300000000000000 (Drive related) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x89800. (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001001 (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2111:phy3 Unplug Notice (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x1001081 (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2083:port 3 ctrl sts=0x199800. (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2085:Port 3 irq sts = 0x10000 (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2138:notify plug in on phy[3] (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1224:port 3 attach dev info is 0 (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1226:port 3 attach sas addr is 3 (System) Mar 6 12:52:03 Tower kernel: drivers/scsi/mvsas/mv_sas.c 378:phy 3 byte dmaded. (System) Mar 6 12:52:03 Tower kernel: sas: sas_form_port: phy3 belongs to port3 already(1)! (Drive related) Mar 6 12:52:05 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1586:mvs_I_T_nexus_reset for device[3]:rc= 0 (System) Mar 6 12:52:05 Tower kernel: sas: I_T 0300000000000000 recovered (Drive related) Mar 6 12:52:05 Tower kernel: sas: sas_ata_task_done: SAS error 8d (Errors) Mar 6 12:52:05 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:52:05 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:52:05 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors) Mar 6 12:52:05 Tower kernel: sas: --- Exit sas_scsi_recover_host (Drive related) Mar 6 12:52:05 Tower kernel: sas: sas_to_ata_err: Saw error 2. What to do? (Errors) Mar 6 12:52:05 Tower kernel: sas: sas_ata_task_done: SAS error 2 (Errors) Mar 6 12:52:05 Tower kernel: ata10: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00 (Drive related) Mar 6 12:52:05 Tower kernel: ata10: status=0x01 { Error } (Errors) Mar 6 12:52:05 Tower kernel: ata10: error=0x04 { DriveStatusError } (Errors)
March 6, 201214 yr Author I take it back.. it's not dead.. but here are more errors.. this occurs several more times in the syslog Mar 6 13:14:55 Tower kernel: sd 0:0:3:0: [sdi] ASC=0x0 ASCQ=0x0 (Drive related) Mar 6 13:14:55 Tower kernel: sd 0:0:3:0: [sdi] CDB: cdb[0]=0x28: 28 00 80 43 a1 47 00 02 00 00 (Drive related) Mar 6 13:14:55 Tower kernel: end_request: I/O error, dev sdi, sector 2151915847 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915784/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915792/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915800/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915808/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915816/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915824/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915832/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915840/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915848/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915856/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915864/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915872/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915880/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915888/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915896/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915904/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915912/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915920/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915928/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915936/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915944/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915952/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915960/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915968/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915976/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915984/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151915992/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916000/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916008/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916016/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916024/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916032/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916040/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916048/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916056/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916064/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916072/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916080/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916088/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916096/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916104/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916112/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916120/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916128/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916136/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916144/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916152/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916160/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916168/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916176/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916184/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916192/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916200/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916208/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916216/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916224/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916232/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916240/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916248/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916256/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916264/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916272/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916280/3, count: 1 (Errors) Mar 6 13:14:55 Tower kernel: md: disk4 read error (Errors) Mar 6 13:14:55 Tower kernel: handle_stripe read error: 2151916288/3, count: 1 (Errors) Mar 6 13:15:26 Tower kernel: sas: command 0xf3c4d480, task 0xf19223c0, timed out: BLK_EH_NOT_HANDLED (Drive related) Mar 6 13:15:26 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related) Mar 6 13:15:26 Tower kernel: sas: trying to find task 0xf19223c0 (Drive related) Mar 6 13:15:26 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf19223c0 (Drive related) Consequently disk4 is full... if that means anything.. So my problem might have nothing to do with my monthly parity check.. This might be new issue
March 6, 201214 yr Author So I took a look at the SMART data for disk 4 and it looks like there are some issue.. OK disk4 sdi 4624 SAMSUNG HD154UI 1AG01118 8 » calibration_retry_count=1 » reported_uncorrect=649 » current_pending_sector=23 » offline_uncorrectable=1 » soft_read_error_rate=101 The question is.. has this been my problem all along? or is this coincidental? The machine didn't crash this time... It slowed way down for a while (it back humming along @ ~50MB/s) Maybe because I killed cache_dirs before the parity check got too far?? Now do I trust that the machine will complete a disk rebuild? or should I try to move all the data off that disk and then take it out of the system? Thanks for the help so far, Jim
March 6, 201214 yr This could be the cause or the result of the crashes. Disk4 needs to be rebuilt. Replace it with a spare and the run pre-clear on the problem disk. If no spare, then pre-clear the problem disk and observe the current_pending_sector and reallocated counts. If the disk passes pre-clear then rebuild disk4.
March 6, 201214 yr So I took a look at the SMART data for disk 4 and it looks like there are some issue.. OK disk4 sdi 4624 SAMSUNG HD154UI 1AG01118 8 » calibration_retry_count=1 » reported_uncorrect=649 » current_pending_sector=23 » offline_uncorrectable=1 » soft_read_error_rate=101 The question is.. has this been my problem all along? or is this coincidental? The machine didn't crash this time... It slowed way down for a while (it back humming along @ ~50MB/s) Maybe because I killed cache_dirs before the parity check got too far?? Now do I trust that the machine will complete a disk rebuild? or should I try to move all the data off that disk and then take it out of the system? Thanks for the help so far, Jim I hope you are doing a non-correcting parity check. Best bet, copy anything precious off of the failing drive and store elsewhere. Then un-assign the drive with the sectors pending re-allcation. Start the array with it un-assigned so unRAID forgets the serial number of the drive. You can then run badblocks on the bad drive, or run a preclear on it, then, if you think it is OK, re-assign it and let unRAID re-construct your data onto it. If it has 650 sectors already re-allocated, you might just want to test that drive's ability to serve as a wheel chock on your car. Place it behind the wheel of your car and repeatedly drive over it, testing its ability to impede your motion. If it cannot stop your forward and back motions, RMA it. If it can, RMA it using a bigger box. Joe L.
March 6, 201214 yr Author I hope you are doing a non-correcting parity check. No.. This was the auto default parity check with the correct feature checked whne the machine started.. Best bet, copy anything precious off of the failing drive and store elsewhere. Then un-assign the drive with the sectors pending re-allcation. Start the array with it un-assigned so unRAID forgets the serial number of the drive. You can then run badblocks on the bad drive, or run a preclear on it, then, if you think it is OK, re-assign it and let unRAID re-construct your data onto it. If it has 650 sectors already re-allocated, you might just want to test that drive's ability to serve as a wheel chock on your car. Place it behind the wheel of your car and repeatedly drive over it, testing its ability to impede your motion. If it cannot stop your forward and back motions, RMA it. If it can, RMA it using a bigger box. Joe L. I think I will start copying the data to another drive! That will take a while since it's full! It didn't have sync errors yet.. and it's past disk4 now.. So I think I'll just let it finish.
March 7, 201214 yr Author Well.. Drive 4 red-balled today! So now it's running with recalculated data, I assume. The parity check completed without error (before red balling), which makes me feel better. Right now I'm in the process of taking all the data off that drive and moving it onto others! My drive is still under warranty.. Hopefully I'll have enough "proof" that the drive is bad so that they will replace it. Now I just have to remember how to remove the drive from the array once it's empty so I don't have to run the array unprotected while I test/RMA the old drive!
March 8, 201214 yr Author Ok.. drive 4 is empty... I unassigned it from the webconsole. It's reported as missing now and says If I start the array I'll be unprotected. So now do I want to run the initconfig from the commandline to have it forget about disk 4 and rebuild parity? I just want to be sure before I type the command! Jim
March 9, 201214 yr Ok.. drive 4 is empty... I unassigned it from the webconsole. It's reported as missing now and says If I start the array I'll be unprotected. So now do I want to run the initconfig from the commandline to have it forget about disk 4 and rebuild parity? I just want to be sure before I type the command! Jim Or, if on your version of unRAID in the Utils tab there is a "New config" option, it will do the same. If you do use the command line, answer exactly "Yes" to its prompt... (Capital "Y", lower case "es")
April 2, 201214 yr Author Well.. I was hoping that removing the questionable drive would make my monthly crash go away.. Well.. It hasn't! It crashed again! I also do think it's not the power supply since I'm one drive less and, in theory, have ample head room! I wish I could tell when it crashed to know if it was immediate or sometime in the parity check!
April 2, 201214 yr I had all kinda issues until I replaced my powersupply with an OCZ... now it's scary stable. and... memory is cheap right now.. take it off line and let it run memtest overnight with the PSU you have.
October 3, 201213 yr Author Crud!!! It's been fine for a while and then yesterday it hung! Not crashed.. Hung. It did this several times while trying to access the array for a write. I just ordered a seasonic X750. I'll throw it in this weekend although I don't think that's my issue. (I needed a new PS for something else anyway)..
November 2, 201213 yr Author Well I"m back to square one! I swapped out the PS sometime last month with a nice seasonic one. I was hoping that my crashes were a thing of the past! Well last night after the monthly parity check kicked off, it crashed... my guess is right after an automatic pc backup started writing to the array. I restarted this AM and all was going well... then I started copying a DVD to the array and it crashed again. I'm pretty sure that if I don't try to write to the array till after it's done, , I'll be fine. Where do I look next? Should I do the memtest thing? I'm thinking that's not it because I run cache_dirs and have no issues... Should I just upgrade to the latest unraid and see if it just all goes away? Should I install linux/windows on some separate drive and load a utility to beat the hell out of the machine to see if it's stable? I can live with the crashes, but I shouldn't have to. It's really annoying!
Archived
This topic is now archived and is closed to further replies.