January 23, 201115 yr Author So it looks like I made a big mistake when I ran the reiserfsck on /dev/sde rather than /dev/sde1. Luckily I cloned that drive onto the new 1TB drive last night with dd_rescue. From there I ran reiserfsck on the new cloned drive that I made (which was /dev/sdd1 in this case) which fixed a few errors and voila, I was able to mount and browse the disk. From the web menu I now had the option to start the array with all disks accounted for. The array is currently running a parity check on the system and is detecting the replaced drive as 1Tb with 750GB used (as the original drive was an almost full 750GB drive this is correct). So in approximately 3000 and some odd minutes my array should be as it was originally again and safe with parity! Now however there is the issue of troubleshooting the bad SATA connection that originally caused this headache. Some of you noticed issues in the system log, could you elaborate? What lines in particular should I be looking at? Like I said previously nothing new has been done to the array and there is a quality 450w power supply installed so I don’t think there is an issue there. Thus I will be trying to determine if it is either the SATA card, the cable or the Disk Enclosure.
January 23, 201115 yr 2 days to run parity? you are a patient man I'm pretty upset that mine takes 6-8 hours
January 23, 201115 yr You asked for more syslog-related explanation, so I'll try. The following is rather long, but I hope it will be useful, perhaps to others also. First, some extracts from 4 different syslogs, tracking 3 different ata channels. One caution, the fact that some of the SCSI numbers, ata numbers, host numbers, and Disk numbers are similar or equal is usually only coincidental. Disk equipment and device ID's: 8 port SATA disk controller (pci-0000:01:01.0), supporting pci-0000:01:01.0-scsi-0:0:0:0 through pci-0000:01:01.0-scsi-7:0:0:0, ata1 through ata8, uses sata_mv (the Marvell driver) 4 port SATA disk controller (pci-0000:01:02.0), supporting pci-0000:01:02.0-scsi-0:0:0:0 through pci-0000:01:02.0-scsi-3:0:0:0, ata9 through ata12, uses sata_sil (the older Silicon Image driver) 2, 4, or 6 motherboard SATA ports (pci-0000:00:1f.2), supporting pci-0000:00:1f.2-scsi-1:0:0:0 etc, ata13 and ata14, currently in an emulated IDE mode with issues, would be better in AHCI mode, currently using ata_piix, would be better using ahci Syslog (roguewarr) 22_01_2011.txt : no Disk 7, this syslog shows that Disk 7 was originally connected to ata4, the 4th port on 8 port SATA card Jan 22 18:16:55 Tower kernel: ata4: SATA link down (SStatus 0 SControl 300) ... Jan 22 18:16:55 Tower kernel: ata7: link is slow to respond, please be patient (ready=0) Jan 22 18:16:55 Tower kernel: ata7: SRST failed (errno=-16) Jan 22 18:16:55 Tower kernel: ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jan 22 18:16:55 Tower kernel: ata7.00: ATA-7: WDC WD7500AYYS-01RCA0, 30.04G30, max UDMA/133 Jan 22 18:16:55 Tower kernel: ata7.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32) Jan 22 18:16:55 Tower kernel: ata7.00: configured for UDMA/133 Jan 22 18:16:55 Tower kernel: scsi 6:0:0:0: Direct-Access ATA WDC WD7500AYYS-0 30.0 PQ: 0 ANSI: 5 Jan 22 18:16:55 Tower kernel: sd 6:0:0:0: [sdg] 1465149168 512-byte logical blocks: (750 GB/698 GiB) Jan 22 18:16:55 Tower kernel: sd 6:0:0:0: [sdg] Write Protect is off Jan 22 18:16:55 Tower kernel: sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00 Jan 22 18:16:55 Tower kernel: sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jan 22 18:16:55 Tower kernel: sdg: sdg1 ... Jan 22 18:16:55 Tower kernel: ata8: SATA link down (SStatus 0 SControl 300) ... Jan 22 18:16:55 Tower emhttp: Device inventory: Jan 22 18:16:55 Tower emhttp: pci-0000:00:1f.2-scsi-1:0:0:0 host14 (sdl) ST31500341AS_9VS40APX Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-0:0:0:0 host0 (sda) WDC_WD20EADS-00R6B0_WD-WCAVY0681517 Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-1:0:0:0 host1 (sdb) SAMSUNG_HD103SJ_S246JDWZ505547 Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-2:0:0:0 host2 (sdc) WDC_WD10EACS-00ZJB0_WD-WCASJ2212341 Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-4:0:0:0 host4 (sdd) WDC_WD20EADS-00R6B0_WD-WCAVY0725247 Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-5:0:0:0 host5 (sde) WDC_WD10EACS-00ZJB0_WD-WCASJ1871740 Jan 22 18:16:55 Tower emhttp: pci-0000:01:01.0-scsi-6:0:0:0 host6 (sdg) WDC_WD7500AYYS-01RCA0_WD-WCAPT0514867 Jan 22 18:16:55 Tower emhttp: pci-0000:01:02.0-scsi-0:0:0:0 host9 (sdh) WDC_WD20EARS-00J2GB0_WD-WCAYY0097406 Jan 22 18:16:55 Tower emhttp: pci-0000:01:02.0-scsi-1:0:0:0 host10 (sdi) WDC_WD7500AAKS-00RBA0_WD-WCAPT0261038 Jan 22 18:16:55 Tower emhttp: pci-0000:01:02.0-scsi-2:0:0:0 host11 (sdj) ST31000528AS_9VP0W2GT Jan 22 18:16:55 Tower emhttp: pci-0000:01:02.0-scsi-3:0:0:0 host12 (sdk) WDC_WD7500AYYS-01RCA0_WD-WCAPT0516136 Jan 22 18:16:55 Tower emhttp: restart_md_driver: stat pci-0000:01:01.0-scsi-3:0:0:0: No such file or directory ... Jan 22 18:16:56 Tower kernel: md: disk7 removed The restart line indicates that the system was configured (in disk.cfg) for another drive on that scsi port, corresponding with ata4, but no drive was found there. Syslog (roguewarr) 22_01_2011_Withdrive.txt : Disk 7 is reinstalled, connected to ata8, the 8th port on 8 port SATA card Jan 22 21:26:45 Tower kernel: ata4: SATA link down (SStatus 0 SControl 300) ... Jan 22 21:26:45 Tower kernel: ata7: link is slow to respond, please be patient (ready=0) Jan 22 21:26:45 Tower kernel: ata7: SRST failed (errno=-16) Jan 22 21:26:45 Tower kernel: ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jan 22 21:26:45 Tower kernel: ata7.00: ATA-7: WDC WD7500AYYS-01RCA0, 30.04G30, max UDMA/133 Jan 22 21:26:45 Tower kernel: ata7.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32) Jan 22 21:26:45 Tower kernel: ata7.00: configured for UDMA/133 Jan 22 21:26:45 Tower kernel: scsi 6:0:0:0: Direct-Access ATA WDC WD7500AYYS-0 30.0 PQ: 0 ANSI: 5 Jan 22 21:26:45 Tower kernel: sd 6:0:0:0: [sdg] 1465149168 512-byte logical blocks: (750 GB/698 GiB) Jan 22 21:26:45 Tower kernel: sd 6:0:0:0: [sdg] Write Protect is off Jan 22 21:26:45 Tower kernel: sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00 Jan 22 21:26:45 Tower kernel: sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jan 22 21:26:45 Tower kernel: sdg: sdg1 ... Jan 22 21:26:45 Tower kernel: ata8: link is slow to respond, please be patient (ready=0) Jan 22 21:26:45 Tower kernel: ata8: SRST failed (errno=-16) Jan 22 21:26:45 Tower kernel: ata8: link is slow to respond, please be patient (ready=0) Jan 22 21:26:45 Tower kernel: ata8: SRST failed (errno=-16) Jan 22 21:26:45 Tower kernel: ata8: link is slow to respond, please be patient (ready=0) Jan 22 21:26:45 Tower kernel: ata8: SRST failed (errno=-16) Jan 22 21:26:45 Tower kernel: ata8: limiting SATA link speed to 1.5 Gbps Jan 22 21:26:45 Tower kernel: ata8: COMRESET failed (errno=-32) Jan 22 21:26:45 Tower kernel: ata8: reset failed, giving up ... Jan 22 21:26:45 Tower emhttp: Device inventory: Jan 22 21:26:45 Tower emhttp: pci-0000:00:1f.2-scsi-1:0:0:0 host14 (sdl) ST31500341AS_9VS40APX Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-0:0:0:0 host0 (sda) WDC_WD20EADS-00R6B0_WD-WCAVY0681517 Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-1:0:0:0 host1 (sdb) SAMSUNG_HD103SJ_S246JDWZ505547 Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-2:0:0:0 host2 (sdc) WDC_WD10EACS-00ZJB0_WD-WCASJ2212341 Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-4:0:0:0 host4 (sdd) WDC_WD20EADS-00R6B0_WD-WCAVY0725247 Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-5:0:0:0 host5 (sde) WDC_WD10EACS-00ZJB0_WD-WCASJ1871740 Jan 22 21:26:45 Tower emhttp: pci-0000:01:01.0-scsi-6:0:0:0 host6 (sdg) WDC_WD7500AYYS-01RCA0_WD-WCAPT0514867 Jan 22 21:26:45 Tower emhttp: pci-0000:01:02.0-scsi-0:0:0:0 host9 (sdh) WDC_WD20EARS-00J2GB0_WD-WCAYY0097406 Jan 22 21:26:45 Tower emhttp: pci-0000:01:02.0-scsi-1:0:0:0 host10 (sdi) WDC_WD7500AAKS-00RBA0_WD-WCAPT0261038 Jan 22 21:26:45 Tower emhttp: pci-0000:01:02.0-scsi-2:0:0:0 host11 (sdj) ST31000528AS_9VP0W2GT Jan 22 21:26:45 Tower emhttp: pci-0000:01:02.0-scsi-3:0:0:0 host12 (sdk) WDC_WD7500AYYS-01RCA0_WD-WCAPT0516136 Jan 22 21:26:46 Tower emhttp: restart_md_driver: stat pci-0000:01:01.0-scsi-3:0:0:0: No such file or directory ... Jan 22 21:26:46 Tower kernel: md: disk7 removed The kernel can see a drive connected on ata8, but was not successful in communicating with it. Syslog (roguewarr) 1009_23-01-11.txt : installed the new disk (copy of Disk 7), connected to ata8, the 8th port on 8 port SATA card; "took almost 10x longer to boot normal" Jan 23 09:57:11 Tower kernel: ata4: SATA link down (SStatus 0 SControl 300) ... Jan 23 09:57:11 Tower kernel: ata7: link is slow to respond, please be patient (ready=0) Jan 23 09:57:11 Tower kernel: ata7: SRST failed (errno=-16) Jan 23 09:57:11 Tower kernel: ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jan 23 09:57:11 Tower kernel: ata7.00: ATA-7: WDC WD7500AYYS-01RCA0, 30.04G30, max UDMA/133 Jan 23 09:57:11 Tower kernel: ata7.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32) Jan 23 09:57:11 Tower kernel: ata7.00: configured for UDMA/133 Jan 23 09:57:11 Tower kernel: scsi 6:0:0:0: Direct-Access ATA WDC WD7500AYYS-0 30.0 PQ: 0 ANSI: 5 Jan 23 09:57:11 Tower kernel: sd 6:0:0:0: [sdg] 1465149168 512-byte logical blocks: (750 GB/698 GiB) Jan 23 09:57:11 Tower kernel: sd 6:0:0:0: [sdg] Write Protect is off Jan 23 09:57:11 Tower kernel: sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00 Jan 23 09:57:11 Tower kernel: sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jan 23 09:57:11 Tower kernel: sdg: sdg1 ... (no ata8!) ... (ata9 through ata14 (the SiI3114 card and the motherboard ports) are incomplete, only 2 of the drives identified, and none are setup) ... Jan 23 09:57:11 Tower kernel: ata14.00: configured for UDMA/133 Jan 23 10:00:11 Tower logger: /etc/rc.d/rc.inet1: /sbin/ifconfig lo 127.0.0.1 ... Jan 23 10:00:11 Tower emhttp: Device inventory: Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-0:0:0:0 host0 (sda) WDC_WD20EADS-00R6B0_WD-WCAVY0681517 Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-1:0:0:0 host1 (sdb) SAMSUNG_HD103SJ_S246JDWZ505547 Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-2:0:0:0 host2 (sdc) WDC_WD10EACS-00ZJB0_WD-WCASJ2212341 Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-4:0:0:0 host4 (sdd) WDC_WD20EADS-00R6B0_WD-WCAVY0725247 Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-5:0:0:0 host5 (sde) WDC_WD10EACS-00ZJB0_WD-WCASJ1871740 Jan 23 10:00:11 Tower emhttp: pci-0000:01:01.0-scsi-6:0:0:0 host6 (sdg) WDC_WD7500AYYS-01RCA0_WD-WCAPT0514867 Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:01:01.0-scsi-3:0:0:0: No such file or directory Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:01:02.0-scsi-0:0:0:0: No such file or directory Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:01:02.0-scsi-1:0:0:0: No such file or directory Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:01:02.0-scsi-3:0:0:0: No such file or directory Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:01:02.0-scsi-2:0:0:0: No such file or directory Jan 23 10:00:11 Tower emhttp: restart_md_driver: stat pci-0000:00:1f.2-scsi-1:0:0:0: No such file or directory ... (does not attempt to start array, so no drive import and assignment table) The system clearly hung on setup of ata8, no reports from ata8 at all! The subsequent drive analysis was apparently blocked on that, so only the drives identified prior to ata8 have been setup. unRAID apparently gives the kernel 3 full minutes to locate and setup all of the hardware, then proceeds on with whatever has been found at that time. Notice 6 drives not found, associated with the 6 restart lines. Syslog (roguewarr) 1303_23-01-11.txt : connected original Disk 7 to ata4, the 4th port on 8 port SATA card, assigned it to Disk 3 Jan 23 11:28:52 Tower kernel: ata4: link is slow to respond, please be patient (ready=0) Jan 23 11:28:52 Tower kernel: ata4: SRST failed (errno=-16) Jan 23 11:28:52 Tower kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jan 23 11:28:52 Tower kernel: ata4.00: ATA-7: WDC WD7500AYYS-01RCA0, 30.04G30, max UDMA/133 Jan 23 11:28:52 Tower kernel: ata4.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32) Jan 23 11:28:52 Tower kernel: ata4.00: configured for UDMA/133 Jan 23 11:28:52 Tower kernel: scsi 3:0:0:0: Direct-Access ATA WDC WD7500AYYS-0 30.0 PQ: 0 ANSI: 5 Jan 23 11:28:52 Tower kernel: sd 3:0:0:0: [sde] 1465149168 512-byte logical blocks: (750 GB/698 GiB) Jan 23 11:28:52 Tower kernel: sd 3:0:0:0: [sde] Write Protect is off Jan 23 11:28:52 Tower kernel: sd 3:0:0:0: [sde] Mode Sense: 00 3a 00 00 Jan 23 11:28:52 Tower kernel: sd 3:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jan 23 11:28:52 Tower kernel: sde: sde1 ... Jan 23 11:28:52 Tower kernel: ata7: link is slow to respond, please be patient (ready=0) Jan 23 11:28:52 Tower kernel: ata7: SRST failed (errno=-16) Jan 23 11:28:52 Tower kernel: ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jan 23 11:28:52 Tower kernel: ata7.00: ATA-7: WDC WD7500AYYS-01RCA0, 30.04G30, max UDMA/133 Jan 23 11:28:52 Tower kernel: ata7.00: 1465149168 sectors, multi 0: LBA48 NCQ (depth 31/32) Jan 23 11:28:52 Tower kernel: ata7.00: configured for UDMA/133 Jan 23 11:28:52 Tower kernel: scsi 6:0:0:0: Direct-Access ATA WDC WD7500AYYS-0 30.0 PQ: 0 ANSI: 5 Jan 23 11:28:52 Tower kernel: sd 6:0:0:0: [sdh] 1465149168 512-byte logical blocks: (750 GB/698 GiB) Jan 23 11:28:52 Tower kernel: sd 6:0:0:0: [sdh] Write Protect is off Jan 23 11:28:52 Tower kernel: sd 6:0:0:0: [sdh] Mode Sense: 00 3a 00 00 Jan 23 11:28:52 Tower kernel: sd 6:0:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jan 23 11:28:52 Tower kernel: sdh: sdh1 ... Jan 23 11:28:52 Tower kernel: ata8: SATA link down (SStatus 0 SControl 300) ... Jan 23 11:28:52 Tower emhttp: Device inventory: Jan 23 11:28:52 Tower emhttp: pci-0000:00:1f.2-scsi-1:0:0:0 host14 (sdm) ST31500341AS_9VS40APX Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-0:0:0:0 host0 (sda) WDC_WD20EADS-00R6B0_WD-WCAVY0681517 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-1:0:0:0 host1 (sdb) SAMSUNG_HD103SJ_S246JDWZ505547 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-2:0:0:0 host2 (sdc) WDC_WD10EACS-00ZJB0_WD-WCASJ2212341 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-3:0:0:0 host3 (sde) WDC_WD7500AYYS-01RCA0_WD-WCAPT0519525 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-4:0:0:0 host4 (sdf) WDC_WD20EADS-00R6B0_WD-WCAVY0725247 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-5:0:0:0 host5 (sdg) WDC_WD10EACS-00ZJB0_WD-WCASJ1871740 Jan 23 11:28:52 Tower emhttp: pci-0000:01:01.0-scsi-6:0:0:0 host6 (sdh) WDC_WD7500AYYS-01RCA0_WD-WCAPT0514867 Jan 23 11:28:52 Tower emhttp: pci-0000:01:02.0-scsi-0:0:0:0 host9 (sdi) WDC_WD20EARS-00J2GB0_WD-WCAYY0097406 Jan 23 11:28:52 Tower emhttp: pci-0000:01:02.0-scsi-1:0:0:0 host10 (sdj) WDC_WD7500AAKS-00RBA0_WD-WCAPT0261038 Jan 23 11:28:52 Tower emhttp: pci-0000:01:02.0-scsi-2:0:0:0 host11 (sdk) ST31000528AS_9VP0W2GT Jan 23 11:28:52 Tower emhttp: pci-0000:01:02.0-scsi-3:0:0:0 host12 (sdl) WDC_WD7500AYYS-01RCA0_WD-WCAPT0516136 ... Jan 23 11:28:53 Tower kernel: md: import disk3: [8,64] (sde) WDC WD7500AYYS-0 WD-WCAPT0519525 offset: 63 size: 732574552 Jan 23 11:28:53 Tower kernel: md: disk3 new disk ... Jan 23 11:28:53 Tower kernel: md: disk7 removed Notice that the handling of ata4 is identical to ata7, happens to be identical drive models. No restart lines! Careful analysis of the above leads to 2 different possibilities, either or both of which could be causing the issues. Possible issue #1: this WD model (WDC_WD7500AYYS-01RCA0) is partly incompatible with the 8 port controller; the only reason I included ata7 above is to show that it has the same model of drive (but serial ending in 867), and it consistently has some difficulty on startup, identical to the Disk 7 on ata4 in the 4th syslog; a flaw with this possibility is that the 'new disk' had even more severe problems when connected to ata8, but while I know it is a different size (1TB), I don't know if it is a very similar model, and also incompatible Troubleshooting suggestion: move all of these WD models off the 8 port card, and connect other drives to each of the 8 port SATA connectors, then check a subsequent syslog. It should clearly prove or disprove this possibility. Possible issue #2: the 8th SATA port has failed, or unreliable (possibly other SATA ports affected too, such as ata4 and ata7) Troubleshooting suggestion: connect a known good drive (but not any of the suspect WD model) to ata8, and check subsequent syslog I don't see any indicators of bad SATA cables. Power is always a possible issue, although you feel you have a good one. Could you elaborate on what you mean by the "Disk Enclosure"? The initial slow speed may be due to fixing the parity errors, it should dramatically increase after it passes the point of the earlier abort. I noted above that there were issues with the onboard ports. If Dell hasn't limited access to it, in the BIOS settings prior to booting, you should be able to change the handling of the SATA drives supported onboard to use an AHCI mode, or at least a native SATA mode, anything else but IDE modes. This is completely irrelevant to any of your issues though, and if you cannot change it, it does work OK.
Archived
This topic is now archived and is closed to further replies.