December 16, 201015 yr Hi, I recently upgraded to 4.5.. range after a real long time running 4.3.3 without issue. Since upgrading I've encountered numerous server issues where I lose connection with it (It appears to hang) and no access is at all possible and a hard reset is required. Finally, I managed to catch a tailed log of 3 crashes - each one with differing results. Crash 1 - Here is the part of the tailed log. All discs where spun down at the time and nothing was accessing the server. The ata8 issue is a known one (seagate drive about to be removed - my last). The crash stems from the ata9 error - this being an interesting bit "Tower kernel: ata9: SRST failed (errno=-16)" Dec 13 12:06:28 Tower kernel: ata9: exception Emask 0x10 SAct 0x0 SErr 0x180000 action 0xe frozen Dec 13 12:06:28 Tower kernel: ata9: hotplug_status 0x8 Dec 13 12:06:28 Tower kernel: ata9: SError: { 10B8B Dispar } Dec 13 12:06:28 Tower kernel: ata9: hard resetting link Dec 13 12:06:34 Tower kernel: ata9: link is slow to respond, please be patient ( ready=-19) Dec 13 12:06:37 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:06:37 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:06:37 Tower kernel: ata8: EH complete Dec 13 12:06:37 Tower kernel: mdcmd (111): spindown 6 Dec 13 12:06:38 [color=red]Tower kernel: ata9: SRST failed (errno=-16)[/color] Dec 13 12:06:38 Tower kernel: ata9: hard resetting link Dec 13 12:06:39 Tower kernel: ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:06:39 Tower kernel: ata9.00: configured for UDMA/133 Dec 13 12:06:39 Tower kernel: ata9: EH complete Dec 13 12:06:39 Tower kernel: mdcmd (112): spindown 12 Dec 13 12:06:44 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:06:44 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:06:44 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:06:44 Tower kernel: ata8: hard resetting link Dec 13 12:06:52 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:06:52 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:06:52 Tower kernel: ata8: EH complete Dec 13 12:06:52 Tower kernel: mdcmd (113): spindown 6 Dec 13 12:07:00 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:07:00 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:07:00 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:07:00 Tower kernel: ata8: hard resetting link Dec 13 12:07:08 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:07:08 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:07:08 Tower kernel: ata8: EH complete Dec 13 12:07:08 Tower kernel: mdcmd (114): spindown 6 Dec 13 12:07:17 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:07:17 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:07:17 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:07:17 Tower kernel: ata8: hard resetting link Dec 13 12:07:24 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:07:24 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:07:24 Tower kernel: ata8: EH complete Dec 13 12:07:24 Tower kernel: mdcmd (115): spindown 6 Dec 13 12:07:34 Tower kernel: mdcmd (116): spindown 6 Dec 13 12:13:46 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:13:46 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:13:46 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:13:46 Tower kernel: ata8: hard resetting link Dec 13 12:13:55 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:13:55 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:13:55 Tower kernel: ata8: EH complete Dec 13 12:13:55 Tower kernel: mdcmd (117): spindown 6 Dec 13 12:14:06 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:14:06 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:14:06 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:14:06 Tower kernel: ata8: hard resetting link Dec 13 12:14:14 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:14:15 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:14:15 Tower kernel: ata8: EH complete Dec 13 12:14:15 Tower kernel: mdcmd (118): spindown 6 Dec 13 12:15:26 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:15:26 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:15:26 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:15:26 Tower kernel: ata8: hard resetting link Dec 13 12:15:36 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:15:36 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:15:36 Tower kernel: ata8: EH complete Dec 13 12:15:36 Tower kernel: mdcmd (119): spindown 6 Dec 13 12:19:53 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:19:53 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:19:53 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:19:53 Tower kernel: ata8: hard resetting link Dec 13 12:20:03 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:20:03 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:20:03 Tower kernel: ata8: EH complete Dec 13 12:20:03 Tower kernel: mdcmd (120): spindown 6 Dec 13 12:27:11 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:27:11 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:27:11 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:27:11 Tower kernel: ata8: hard resetting link Dec 13 12:27:20 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:27:20 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:27:20 Tower kernel: ata8: EH complete Dec 13 12:27:20 Tower kernel: mdcmd (121): spindown 6 Dec 13 12:27:37 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:27:37 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:27:37 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:27:37 Tower kernel: ata8: hard resetting link Dec 13 12:27:46 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:27:46 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:27:46 Tower kernel: ata8: EH complete Dec 13 12:27:46 Tower kernel: mdcmd (122): spindown 6 Dec 13 12:28:13 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:28:13 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:28:13 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:28:13 Tower kernel: ata8: hard resetting link Dec 13 12:28:22 Tower kernel: ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Dec 13 12:28:22 Tower kernel: ata8.00: configured for UDMA/33 Dec 13 12:28:22 Tower kernel: ata8: EH complete Dec 13 12:28:22 Tower kernel: mdcmd (123): spindown 6 Dec 13 12:39:16 Tower kernel: ata8: exception Emask 0x10 SAct 0x0 SErr 0x990000 action 0xe frozen Dec 13 12:39:16 Tower kernel: ata8: irq_stat 0x00400000, PHY RDY changed Dec 13 12:39:16 Tower kernel: ata8: SError: { PHYRdyChg 10B8B Dispar LinkSeq } Dec 13 12:39:16 Tower kernel: ata8: hard resetting link Dec 13 12:39:16 Tower kernel: ata9: exception Emask 0x10 SAct 0x0 SErr 0x180000 action 0xe frozen Dec 13 12:39:16 Tower kernel: ata9: hotplug_status 0x8 Dec 13 12:39:16 Tower kernel: ata9: SError: { 10B8B Dispar } Dec 13 12:39:16 Tower kernel: ata9: hard resetting link Crash 2 - ata9 again seems to cause the problem, but this time the log reports something different. In this crash all the drives were spun up (it happened shortly after starting the server) but there was nothing accessing them. Dec 16 15:49:00 Tower kernel: ata9.00: exception Emask 0x0 SAct 0x0 SErr 0x0 act ion 0x6 frozen Dec 16 15:49:00 Tower kernel: ata9.00: failed command: CHECK POWER MODE Dec 16 15:49:00 Tower kernel: ata9.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 t ag 0 Dec 16 15:49:00 Tower kernel: res 40/00:ff:00:00:00/00:00:00:00:00/00 E mask 0x4 (timeout) Dec 16 15:49:00 Tower kernel: ata9.00: status: { DRDY } Dec 16 15:49:00 Tower kernel: ata9: hard resetting link Crash 3 - Even though the server was tailed there was no output in the log (very strange). In this instance, all the drives were spun down except the parity and disk7. I was about to copy something and opening the location window seemed to trigger it. ata9 is tied to Disk 12. I googled the output and it always brought me to linux bug reports. Not sure what this all means. All my H/W is in my sig. I have decided revert to 4.3.3. Can someone help me with that, as I've never down-graded before and don't know the steps. Couldn't find it in the wiki. I saved the 4.3.3 setup before upgrading, but my config has changed since then. Can I just copy the 4.3.3 bzroot, bzimage to the USB and we're good to go??
December 16, 201015 yr Kernel bug should have output like this one, you should see "Call Trace". Yours looks more like SATA error. Sep 26 11:24:57 Tower kernel: BUG: unable to handle kernel paging request at f971f000 Sep 26 11:24:57 Tower kernel: IP: [] md_cmd_proc_read+0x41/0x54 [md_mod] Sep 26 11:24:57 Tower kernel: *pdpt = 0000000001447001 *pde = 000000002d452067 *pte = 0000000000000000 Sep 26 11:24:57 Tower kernel: Oops: 0000 [#1] SMP Sep 26 11:24:57 Tower kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/host0/port-0:5/end_device-0:5/target0:0:5/0:0:5:0/block/sds/stat Sep 26 11:24:57 Tower kernel: Modules linked in: md_mod xor i2c_i801 i2c_core sata_promise pata_jmicron jmicron ahci sata_mv e1000e mvsas libsas scst scsi_transport_sas [last unloaded: md_mod] Sep 26 11:24:57 Tower kernel: Sep 26 11:24:57 Tower kernel: Pid: 1899, comm: emhttp Not tainted (2.6.32.9-unRAID #5) System Product Name Sep 26 11:24:57 Tower kernel: EIP: 0060:[] EFLAGS: 00210203 CPU: 0 Sep 26 11:24:57 Tower kernel: EIP is at md_cmd_proc_read+0x41/0x54 [md_mod] Sep 26 11:24:57 Tower kernel: EAX: c3469f38 EBX: fffffdfe ECX: 3ffff84e EDX: fffffdfe Sep 26 11:24:57 Tower kernel: ESI: f971f000 EDI: e6c5acc4 EBP: c3469f04 ESP: c3469ef4 Sep 26 11:24:57 Tower kernel: DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Sep 26 11:24:57 Tower kernel: Process emhttp (pid: 1899, ti=c3468000 task=c44686e0 task.ti=c3468000) Sep 26 11:24:57 Tower kernel: Stack: Sep 26 11:24:57 Tower kernel: c3469f38 f97158a3 00000000 e6c59000 c3469f4c c109d5f5 00000400 c3469f3c Sep 26 11:24:57 Tower kernel: <0> 00000000 00000400 00000000 00000400 b7834000 c3477c60 00000000 c3477c60 Sep 26 11:24:57 Tower kernel: <0> 00000400 e6c59000 00000001 c3477c60 c109d4ea fffffffb c3469f70 c1099f72 Sep 26 11:24:57 Tower kernel: Call Trace: Sep 26 11:24:57 Tower kernel: [] ? md_cmd_proc_read+0x0/0x54 [md_mod] Sep 26 11:24:57 Tower kernel: [] ? proc_file_read+0x10b/0x22d Sep 26 11:24:57 Tower kernel: [] ? proc_file_read+0x0/0x22d Sep 26 11:24:57 Tower kernel: [] ? proc_reg_read+0x56/0x6a Sep 26 11:24:57 Tower kernel: [] ? proc_reg_read+0x0/0x6a Sep 26 11:24:57 Tower kernel: [] ? vfs_read+0x8a/0x114 Sep 26 11:24:57 Tower kernel: [] ? sys_read+0x3b/0x60 Sep 26 11:24:57 Tower kernel: [] ? syscall_call+0x7/0xb
December 16, 201015 yr What you posted is NOT a Kernel Oops bug. The Kernel Oops bug was indeed fixed in the 4.6 release. What you're experiencing is a normal SATA error possibly caused by a bad data cable, bad power cable, a borderline power supply, a faulty controller, or a faulty drive.
December 16, 201015 yr I will echo everything that has been said above. These are disk errors and not unRAID kernel errors. Check all your connections to your drives and make sure they are all good, get smart reports on all your drives to see what they look like.
December 16, 201015 yr Author Thx for the help. I re-seated all the cables and did smart reports on all the drives. They all passed. I will replace the data cable. The power supply has been running 16 HDD's for several years without issue. My concern would be a controller problem - in this case ata9 (Drive12) is off a Promise SATA300 TX4 - 4 port card. How does one determine what is it? Is there anything I can do? Funny this only started to happen after upgrading to the 4.5 series. Murphy's Law I guess. I'd still like to down-grade to 4.3.3 see if this is rectified. Can I just copy the 4.3.3 bzroot, bzimage files to the USB and leave the rest of the 4.6 installation as it is?? Thx
December 16, 201015 yr The power supply has been running 16 HDD's for several years without issue. You do realize power supplies degrade over time, right? It's caused by capacitor aging.
December 17, 201015 yr Author The power supply has been running 16 HDD's for several years without issue. You do realize power supplies degrade over time, right? It's caused by capacitor aging. Can't say I know a lot about this sort of thing, but I can accept that nothing lasts forever. If it is a power supply issue, then I would reason the drives on the same line would also be affected. I'm not exactly sure when I installed the power supply, but is was an upgrade from the original one in the case. I built the server mid May 2007 I think, so maybe it's not as old as I thought? I'm going to try a new cable first. Try to eliminate things one at a time. One last question...do those locking SATA data cables have the locking things on both ends? Thx again for the help.
December 17, 201015 yr One last question...do those locking SATA data cables have the locking things on both ends? So do, some don't but most do have locking on both ends
December 18, 201015 yr Author One last question...do those locking SATA data cables have the locking things on both ends? So do, some don't but most do have locking on both ends Thx, I'll see if I can locate some with them on both ends.
Archived
This topic is now archived and is closed to further replies.