April 15, 201214 yr I woke this morning to a system in a coma. Running 5.0b14 Background: Yesterday, I lost power and on restart the system said parry was not valid, so I restarted a parity check. That seem to complete at midnight. the mover started at 3:40, along the way: Message from syslogd@HAL_9000 at Mon Apr 16 03:52:24 2012 ... HAL_9000 kernel: Disabling IRQ #17 The mover kept on working till al hell broke loose: Apr 16 03:52:59 HAL_9000 kernel: ata1: lost interrupt (Status 0x50) Apr 16 03:52:59 HAL_9000 kernel: ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen Apr 16 03:52:59 HAL_9000 kernel: ata1.00: failed command: READ DMA Apr 16 03:52:59 HAL_9000 kernel: ata1.00: cmd c8/00:08:a8:4a:00/00:00:00:00:00/e0 tag 0 dma 4096 in Apr 16 03:52:59 HAL_9000 kernel: res 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout) Apr 16 03:52:59 HAL_9000 kernel: ata1.00: status: { DRDY } Apr 16 03:52:59 HAL_9000 kernel: ata1: hard resetting link Apr 16 03:52:59 HAL_9000 kernel: ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Apr 16 03:53:04 HAL_9000 kernel: ata1.00: qc timeout (cmd 0x27) Apr 16 03:53:04 HAL_9000 kernel: ata1.00: failed to read native max address (err_mask=0x4) Apr 16 03:53:04 HAL_9000 kernel: ata1.00: HPA support seems broken, skipping HPA handling Apr 16 03:53:04 HAL_9000 kernel: ata1.00: revalidation failed (errno=-5) Apr 16 03:53:04 HAL_9000 kernel: ata1: hard resetting link Apr 16 03:53:05 HAL_9000 kernel: ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Apr 16 03:53:05 HAL_9000 kernel: ata1.00: configured for UDMA/100 Apr 16 03:53:05 HAL_9000 kernel: ata1.00: device reported invalid CHS sector 0 Apr 16 03:53:05 HAL_9000 kernel: ata1: EH complete Full syslog attached I have not done anything to the system. I do have command line access and restarted unmenu (which freezes up) to see a message: STARTED, 6 disks in array. PARITY NOT VALID: DISK_DSBL I would like recommendations on how to gracefully bring the system down at the command line (I can spell Linux, but that is about it). And of course bring back up to diagnose the issues. Thanks in advance System: ASUS P8Z68-V LX mother board with 6 onboard SATA; Intel i5-2300, 2.8Ghz; 2-2G RAM Additional SATA card (4 port) - no name chinese card Antec 750W PSU 5- 2TB data, 1- 2TB Parity, 1- 500G Cache - all Seagate drives Antec 300 case Sony 4GB flash drive Installed: unMenu Additional Packages: "C" Compiler Mail & ssmtp UnRaid-Web syslog_20120416.txt.zip
April 15, 201214 yr Post a SMART report for the parity disk. To rebuild parity start the array with the parity disk unassigned. Then start the array with the parity disk assign and it will rebuild.
April 16, 201214 yr Author === START OF INFORMATION SECTION === Device Model: ST2000DL001-9VT156 Serial Number: 5YD0WX92 Firmware Version: CC96 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 4 Local Time is: Mon Apr 16 08:47:35 2012 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 612) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x103b) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 128 100 006 Pre-fail Always - 1766439400 3 Spin_Up_Time 0x0003 093 092 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 294 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 071 069 030 Pre-fail Always - 14267327 9 Power_On_Hours 0x0032 098 098 000 Old_age Always - 1790 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 71 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 9 189 High_Fly_Writes 0x003a 098 098 000 Old_age Always - 2 190 Airflow_Temperature_Cel 0x0022 072 034 045 Old_age Always In_the_past 28 (1 27 28 28) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 19 193 Load_Cycle_Count 0x0032 096 096 000 Old_age Always - 8433 194 Temperature_Celsius 0x0022 028 066 000 Old_age Always - 28 (0 22 0 0) 195 Hardware_ECC_Recovered 0x001a 128 100 000 Old_age Always - 1766439400 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 19 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 175728586917572 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 1759028975 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 1928648207 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t]
April 16, 201214 yr Author Back in business - all green balls. Message at the top of Unmenu - interesting STARTED, 6 disks in array. Parity is Valid:. Last parity check 15446 days ago with no sync errors. Does the community have any thoughts on the cause? And how I can avoid the issues again? Thanks in advance.
April 16, 201214 yr I believe the last parity check being 15446 days ago is because you rebuilt the array. Even though it's the same disk, the array is considered new, and parity has never been checked. The same thing happens if you add another disk to the array, if I recall correctly.
April 16, 201214 yr Must be that no name Chinese card. They play tricks on you now and then. Can't trust them. ;D
April 21, 201214 yr Author After all my webGUO issues (http://lime-technology.com/forum/index.php?topic=19560.0), I reformatted my flash, started over with 5.0b14. All was running well, rebuilding parity, all complete and happy ...then. Apr 21 12:11:58 HAL_9000 kernel: md: sync done. time=56692sec Apr 21 12:11:58 HAL_9000 kernel: md: recovery thread sync completion status: 0 Apr 21 12:34:55 HAL_9000 kernel: irq 17: nobody cared (try booting with the "irqpoll" option) Apr 21 12:34:55 HAL_9000 kernel: Pid: 0, comm: swapper Not tainted 3.1.1-unRAID #1 Apr 21 12:34:55 HAL_9000 kernel: Call Trace: Apr 21 12:34:55 HAL_9000 kernel: [<c104fa8c>] __report_bad_irq+0x1f/0x95 Apr 21 12:34:55 HAL_9000 kernel: [<c104fc39>] note_interrupt+0x137/0x1a8 Apr 21 12:34:55 HAL_9000 kernel: [<f8492832>] ? sil_interrupt+0x1d/0x67 [sata_sil] Apr 21 12:34:55 HAL_9000 kernel: [<c104e776>] handle_irq_event_percpu+0xef/0x100 Apr 21 12:34:55 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb Apr 21 12:34:55 HAL_9000 kernel: [<c104e7ab>] handle_irq_event+0x24/0x3b Apr 21 12:34:55 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb Apr 21 12:34:55 HAL_9000 kernel: [<c10501bb>] handle_fasteoi_irq+0x69/0x82 Apr 21 12:34:55 HAL_9000 kernel: <IRQ> [<c1003566>] ? do_IRQ+0x37/0x90 Apr 21 12:34:55 HAL_9000 kernel: [<c130c669>] ? common_interrupt+0x29/0x30 Apr 21 12:34:55 HAL_9000 kernel: [<c11ddd89>] ? acpi_idle_enter_bm+0x22a/0x25e Apr 21 12:34:55 HAL_9000 kernel: [<c12734f0>] ? cpuidle_idle_call+0x75/0xbd Apr 21 12:34:55 HAL_9000 kernel: [<c1001a5f>] ? cpu_idle+0x39/0x5a Apr 21 12:34:55 HAL_9000 kernel: [<c12fbd40>] ? rest_init+0x58/0x5a Apr 21 12:34:55 HAL_9000 kernel: [<c145172d>] ? start_kernel+0x28c/0x291 Apr 21 12:34:55 HAL_9000 kernel: [<c14510b0>] ? i386_start_kernel+0xb0/0xb7 Apr 21 12:34:55 HAL_9000 kernel: handlers: Apr 21 12:34:55 HAL_9000 kernel: [<f8492815>] sil_interrupt Apr 21 12:34:55 HAL_9000 kernel: Disabling IRQ #17 Apr 21 12:35:10 HAL_9000 kernel: ata2: lost interrupt (Status 0x50) The system can be access via telnet or the console, not web access (unRaid or unMenu - even with a re-start of both), AFP is off line, can still get to the flash drive via SMB. Full syslog attached. Help appreciated. SH syslog.txt.zip
April 21, 201214 yr Author I am looking into everything to try to find why things keep going upside-down. I have an ASUS P8Z68-V LX mother board. I was going through the BIOS. There are 3 options for SATA configuration [AHCI Mode default] : 1. IDE Mode: Set to [iDE Mode] when you wan tot use the Serial ATA hard disk drives as a parallel ATA physical storage device. 2. AHCI Mode (Default) : Set to [AHCI Mode] when you wan to the SAT hard disk drives to use the AHCI (Advance Host Controller Interface). The AHCI allows the onboard storage driver to enable advances Serial ATA features that increases storage performance on random warlords by allowing the drive to internally optimizes the order of commands 3. RAID Mode: Set to the [RAID Mode] win you want to create a RAID configuration from the SATA hard disk drives. In the [iDE Mode], you can set: Serial ATA Controller 0 [Enhanced] Disabled Mode: Disables SATA function Enhanced: Supports form than four SATA devices Compatible: When using Windows 98/NT/2000/MS Dos Serial ATA Controller 1 [Enhanced] Disabled Mode: Disables SATA function Enhanced: Supports form than four SATA devices I also have a PCI 4 port SATA card based on SiI 3114 chipset. A question for the community on the SATA BIOS settings: At present, they are set to AHCI Mode, which was default in the BIOS. Should they be set to something else - i.e. IDE mode?? Advice appreciated SH
April 21, 201214 yr Author After another day and a anther successful parity rebuild, same results. Any idea why the system disables IRQ17? Thanks, SH Apr 22 04:20:34 HAL_9000 kernel: irq 17: nobody cared (try booting with the "irqpoll" option) Apr 22 04:20:34 HAL_9000 kernel: Pid: 0, comm: swapper Not tainted 3.1.1-unRAID #1 Apr 22 04:20:34 HAL_9000 kernel: Call Trace: Apr 22 04:20:34 HAL_9000 kernel: [<c104fa8c>] __report_bad_irq+0x1f/0x95 Apr 22 04:20:34 HAL_9000 kernel: [<c104fc39>] note_interrupt+0x137/0x1a8 Apr 22 04:20:34 HAL_9000 kernel: [<f8492832>] ? sil_interrupt+0x1d/0x67 [sata_sil] Apr 22 04:20:34 HAL_9000 kernel: [<c104e776>] handle_irq_event_percpu+0xef/0x100 Apr 22 04:20:34 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb Apr 22 04:20:34 HAL_9000 kernel: [<c104e7ab>] handle_irq_event+0x24/0x3b Apr 22 04:20:34 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb Apr 22 04:20:34 HAL_9000 kernel: [<c10501bb>] handle_fasteoi_irq+0x69/0x82 Apr 22 04:20:34 HAL_9000 kernel: <IRQ> [<c1003566>] ? do_IRQ+0x37/0x90 Apr 22 04:20:34 HAL_9000 kernel: [<c130c669>] ? common_interrupt+0x29/0x30 Apr 22 04:20:34 HAL_9000 kernel: [<c11ddd89>] ? acpi_idle_enter_bm+0x22a/0x25e Apr 22 04:20:34 HAL_9000 kernel: [<c12734f0>] ? cpuidle_idle_call+0x75/0xbd Apr 22 04:20:34 HAL_9000 kernel: [<c1001a5f>] ? cpu_idle+0x39/0x5a Apr 22 04:20:34 HAL_9000 kernel: [<c12fbd40>] ? rest_init+0x58/0x5a Apr 22 04:20:34 HAL_9000 kernel: [<c145172d>] ? start_kernel+0x28c/0x291 Apr 22 04:20:34 HAL_9000 kernel: [<c14510b0>] ? i386_start_kernel+0xb0/0xb7 Apr 22 04:20:34 HAL_9000 kernel: handlers: Apr 22 04:20:34 HAL_9000 kernel: [<f8492815>] sil_interrupt Apr 22 04:20:34 HAL_9000 kernel: Disabling IRQ #17 Message from syslogd@HAL_9000 at Sun Apr 22 04:20:34 2012 ... HAL_9000 kernel: Disabling IRQ #17 Apr 22 04:21:04 HAL_9000 kernel: ata2: lost interrupt (Status 0x58) Apr 22 04:21:04 HAL_9000 kernel: ata2: drained 512 bytes to clear DRQ Apr 22 04:21:04 HAL_9000 kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen Apr 22 04:21:04 HAL_9000 kernel: ata2.00: failed command: IDENTIFY DEVICE Apr 22 04:21:04 HAL_9000 kernel: ata2.00: cmd ec/00:00:00:00:00/00:00:00:00:00/00 tag 0 pio 512 in Apr 22 04:21:04 HAL_9000 kernel: res 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout) Apr 22 04:21:04 HAL_9000 kernel: ata2.00: status: { DRDY } Apr 22 04:21:04 HAL_9000 kernel: ata2: hard resetting link Apr 22 04:21:05 HAL_9000 kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Apr 22 04:21:10 HAL_9000 kernel: ata2.00: qc timeout (cmd 0x27) Apr 22 04:21:10 HAL_9000 kernel: ata2.00: failed to read native max address (err_mask=0x4) Apr 22 04:21:10 HAL_9000 kernel: ata2.00: HPA support seems broken, skipping HPA handling Apr 22 04:21:10 HAL_9000 kernel: ata2.00: revalidation failed (errno=-5) syslog_2012422.txt.zip
April 22, 201214 yr After another day and a anther successful parity rebuild, same results. Any idea why the system disables IRQ17? Thanks, SH Apr 22 04:20:34 HAL_9000 kernel: irq 17: nobody cared (try booting with the "irqpoll" option) Apr 22 04:20:34 HAL_9000 kernel: [<f8492815>] sil_interrupt Apr 22 04:20:34 HAL_9000 kernel: Disabling IRQ #17 IRQ 17 appears to be associated with your Silicon Image disk controller card, possibly based on the SII3114 chipset? In my experience, a 'nobody cared' IRQ error is usually associated with the motherboard or a disk controller addon card. And once the kernel gives up on an IRQ and disables it, then everything depending on that IRQ goes down. Possible solutions (in priority order, hopefully the first one works and the rest aren't needed): - download and update the firmware for that card (from the Silicon Image web site) - replace that card with a comparable or better card (a better performing PCIe card will work better any way, since you have attached the parity drive to it) - install a newer BIOS on the motherboard - replace the motherboard (really hope one of the solutions above work, and make this unnecessary)
April 22, 201214 yr Author dgaschk, Rob L - thanks. That is the direction I have been going. I had already re-flashed the BIOS on the SiI 3114 card to 5.4.0.3. But this was not too successful. I saw that there was 5.5.0.0 version BIOS, but not sure on the compatibility - any thoughts?. Currently, I have pulled my Cache drive from the array and connected the other 6 drives to the SATA ports on the MB to run and stabilize for a bit. I am looking to add-on another card - Any thoughts on LSI 3081E compatibility? (There are limited quality add-on cards here in Shanghai - lots of gaming gear, but no server gear) Thanks, SH
May 2, 201214 yr Author Moved to a SuperMicro AOC SASLP MV8 card. 4 days and so far so good. Have marked SOLVED. Will keep monitoring for further issues. Lesson learned: Don't buy a cheap Chinese SiI card - most likely a fake. SH
May 3, 201214 yr Author After 3 days running with no reboots (had added a new drive the other day), my issue returned. Running 5.0b14. At the time, was running a TM back-up from my desktop Mac to a dedicated drive/user share. Also accessing music files in user shares for iTunes from the Media Center when everything locked up - both unMenu and unRaid GUI. Powered down via hard power down, but with script installed, the array stopped cleanly. The system never fully powered down - had to do a physical reset. When I restarted the system, the array required a manual restart through the GUI. It's off on a parity check now. SMB is accessible, but AFP is slow to give access. The array shows in a Mac Finder window, but none of the shares are accessible. Within a few minutes they become accessible. [is this a usual behavior that AFP is not accessible during the start of parity check?] System: ASUS P8Z68-V LX mother board with 6 onboard SATA; running latest 3703 BIOS Intel i5-2300, 2.8Ghz; 2-2G RAM Supermicro AOC SASLP MV8 Antec 750W PSU 6- 2TB data, 1- 2TB Parity, 1- 500G Cache - all Seagate drives Antec 300 case Sony 4GB flash drive Drive connections: Motherboard SATA: 5 data + 1 parity (4 data in user shares, 1 data dedicated to TM back up) SuperMicro SATA: 1 data, 1 cache (1 data slated for future TM back up - not used yet) Have checked all the cables, seem tight and well connected. SuperMicro card is well seated. Installed: unMenu Additional Packages: "C" Compiler Mail & ssmtp Monthly Parity Apple . file remover Hourly mail update Network and disk performance Powerdown script Thoughts and guidance appreciated. SH Here is an excerpt - full syslog attached + a syslog after the hard power down was initiated. May 3 19:43:02 HAL_9000 vsftpd[18661]: connect from 192.168.11.1 (192.168.11.1) (Routine) HERE IS WHERE EVERYTHING WENT SIDEWAYS: May 3 19:44:20 HAL_9000 kernel: irq 16: nobody cared (try booting with the "irqpoll" option) (Errors) May 3 19:44:20 HAL_9000 kernel: Pid: 0, comm: swapper Not tainted 3.1.1-unRAID #1 (Errors) May 3 19:44:20 HAL_9000 kernel: Call Trace: (Errors) May 3 19:44:20 HAL_9000 kernel: [<c104fa8c>] __report_bad_irq+0x1f/0x95 (Errors) May 3 19:44:20 HAL_9000 kernel: [<c104fc39>] note_interrupt+0x137/0x1a8 (Errors) May 3 19:44:20 HAL_9000 kernel: [<c104e776>] handle_irq_event_percpu+0xef/0x100 (Errors) May 3 19:44:20 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb (Errors) May 3 19:44:20 HAL_9000 kernel: [<c104e7ab>] handle_irq_event+0x24/0x3b (Errors) May 3 19:44:20 HAL_9000 kernel: [<c1050152>] ? handle_edge_irq+0xcb/0xcb (Errors) May 3 19:44:20 HAL_9000 kernel: [<c10501bb>] handle_fasteoi_irq+0x69/0x82 (Errors) May 3 19:44:20 HAL_9000 kernel: <IRQ> [<c1003566>] ? do_IRQ+0x37/0x90 May 3 19:44:20 HAL_9000 kernel: [<c130c669>] ? common_interrupt+0x29/0x30 (Errors) May 3 19:44:20 HAL_9000 kernel: [<c11ddd89>] ? acpi_idle_enter_bm+0x22a/0x25e (Errors) May 3 19:44:20 HAL_9000 kernel: [<c12734f0>] ? cpuidle_idle_call+0x75/0xbd (Errors) May 3 19:44:20 HAL_9000 kernel: [<c1001a5f>] ? cpu_idle+0x39/0x5a (Errors) May 3 19:44:20 HAL_9000 kernel: [<c12fbd40>] ? rest_init+0x58/0x5a (Errors) May 3 19:44:20 HAL_9000 kernel: [<c145172d>] ? start_kernel+0x28c/0x291 (Errors) May 3 19:44:20 HAL_9000 kernel: [<c14510b0>] ? i386_start_kernel+0xb0/0xb7 (Errors) May 3 19:44:20 HAL_9000 kernel: handlers: May 3 19:44:20 HAL_9000 kernel: [<f84750c9>] mvs_interrupt May 3 19:44:20 HAL_9000 kernel: Disabling IRQ #16 May 3 19:44:35 HAL_9000 kernel: sas: command 0xdfeb7e40, task 0xf2acf2c0, timed out: BLK_EH_NOT_HANDLED (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: command 0xdff5fb40, task 0xdf9b6140, timed out: BLK_EH_NOT_HANDLED (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: Enter sas_scsi_recover_host (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: trying to find task 0xf2acf2c0 (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: sas_scsi_find_task: aborting task 0xf2acf2c0 (Drive related) May 3 19:44:35 HAL_9000 kernel: drivers/scsi/mvsas/mv_sas.c 1678:mvs_abort_task() mvi=dfa40000 task=f2acf2c0 slot=dfa516a0 slot_idx=x0 (System) May 3 19:44:35 HAL_9000 kernel: sas: sas_scsi_find_task: task 0xf2acf2c0 is aborted (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: sas_eh_handle_sas_errors: task 0xf2acf2c0 is aborted (Errors) May 3 19:44:35 HAL_9000 kernel: sas: trying to find task 0xdf9b6140 (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: sas_scsi_find_task: aborting task 0xdf9b6140 (Drive related) May 3 19:44:35 HAL_9000 kernel: drivers/scsi/mvsas/mv_sas.c 1678:mvs_abort_task() mvi=dfa40000 task=df9b6140 slot=dfa516d4 slot_idx=x1 (System) May 3 19:44:35 HAL_9000 kernel: sas: sas_scsi_find_task: task 0xdf9b6140 is aborted (Drive related) May 3 19:44:35 HAL_9000 kernel: sas: sas_eh_handle_sas_errors: task 0xdf9b6140 is aborted (Errors) May 3 19:44:35 HAL_9000 kernel: ata5: sas eh calling libata port error handler (Errors) May 3 19:44:35 HAL_9000 kernel: ata6: sas eh calling libata port error handler (Errors) May 3 19:44:35 HAL_9000 kernel: sas: --- Exit sas_scsi_recover_host (Drive related) May 3 19:44:53 HAL_9000 kernel: sas: command 0xdca7af00, task 0xf2b6fcc0, timed out: BLK_EH_NOT_HANDLED (Drive related) May 3 19:44:53 HAL_9000 kernel: sas: command 0xea103300, task 0xdf9b6140, timed out: BLK_EH_NOT_HANDLED (Drive related) May 3 19:44:53 HAL_9000 kernel: sas: Enter sas_scsi_recover_host (Drive related) May 3 19:44:53 HAL_9000 kernel: sas: trying to find task 0xf2b6fcc0 (Drive related) May 3 19:44:53 HAL_9000 kernel: sas: sas_scsi_find_task: aborting task 0xf2b6fcc0 (Drive related) May 3 19:44:53 HAL_9000 kernel: drivers/scsi/mvsas/mv_sas.c 1678:mvs_abort_task() mvi=dfa40000 task=f2b6fcc0 slot=dfa516d4 slot_idx=x1 (System) Syslog_20120503_@_Powerdown.txt.zip Syslog_20120503.txt.zip
May 3, 201214 yr The only other thing I can think of that you might try is: http://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re18.html My ESXi hardware does not particularly like the kernel in the later versions of unRAID. Not a huge deal to me as I do not upgrade that often on my production machine. There is a BIOS update for the board in my ESXi box that might fix this but frankly it is to much trouble to mess with right now.
May 5, 201214 yr Author Prostuff1, Looks interesting, but configuring a kernel is way outside my comfort zone. I know basic command line stuff in Linux, but that's about it. I have no idea where to start. It seems this error comes up after the system is running a few days. Not sure why or where this IRQ is. But the error only happens when the additional disk controller is installed. I have looked at the BIOS - is the latest. Any step by step guidance on how to configure a kernel appreciated. Or should I try 5rc2 with the 3.0.30 kernel (versus 5b14 with the 3.1.1 kernel)? Thanks SH
May 5, 201214 yr Prostuff1, Looks interesting, but configuring a kernel is way outside my comfort zone. I know basic command line stuff in Linux, but that's about it. I have no idea where to start. It seems this error comes up after the system is running a few days. Not sure why or where this IRQ is. But the error only happens when the additional disk controller is installed. I have looked at the BIOS - is the latest. Any step by step guidance on how to configure a kernel appreciated. Or should I try 5rc2 with the 3.0.30 kernel (versus 5b14 with the 3.1.1 kernel)? Thanks SH could try 5.0rc and see what happens. that line can be added to the syslinux.cfg file default menu.c32 menu title Lime Technology LLC prompt 0 timeout 50 label unRAID 5b14 kernel bzimage.5b14 append initrd=bzroot.5b14 irqpoll rootdelay=10 Where the irqpoll is above just replace that with the "noirqdebug". The default syslinux.cfg file does not have the irqpoll in it so just add whatever you need to in it's place.
May 5, 201214 yr Author Prostuff1, Thank you! I will give this a go and wait for 5rc2 to roll out (not yet posted). 5rc2test seems to still be a bit buggy. SH
May 5, 201214 yr After 3 days running with no reboots (had added a new drive the other day), my issue returned. ... May 1 18:44:50 HAL_9000 kernel: mvsas 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 [mvsas module is assigned IRQ 16] ... May 3 19:44:20 HAL_9000 kernel: irq 16: nobody cared (try booting with the "irqpoll" option) [iRQ 16 tripped, but no one responds (where are you, mvsas?)] ... May 3 19:44:20 HAL_9000 kernel: <IRQ> handlers: [<f84750c9>] mvs_interrupt [(line reassembled from parts) mvs_interrupt is probably mvsas interrupt handling function, which *should* have dealt with the interrupt] ... May 3 19:44:20 HAL_9000 kernel: Disabling IRQ #16 [disaster! even if mvsas decides to go back to work, it won't be called now, so all attached drives are lost until reboot] May 3 19:44:35 HAL_9000 kernel: sas: command 0xdfeb7e40, task 0xf2acf2c0, timed out: BLK_EH_NOT_HANDLED [all subsequent errors can be ignored, because mvsas and the controller are out of action] ... Obviously not good news! What is striking is the similarities between this failure and the previous one with the Silicon Image controller card. In both cases, an interrupt was assigned to an addon disk controller card, and then later during normal operation failed with an "irq ??: nobody cared" error message, and so was disabled. If you had not had the previous issues with the Sil card, I would suggest this may be a faulty SAS card or it needs a firmware update, but this seems too coincidental with the last failure. I would have suggested updating the motherboard BIOS, but I noticed that you have just done that. You can try looking for a firmware update for the SAS card, and hope that fixes it. Otherwise, I don't have any good suggestions. I suppose that it is remotely possible that both cards have the same issue, and it is also remotely possible that there is still an issue with that motherboard.
Archived
This topic is now archived and is closed to further replies.