November 6, 20178 yr I am running 6.4.0 RC7a. I am using 2 8TB parity drives. The MB/CPU and RAM were replaced about 2 months ago and have done 1 monthly parity check without a problem. The trouble started when the second scheduled parity check started. I checked the system the next day and found that the parity check was at about 270MB out of 8TB. It was not progressing but web interface was responsive. I attempted to stop it, the interface accepted the requet but it never stopped. I changed settings to disable array autostart and pressed reboot in the interface and the system shut down and rebooted. During controller POST (AOC-SASLPMV8 FW 3.1.0.21) both drives failed to be detected but once the kernel and the driver loaded it saw all disks. I did a new config and started a rebuild. The rebuild immediately failed (about the same spot as the scheduled one). I tried again after another reboot and it rebuilt successfully. Everything worked fine for about a day. I decided to install the "tips and tweaks" "fix common problems" plugins. I started and scan/check in fix common problems and the array died. It was in exactly the same state as described above (disabled parity1 and data9). I replaced parity1 with a new drive and upgraded to 6.4.0 RC10b and am currently rebuilding again. The new 8TB parity1 still has the issue of sometimes not being seen by the controller at POST but comes up fine after the kernel loads. Other things I have done: Full pass of memtest86+, no errors seatools tests on parity1, passed moved drives to other slots on the backplane Attached are several diagnostic packs. I suspect the controller(s) somewhat at this point but would like some feedback. I am not opposed to replacing them all if it seems they are the source of the trouble. This account reflects everything I remember but since this has happened over several days I may have left something out so questions are welcome. tower-diagnostics-20171103-2113.zip tower-diagnostics-20171102-0035.zip tower-diagnostics-20171103-2003.zip tower-diagnostics-20171103-2045.zip
November 6, 20178 yr It's a know problem with those controllers on some configurations, same with the newer SAS2LP, one thing than can help is disabling VT-d, but best bet would be to replace them with LSI controllers, e.g.: 9201-8i, 9211-8i and clones.
November 6, 20178 yr Author 4 hours ago, johnnie.black said: It's a know problem with those controllers on some configurations, same with the newer SAS2LP, one thing than can help is disabling VT-d, but best bet would be to replace them with LSI controllers, e.g.: 9201-8i, 9211-8i and clones. Thank you for your fast reply. A couple of questions. Which part is a known problem (detection, dropping or both)? I have been running these controllers with unraid (different MB/CPU/RAM) for at least 3 years. I've had an occasional problem like this (where a drive drops) but not as persistently. In addition to the 2 controllers you mentioned there appears to be an IBM (M1015) and a Dell H310. Since all of these appear to be selling at about the same price point is there any reason to pick one over the other (or another I may have missed)? I know they are based on the same chip but they seem to have layout differences. From what I have read in another post these should all be flashed to an LSI 9211-8i IT mode in order to allow non-RAID pass-through operation.
November 6, 20178 yr 54 minutes ago, WizADSL said: Which part is a known problem (detection, dropping or both)? Dropping disks, 57 minutes ago, WizADSL said: I have been running these controllers with unraid (different MB/CPU/RAM) for at least 3 years Same happened to other users, the issue seems to be worse with latest releases. 58 minutes ago, WizADSL said: Since all of these appear to be selling at about the same price point is there any reason to pick one over the other (or another I may have missed)? I know they are based on the same chip but they seem to have layout differences. From what I have read in another post these should all be flashed to an LSI 9211-8i IT mode in order to allow non-RAID pass-through operation. They are all the same, Dell and IBM need to be crossflashed, but that's usually not very difficult.
November 11, 20178 yr Author I received and installed 3 9211-8i controllers. Everything is working fine so far but based on how long the previous SASLP-MV8 would run without a problem, I'm not sure I'm out of the woods. My question is regarding the firmware on the controllers which came flashed in IT mode and are all identical. The firmware is 20.00.07.00 which I believe is the latest but the BIOS is 07.37.00.00 where 07.39.02.00 appears to be the latest. Is it worth the trouble to update the BIOS? Is it used after initial system boot? On a separate note especially to anyone moving from the Supermicro controllers which POST quickly, the LSI controllers take a LONG time to POST. They will display some text (copyright date, etc) and then just sit for quite a while without further messages and then they'll start to output information (time between "header" and additional information about 1 minute, full POST abut 5 minutes). I thought my cards were incompatible or defective at first.
November 11, 20178 yr Author Well, looks like I spoke too soon, just saw this at the end of my log (just happened based on time/date): Nov 11 14:56:54 Tower kernel: irq 16: nobody cared (try booting with the "irqpoll" option) Nov 11 14:56:54 Tower kernel: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.13.10-unRAID #1 Nov 11 14:56:54 Tower kernel: Hardware name: Gigabyte Technology Co., Ltd. Z270X-UD3/Z270X-UD3-CF, BIOS F7 07/07/2017 Nov 11 14:56:54 Tower kernel: Call Trace: Nov 11 14:56:54 Tower kernel: <IRQ> Nov 11 14:56:54 Tower kernel: dump_stack+0x61/0x7e Nov 11 14:56:54 Tower kernel: __report_bad_irq+0x37/0xb4 Nov 11 14:56:54 Tower kernel: note_interrupt+0x1da/0x22b Nov 11 14:56:54 Tower kernel: handle_irq_event_percpu+0x3d/0x46 Nov 11 14:56:54 Tower kernel: handle_irq_event+0x36/0x54 Nov 11 14:56:54 Tower kernel: handle_fasteoi_irq+0x91/0xf7 Nov 11 14:56:54 Tower kernel: handle_irq+0x1a/0x1f Nov 11 14:56:54 Tower kernel: do_IRQ+0x41/0xbd Nov 11 14:56:54 Tower kernel: common_interrupt+0x89/0x89 Nov 11 14:56:54 Tower kernel: RIP: 0010:cpuidle_enter_state+0xe7/0x13e Nov 11 14:56:54 Tower kernel: RSP: 0018:ffffffff81803e00 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdb Nov 11 14:56:54 Tower kernel: RAX: ffff88043ec19f40 RBX: 00008536b9078e14 RCX: 000000000000001f Nov 11 14:56:54 Tower kernel: RDX: 00008536b9078e14 RSI: 0000000000017740 RDI: 0000000000000000 Nov 11 14:56:54 Tower kernel: RBP: ffffffff81803e30 R08: 000231f9f91d9343 R09: 0000000000000018 Nov 11 14:56:54 Tower kernel: R10: ffffffff81803dd0 R11: 071c71c71c71c71c R12: ffff88043ec21f00 Nov 11 14:56:54 Tower kernel: R13: 0000000000000001 R14: 0000000000000000 R15: 00008536b9078142 Nov 11 14:56:54 Tower kernel: </IRQ> Nov 11 14:56:54 Tower kernel: cpuidle_enter+0x12/0x14 Nov 11 14:56:54 Tower kernel: call_cpuidle+0x33/0x35 Nov 11 14:56:54 Tower kernel: do_idle+0x11b/0x17a Nov 11 14:56:54 Tower kernel: cpu_startup_entry+0x1a/0x1c Nov 11 14:56:54 Tower kernel: rest_init+0xd0/0xd3 Nov 11 14:56:54 Tower kernel: start_kernel+0x3c8/0x3d5 Nov 11 14:56:54 Tower kernel: ? early_idt_handler_array+0x120/0x120 Nov 11 14:56:54 Tower kernel: x86_64_start_reservations+0x2a/0x2c Nov 11 14:56:54 Tower kernel: x86_64_start_kernel+0xdf/0xec Nov 11 14:56:54 Tower kernel: secondary_startup_64+0x9f/0x9f Nov 11 14:56:54 Tower kernel: handlers: Nov 11 14:56:54 Tower kernel: [<ffffffffa008ecfc>] i801_isr [i2c_i801] Nov 11 14:56:54 Tower kernel: Disabling IRQ #16 Any suggestions?
November 11, 20178 yr That means IQR16 was disabled, a bios update can help, but it may not be serious, depending on what's using that IRQ, you can check with: cat /proc/interrupts 1 hour ago, WizADSL said: The firmware is 20.00.07.00 which I believe is the latest but the BIOS is 07.37.00.00 where 07.39.02.00 appears to be the latest. Is it worth the trouble to update the BIOS? Is it used after initial system boot? Ignore the bios, you don't even need a bios for unRAID since none of those devices will be a boot device, and it would get rid of the 5 minute boot time.
November 11, 20178 yr Author The error mentioned in the previous message did not affect array operation (so far as I can tell). Although lspci -vv reports that the LSI conrollers (2 of 3) are using interrupt 16, the error is the log as well as /proc/interrupts show that only i2c_i801 was using interrupt 16. It is possible I have 2 issues overlapping. The mvsas cards/driver causing a crash which should be resolved now that the cards are no longer in the system and separately the IRQ 16 not being handled. In previous syslogs (when the SASLP-MV8 cards were in the system) the mvsas driver was also listed on IRQ 16 so that would cause serious problems. Edited November 11, 20178 yr by WizADSL clarification
November 11, 20178 yr Author Just noticed you replied as I wrote that last post. Here is the output you requested. CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 0: 42 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer 5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0 8: 61 0 0 0 0 0 0 0 IR-IO-APIC 8-edge rtc0 9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi 16: 100000 0 0 0 0 0 0 0 IR-IO-APIC 16-fasteoi i801_smbus 120: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0 121: 0 0 0 0 0 0 0 0 DMAR-MSI 1-edge dmar1 122: 0 0 0 0 0 0 0 0 DMAR-MSI 128-edge dmar0-prq 123: 622567 0 0 0 0 0 0 0 IR-PCI-MSI 327680-edge xhci_hcd 124: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194304-edge xhci_hcd 125: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194305-edge xhci_hcd 126: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194306-edge xhci_hcd 127: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194307-edge xhci_hcd 128: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194308-edge xhci_hcd 129: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194309-edge xhci_hcd 130: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194310-edge xhci_hcd 131: 0 0 0 0 0 0 0 0 IR-PCI-MSI 4194311-edge xhci_hcd 132: 3883335 0 0 0 0 0 0 0 IR-PCI-MSI 376832-edge ahci[0000:00:17.0] 133: 8325882 0 0 0 0 0 0 0 IR-PCI-MSI 520192-edge eth0 134: 58947227 0 0 0 0 0 0 0 IR-PCI-MSI 524288-edge mpt2sas0-msix0 135: 69485981 0 0 0 0 0 0 0 IR-PCI-MSI 1048576-edge mpt2sas1-msix0 136: 15637221 0 0 0 0 0 0 0 IR-PCI-MSI 3145728-edge mpt2sas2-msix0 NMI: 0 0 0 0 0 0 0 0 Non-maskable interrupts LOC: 51085420 37205118 37456291 37297385 35214115 35877214 35671574 35254584 Local timer interrupts SPU: 0 0 0 0 0 0 0 0 Spurious interrupts PMI: 0 0 0 0 0 0 0 0 Performance monitoring interrupts IWI: 1 0 0 0 0 0 0 0 IRQ work interrupts RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries RES: 4093833 5370199 1856392 519379 349274 270421 243366 234074 Rescheduling interrupts CAL: 257596 138660 134374 130669 131325 133250 132604 136736 Function call interrupts TLB: 233058 124849 120392 116868 126409 128373 127891 131998 TLB shootdowns TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts MCE: 0 0 0 0 0 0 0 0 Machine check exceptions MCP: 482 482 482 482 482 482 482 482 Machine check polls ERR: 0 MIS: 0 PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event NPI: 0 0 0 0 0 0 0 0 Nested posted-interrupt event PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
November 11, 20178 yr LSIs don't usually use IRQ16 like the Marvells, if all is well you can ignore, but still look for a bios update.
November 11, 20178 yr Author Not using IRQ 16 is certainly a +1 versus the SASLP-MV8. The BIOS I am currently on is the latest available for my MB but I will check periodically for an update. If I don't change anything in my system (hardware-wise) is it reasonably safe to assume IRQ assignments won't change or do they typical with each reboot?
November 11, 20178 yr 23 minutes ago, WizADSL said: is it reasonably safe to assume IRQ assignments won't change or do they typical with each reboot? Yes, especially the lower ones, without changes nothing else should start using IRQ16.
Archived
This topic is now archived and is closed to further replies.