Vanilla RC3 + unmenu. No packages.

 

emhttp unresponsive but I can still telnet.

 

Tasks: 130 total,   1 running, 128 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.0%us,  0.3%sy,  0.0%ni, 97.1%id,  2.5%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   4137476k total,  4113220k used,    24256k free,   344152k buffers
Swap:        0k total,        0k used,        0k free,  3533444k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      20   0   828  284  240 S    0  0.0   0:11.53 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S    0  0.0   0:00.98 ksoftirqd/0
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0
    7 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/1
    9 root      20   0     0    0    0 S    0  0.0   0:00.56 ksoftirqd/1
   11 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/2
   13 root      20   0     0    0    0 S    0  0.0   0:00.65 ksoftirqd/2
   14 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/3
   16 root      20   0     0    0    0 S    0  0.0   0:00.71 ksoftirqd/3
   17 root       0 -20     0    0    0 S    0  0.0   0:00.00 khelper
  153 root      20   0     0    0    0 S    0  0.0   2:12.72 sync_supers
  155 root      20   0     0    0    0 S    0  0.0   0:00.04 bdi-default
  157 root       0 -20     0    0    0 S    0  0.0   0:00.00 kblockd
  272 root       0 -20     0    0    0 S    0  0.0   0:00.00 ata_sff
  282 root      20   0     0    0    0 S    0  0.0   0:00.00 khubd
  392 root       0 -20     0    0    0 S    0  0.0   0:00.00 rpciod

 

ps ax | grep 8066...
8066 ?        Z      0:00 [hdparm] <defunct>

 

I can't access any sleeping drives. Captured syslog via cat to console with putty logging enabled.

 

I see disk problems I don't understand. Then good old "timed out: BLK_EH_NOT_HANDLED" after the last movie we watched.

 

The server is in a room by itself. No movement or changes for the last few weeks. I should probably run SMART tests but will have to reboot first. Anyone want to see anything from the current state?

syslog1.txt

Old BIOS, i3 CPU hence no ECC and no UPS - upon boot-up Unraid runs from the memory and any single memory bit flip in that critical area due to whatever reasons (unclean power, power surge due to the stormy weather, etc...) can crash one module and then you will end with similar results.

 

And check the syslog around May 14 starting around 23:59:57

Just IMHO.

I've seen the same errors on this system, with rc2, but I had old plugins running so couldn't trust the test. Both previous times hdparm died not long after a BLK_EH_NOT_HANDLED.

 

Given the messy history of BLK_EH_NOT_HANDLED errors in mv_sas.c I'm not inclined to think the failures are random. More likely something is borderline in my system and/or the code. It feels a bit like hdparm dies trying to send command if  mv_sas is busy re-establishing link with a drive. OTOH, I haven't had much time to search other reports. If I'm the only one still seeing this bugger, great!

 

You're right, time to update that BIOS.

 

Supermicro X9SCL-F-O + i3-2100

4GB

Supermicro AOC-SASLP-MV8

Seasonic X650 GOLD

2x Supermicro CSE-M35T1 5-in-3 bays + 2x trayless hotswap bays (monoprice)

APC ES650 UPS

Parity + 5 data drives on motherboard

4 data drives + cache on MV8

I know unMenu is considered relatively "rc3 stable" but have you tried a fresh install of RC3 with no addons? It seems like everyone wants to test these RCs with addons running and it just ends up making more work for Tom.

I will be posting a special release called 5.0-rc4-scst-1, which uses a different mvsas driver.  Watch for that announcement and then let's see how it behaves in your system.

Was about to ask about that. Thanks. Running standard rc4 now.

 

I extended the inactivity timer before drives sleep. The problem always seems to show itself when hdparm tries to put a drive to sleep after that drive had a BLK_EH error. Might just die the next time anything tries to access that drive after the error, be it hdparm or whatever, but hdparm (or what it calls) certainly does not handle it well.

anymore on this test release.  I am starting to get these at least once a day now after going to RC4

 

just happened again 5 min ago, have to force reset the array again to get the GUI - which causes a dirty shutdown, the shutdown or powerdown hangs.....

 

otherwise I can leave it, shares are still working, and I can still telnet in....

 

Jun 11 05:39:08 Tower kernel: sas: command 0xf4b3a6c0, task 0xf7635b80, timed out: BLK_EH_NOT_HANDLED (Drive related)
Jun 11 05:39:08 Tower kernel: sas: Enter sas_scsi_recover_host (Drive related)
Jun 11 05:39:08 Tower kernel: sas: trying to find task 0xf7635b80 (Drive related)
Jun 11 05:39:08 Tower kernel: sas: sas_scsi_find_task: aborting task 0xf7635b80 (Drive related)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1818:<7>mv_abort_task() mvi=f70c0000 task=f7635b80 slot=f70d1640 slot_idx=x2 (System)
Jun 11 05:39:08 Tower kernel: sas: sas_scsi_find_task: querying task 0xf7635b80 (Drive related)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1747:mvs_query_task:rc= 5 (System)
Jun 11 05:39:08 Tower kernel: sas: sas_scsi_find_task: task 0xf7635b80 failed to abort (Minor Issues)
Jun 11 05:39:08 Tower kernel: sas: task 0xf7635b80 is not at LU: I_T recover (Drive related)
Jun 11 05:39:08 Tower kernel: sas: I_T nexus reset for dev 0300000000000000 (Drive related)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x89800. (System)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x1001001 (System)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2226:phy3 Unplug Notice (System)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x199800. (System)
Jun 11 05:39:08 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x1001081 (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x199800. (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x10000 (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2253:notify plug in on phy[3] (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1338:port 3 attach dev info is 0 (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1340:port 3 attach sas addr is 3 (System)
Jun 11 05:39:09 Tower kernel: drivers/scsi/mvsas/mv_sas.c 379:phy 3 byte dmaded. (System)
Jun 11 05:39:09 Tower kernel: sas: sas_form_port: phy3 belongs to port2 already(1)! (Drive related)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1701:mvs_I_T_nexus_reset for device[2]:rc= 0 (System)
Jun 11 05:39:11 Tower kernel: sas: I_T 0300000000000000 recovered (Drive related)
Jun 11 05:39:11 Tower kernel: sas: sas_ata_task_done: SAS error 8d (Errors)
Jun 11 05:39:11 Tower kernel: ata15: sas eh calling libata port error handler (Routine)
Jun 11 05:39:11 Tower kernel: ata16: sas eh calling libata port error handler (Routine)
Jun 11 05:39:11 Tower kernel: ata17: sas eh calling libata port error handler (Routine)
Jun 11 05:39:11 Tower kernel: ata17.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 t0 (Errors)
Jun 11 05:39:11 Tower kernel: ata17.00: failed command: CHECK POWER MODE (Minor Issues)
Jun 11 05:39:11 Tower kernel: ata17.00: cmd e5/00:00:00:00:00/00:00:00:00:00/00 tag 0 (Drive related)
Jun 11 05:39:11 Tower kernel:          res 01/04:00:00:00:00/00:00:00:00:00/00 Emask 0x3 (HSM violation) (Errors)
Jun 11 05:39:11 Tower kernel: ata17.00: status: { ERR } (Drive related)
Jun 11 05:39:11 Tower kernel: ata17.00: error: { ABRT } (Errors)
Jun 11 05:39:11 Tower kernel: ata17: hard resetting link (Minor Issues)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x89800. (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x1001001 (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2226:phy3 Unplug Notice (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x199800. (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x1001081 (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2198:port 3 ctrl sts=0x199800. (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2200:Port 3 irq sts = 0x10000 (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 2253:notify plug in on phy[3] (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1338:port 3 attach dev info is 0 (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1340:port 3 attach sas addr is 3 (System)
Jun 11 05:39:11 Tower kernel: drivers/scsi/mvsas/mv_sas.c 379:phy 3 byte dmaded. (System)
Jun 11 05:39:11 Tower kernel: sas: sas_form_port: phy3 belongs to port2 already(1)! (Drive related)
Jun 11 05:39:13 Tower kernel: drivers/scsi/mvsas/mv_sas.c 1701:mvs_I_T_nexus_reset for device[2]:rc= 0 (System)
Jun 11 05:39:13 Tower kernel: sas: sas_ata_hard_reset: Found ATA device. (Drive related)
Jun 11 05:39:13 Tower kernel: ata17.00: configured for UDMA/133 (Drive related)
Jun 11 05:39:13 Tower kernel: ata17: EH complete (Drive related)
Jun 11 05:39:13 Tower kernel: sas: --- Exit sas_scsi_recover_host (Drive related)
Jun 11 05:39:13 Tower kernel: mdcmd (113): spindown 5 (Routine)
Jun 11 05:39:23 Tower kernel: mdcmd (114): spindown 9 (Routine)
Jun 11 06:07:28 Tower kernel: sas: command 0xf4be7c00, task 0xf7645e00, timed out: BLK_EH_NOT_HANDLED (Drive related)
Jun 11 06:11:31 Tower in.telnetd[4775]: connect from 192.168.0.99 (192.168.0.99) (Routine)
Jun 11 06:11:32 Tower login[4776]: ROOT LOGIN  on '/dev/pts/0' from '192.168.0.99' (Logins)

 

Myk

 

I have the exact same problem here. I was forced to downgrade back to beta 11 which is confirmed stable on my system.

 

Running an i3 540, Supermicro MBD-X8SIL-F-O, and 2x Supermicro AOC-SASLP-MV8s

 

Looking forward to 5.0-rc4-scst-1.