I recently got my system back to stable after some bad sata cables caused several drives to fail simultaneously (thankfully no critical data lost!). After replacing the cables and returning things to stable I decided to upgrade my 2x 8TB parity drives to 2x 14TB drives after the recent BBY sale. After installing the drives and starting parity rebuild and preclear on the old 8's I came back the next day to find that parity was only building on one of the drives and the other was disabled. I decided to let parity continue building on the one and let the preclears finish. 

 

Once that was done I stopped the array and tried to re-add the "disabled" parity drive but it didn't show up... Strange. OK, well lets add the 8TBs to the array for now. That worked fine.

 

Decided to restart the system to try adding the second parity drive. Same basic behavior. After a while it failed and disabled it self. I should also mention that during this time I've started to notice long boot times, however not every boot is long. Also sometimes the array takes forever to mount drives, but again, not always. 

 

I started to suspect more hardware issues so I monitored the boot sequence. But other than not always catching all 3 LSI screens (I have 3 LSI cards) I'm not seeing anything. Even when I don't see all three LSI screens during boot every drive is still showing up (except the 14's occasionally as stated). 

 

Now these are shucked drives, however I'm in a Storinator chassis so every drive bay is routed through molex power so that shouldn't be an issue. 

 

I did update my BIOS. I want to check the LSI BIOS but I'm not sure how. I'm attaching my diagnostics and looking through them myself but I'm hoping someone here can help me out. 

ETA: Oh, I've also tried moving them to other bays with no luck.

tower-diagnostics-20220129-1754.zip

Edited January 30, 20224 yr by argash

Looks like a controller problem

Jan 28 18:27:46 Tower kernel: mpt2sas_cm1: diag reset: FAILED

Nothing assigned to either parity or disks 9,10,11

8 hours ago, trurl said:

Nothing assigned to either parity or disks 9,10,11

Correct, at the moment nothing is assigned to parity because of the issues that I'm having. 9, 10, and 11 were the drives that failed before swapping out my sata cables and I still need to replace. 

How do I best check the LSI controllers though?

4 hours ago, argash said:

How do I best check the LSI controllers though?

You might try reseating. Do they have adequate cooling?

Ok so I've made some progress. 

 

 

First I noticed that the firmware was off so I flashed that bad boy to get to here:

 

 

Now I'm concerned that it doesn't have anything in it's boot order but it wouldn't let me change it. Now in safe mode when I run sas2flash -listall it only shows two of the cards though.

 

 

Again I think it's the boot order. With out anything set for it I don't think it's actually booting that card. But again it wouldn't let me set it an order.

Got it set in the boot order. BIOS now sees it every time. Still not showing up in unraid and I have no idea why. I even edited my /boot/syslinux/syslinux.cfg as mentioned in another thread:

 

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  kernel /bzimage
  append pci=realloc=off initrd=/bzroot
label Unraid OS GUI Mode
  menu default
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

I still haven't made any progress on this. I've ordered another card to try. However if anyone has any other suggestions I'm all ears. I think it's a software issue in unRAID but I can't confirm until the new card arrives.