I have a Supermicro X10DRH-iT and have filled out all available on-board storage ports.  I've successfully added a 4 port internal storage expansion card, can't remember the model but it works and has 3 drives attached.

 

I built a JBOD box, have all the cables correctly configured to 4 drives (2-8TB and 2-14TB).

 

I added a 9201-16e to the server box, flashed to P20, bios, everything looks good through sas2flsh.  The drives connected to the JBOD don't want to show up.

 

Thoughts?

 

Diagnostics are attached.  

 

 

tower-diagnostics-20240902-1957.zip

Edited September 6, 20241 yr by jongregory75

HBA is being correctly detected and initialized, suggesting a power/connection issue with the disks, do you hear the disks spinning up? Which cables are you using?

I'm using SFF-8088 to 29pin SFF-8482 with Molex from the PS.  I've checked each drive and they are spinning. 

 

I just pulled all of the drives and I never covered/removed the 3.3v reset pins.  It's been awhile but I thought that they wouldn't even spin without covering/removing pin 3 from shucked drives.  The 2 14TB are WD whites from shucked EZStores.  The 2 8TB are WD Reds.  I'll test 1 of the 14TB by removing pin 3 and see if it comes up.  This seemed to be the case in the past and I remember having to go through 5-6 drives to remove pin 3 entirely as I got tired of arts and crafts with painters tape trying to cover the pin.

Edited September 4, 20241 yr by jongregory75
update

If disk spinning then not 3.3v issue.

Update 2:  I removed the #3 pin on a WD 14TB white and a WD 8TB red.  Rebooted.  Still nothing.  Am I missing something with the card?  It's in IT mode and flashed to P20.  

 

Perhaps the cable is trash which seems really unlikely.

 

Thoughts?

Deleted.

 

Pls provide the detail of that cable, i.e. internet link.

Edited September 4, 20241 yr by Vr2Io

I have power.  Disks appear to be spinning.  I've tried each of the 4 ports on the 9201-16e card.  

 

Running lspci the LSI9201-16e shows up as LSI SAS2116.

PCI Address 03:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2116 PCI-Express Fusion-MPT SAS-2 Rev 02 - Should a 9201-16e be detected as SAS2116?

PCI Address 81:00.0 RAID bus controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 rev 03 - Working card.

3 minutes ago, jongregory75 said:

Should a 9201-16e be detected as SAS2116?

Yes correct.

PCI bifurcation on the MB?

 

I'll have to do some research to understand which BIOS bifurcation settings correspond to the specific PCIe slots on the board.  It's an x10DRH-iT board.

Working card is in slot 6. (x8) CPU2

NVidia card is inserted in slot 4 (x16) CPU2

Non-working card is in slot 2. (x8) CPU1

 

Here's the link to the MB on SuperMicro.  https://www.supermicro.com/manuals/motherboard/C600/MNL-1628.pdf

 

Should I run both cards on CPU2 along with the NVidia card?

Edited September 4, 20241 yr by jongregory75

Found this on Reddit, hope useful...

Commenting on an old thread here, but I've found that drives are not recognized with firmware v20 on the LSI 9201-16e. I downgraded to v15 and that seems to work great (drives are recognized, etc). v16-19 might also work, I don't know - and re-flashing v20 might also work. I used v15 because I had another card with that firmware which has worked very well. Note that you cannot downgrade from v20 from within Windows, Linux, etc. You must use a bootable DOS or EFI shell USB disk and the associated sas2flash utility. And you must first erase the current firmware before flashing different firmware. More info: https://www.broadcom.com/support/knowledgebase/1211161501344/flashing-firmware-and-bios-on-lsi-sas-hbas

 

 

Update:

 

I ran the following to erase both bios and firmware.

sas2flsh -c 1 -o -e 6

Shutdown and restarted.

Then ran P20 firmware and bios update.

sas2flsh -c 1 -o -f 9201-16e.bin -b mptsas2.rom

No errors

 

Went back into the MB bios and changed bifurcation.  According to the SM X10DRH-iT manual, PCI-E slot 6 & 7 are on CPU2 and on the same IOU so I set the CPU2a and CPU2c bifurcation to 8X8.

 

Rebooted and got an 8TB red to populate.  I plugged another 8TB red that I didn't remove the 3.3v pin and a 14TB white with 3.3v pin removed but neither of them populated. 

 

I think there is still something with bifurcation or possibly something else in the MB bios that is preventing the other drives from populating.  I'll wait for formatting to finish and then see about my next steps.

 

All 3 drives on the 9211-8i in slot 6 populate.

Only 1 drive on the 9201-16e in slot 7 populate.  

Edited September 5, 20241 yr by jongregory75

Bifurcation should make any difference for this.

Should or shouldn't?

 

I guess the question now becomes, "Why would only one drive show up?"  The 9201-16e is recognized by the OS.  A single drive populates but the other 3 don't.  I can feel them spinning.  More to figure out I guess.  

Update:

 

It's got to be something with the card/firmware.  Since there is a already an 8i card already working, I found a sff-8087 to 4 sata forward breakout cable that I could run through the back of the server and down into the JBOD.  It barely reaches but I was able to plug in 2 of the 14TB WD whites.  I used one of the sata breakout power cables.  Plugged everything in, stopped the array, the drives populated in unassigned devices, I assigned them to the array pool, started the array and they are pre-clearing now.  

 

So at the end of the day, I wasted a lot of time trying to get the 9201-16e to work.  I'll go on amazon and buy a little longer 8087 breakout and plug all 4 in.  That'll give me an additional 36TB which should carry me for a couple years.

 

I won't call this solved but oh well.

9 hours ago, jongregory75 said:

Should or shouldn't?

Sorry, shouldn't