1/3 of backplane drives not showing in unRAID

[lazerdave]

I've got a Supermicro server with 12 drive bays running unRAID 6.8.2 Plus. Right now, the array has 5 data drives, 2 parity, and 1 SSD cache drive.  It had shipped with SAS#3 completely disconnected and SAS#2 running from a breakout cable with only 2/4 of the SATA cables with anywhere to go. SAS#1 is using the first 4 SATA ports on the motherboard and has always been fine.  

 

So I bought a controller card with 2 mini SAS ports - an IOCrest 88SE9215, which according to the Hardware Compatibility List seemed it would be supported out of the box.  First order of business was to get SAS#3's bays working, which it did the first time I powered it up after installing the card.

 

Next, I disconnected the breakout cable from SAS#2 and ran it to the other port on the controller. During boot, it detects the drives connected to SAS#2, reporting their details properly. However, they don't appear in unRAID in the CA Unassigned Drives section or in the PreClear plugin.

 

Something that might be noteworthy, the breakout cable has the usual 4 SATAs, but an additional ribbon cable going to an 8-pin power(?) connector.  Can't find any other mini-SAS to SATA cables online that have that configuration.  The backplane has a Molex for each of the SAS connectors, and all of the indicator lights are working, so the assumption is that the issue isn't power.

 

The splash screen for the drives that aren't showing is:

Marvell 88SE92xx Adapter - BIOS Version 1.0.0.1028 PCIe x1 5.0Gbps [Bus:Dev] = [4:0] 
Mode: PassThru AHCI 
Port Disk Name Size Speed 
S2 SATA: ST3000DM001-serial 3TB 6Gb/s 
S3 SATA: ST3000VN000-serial 3TB 6Gb/s

For the ones that DO show, it's:

Marvell 88SE92xx Adapter - BIOS Version 1.0.0.1028 PCIe x1 5.0Gbps [Bus:Dev] = [6:0] 
Mode: PassThru AHCI 
Port Disk Name Size Speed 
S0 SATA: WCB serial 8TB 6Gb/s 
S1 SATA: SanDisk SSD Plus Serial 
S2 SATA: ST2000DM006-serial 3TB 6Gb/s 
S3 SATA: ST2000VM001-serial 3TB 6Gb/s

The mode of PassThru AHCI is concerning.  I understand that many controller cards need to be placed into IT mode, but I can't find how to get into the on card BIOS to try to change that.

 

I'm not sure what to do next to get those drives into my array. Any kind souls out there with advice, please?

rack-diagnostics-20200402-1715.zip

[JorgeB]

Marvell controllers have various known issues, and are not recommended, but this can sometimes help:

https://forums.unraid.net/bug-reports/stable-releases/disks-missing-after-upgrading-to-670-r536/?do=findComment&comment=4583

 

[lazerdave]

Thanks for the quick reply.  No luck, I'm afraid.  I actually tried both solutions separately, append initrd=/bzroot scsi_mod.use_blk_mq=1 and append initrd=/bzroot iommu=pt.  (I actually did it on bzroot-gui, if that matters...)

 

Should I just give up on this controller and order a Dell H310 like I saw someone else mention in the link you sent?  I found a Dell H310 6Gbps SAS HBA w/ LSI 9211-8i P20 IT Mode for ZFS FreeNAS unRAID for a good price.  Just want the damned thing working, y'know?

[Vr2Io]

12 hours ago, lazerdave said:

So I bought a controller card with 2 mini SAS ports - an IOCrest 88SE9215

9215 was a 4 port controller, how come have 2 mini SAS connector ( 8 disk ), pls provide the product link.

 

What are the connecter type on other normal 9215 ?

[lazerdave]

This is what I bought and have been fighting.

[Vr2Io]

38 minutes ago, lazerdave said:

This is what I bought and have been fighting.

Note. ASM1806 PCIe bridge elimate the need for a Port Multiper ... There are two 9215 on card for 8 disk.

 

14 hours ago, lazerdave said:

an additional ribbon cable going to an 8-pin power(?) connector.

The cable should be a reverse type, the ribbon cable was GPIO and always not use. Does 3 minSAS cable are this type ? If not, pls swap between them.

 

You post the splash screen show it have two 9215, it could be come from this card all. Due to you haven't say what other controller to provide the 3rd minSAS link.

 

If the 3rd minSAS was come from onboard, then actually you need one reverse + two forward cable.

 

But it is true, Marvell controller such 9215 should be avoid, if you buy LSI HBA, pls save the money for buying minSAS to SATA cable.

 

Edit : I misunderstang the connector at the backplane was SATA

Edited April 3, 2020 by Benson

[lazerdave]

SAS#1 from the backplane also uses the same breakout cable with 4 SATA and the weird ribbon thing.  Because the motherboard has sufficient SATA ports for it to be used as-is, that's what I've done.

[JorgeB]

The problem is that the Marvell controller it's failing to identify the disks, this a known issue with Marvell and IOMMU enable, though usually the 9230 is the more affected model, if the workaround above didn't work it should work with IOMMU disable (if you don't need it):

 

Apr  2 10:26:05 Rack kernel: ata9.00: qc timeout (cmd 0xec)
Apr  2 10:26:05 Rack kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Apr  2 10:26:05 Rack kernel: ata10.00: qc timeout (cmd 0xec)
Apr  2 10:26:05 Rack kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Apr  2 10:26:05 Rack kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Apr  2 10:26:05 Rack kernel: ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Apr  2 10:26:05 Rack kernel: DMAR: DRHD: handling fault status reg 102
Apr  2 10:26:05 Rack kernel: DMAR: [INTR-REMAP] Request device [04:00.0] fault index 1b [fault reason 38] Blocked an interrupt request due to source-id verification failure
Apr  2 10:26:05 Rack kernel: DMAR: DRHD: handling fault status reg 202
Apr  2 10:26:05 Rack kernel: DMAR: [INTR-REMAP] Request device [04:00.0] fault index 1b [fault reason 38] Blocked an interrupt request due to source-id verification failure
Apr  2 10:26:16 Rack kernel: ata9.00: qc timeout (cmd 0xec)
Apr  2 10:26:16 Rack kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Apr  2 10:26:16 Rack kernel: ata9: limiting SATA link speed to 3.0 Gbps
Apr  2 10:26:16 Rack kernel: ata10.00: qc timeout (cmd 0xec)
Apr  2 10:26:16 Rack kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Apr  2 10:26:16 Rack kernel: ata10: limiting SATA link speed to 3.0 Gbps
Apr  2 10:26:16 Rack kernel: ata9: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
Apr  2 10:26:16 Rack kernel: ata10: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
Apr  2 10:26:16 Rack kernel: DMAR: DRHD: handling fault status reg 302
Apr  2 10:26:16 Rack kernel: DMAR: [INTR-REMAP] Request device [04:00.0] fault index 1b [fault reason 38] Blocked an interrupt request due to source-id verification failure
Apr  2 10:26:16 Rack kernel: DMAR: DRHD: handling fault status reg 402
Apr  2 10:26:16 Rack kernel: DMAR: [INTR-REMAP] Request device [04:00.0] fault index 1b [fault reason 38] Blocked an interrupt request due to source-id verification failure
Apr  2 10:26:46 Rack kernel: ata9.00: qc timeout (cmd 0xec)
Apr  2 10:26:46 Rack kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Apr  2 10:26:46 Rack kernel: ata10.00: qc timeout (cmd 0xec)
Apr  2 10:26:46 Rack kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)

 

[Vr2Io]

If 04:00 and 06:00 are the two 9215 from this controller, then quite interesting for 06:00 9215 without problem.

Problem may come from ASM1806 mainly.

 

** I haven't check the diagnostics **

Edited April 3, 2020 by Benson

[lazerdave]

Pardon my ignorance, but do I turn off IOMMU in BIOS or unRAID?

[JorgeB]

In the board BIOS.

[lazerdave]

That's what I thought, too, but I can't find it anywhere in there...  My System Information shows that IOMMU is enabled, so I know it's gotta be there someplace, but damned if I can find it.

[JorgeB]

It's called VT-d for Intel CPUs.

[lazerdave]

Found it in the North Bridge settings.  Disabled it and now the drives appear in Unassigned Devices!!  Thank you, Johnnie.Black!

 

Of course, I probably don't really want to live without IOMMU in the long run.  Think I'll try to get a refund on this card once that Dell arrives and I get it up and running with VT-d enabled.

 

Thanks a million everyone, especially Johnnie.