Upgraded to 6.9.2, and now none of my drives show up. I have been troubleshooting for hours and nothing is working. Please help!

[thejellydude]

I honestly have no idea what's going on. It started with one drive not working after the upgrade. Then I started troubleshooting (Powering off, reseating drives, unplugging and replugging in all cables, plugging in to mobo directly instead of expansion card, etc) and now only one of my eight drives shows up. Can someone please help? I have no idea why this is happening.

nasbox-diagnostics-20210408-1635.zip

[Hoopster]

1 hour ago, thejellydude said:

and now only one of my eight drives shows up

Has your Dell H310 always been in RAID mode?  If so, the only way to get drives to show up with the Megaraid driver is if each drive is configured as a separate RAID 0 or in JBOD mode if that is supported.  Drives attached to a controller in RAID mode do not generally show up in unRAID.  Megaraid driver and unRAID is usually a problematic combination.

 

This is your controller (note it is using the MegaRAID driver):

01:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS 2008 [Falcon] [1000:0073] (rev 03)
    Subsystem: Dell PERC H310 [1028:1f78]
    Kernel driver in use: megaraid_sas
    Kernel modules: megaraid_sas

 

Whatever mode your card is in, loading the MegaRAID driver only shows one drive recognized:

ASBOX kernel: megaraid_sas 0000:01:00.0: pci id		: (0x1000)/(0x0073)/(0x1028)/(0x1f78)
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: unevenspan support	: no
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: firmware crash dump	: no
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: JBOD sequence map	: disabled
Apr  8 14:33:25 NASBOX kernel: scsi host1: Avago SAS based MegaRAID driver
Apr  8 14:33:25 NASBOX kernel: i2c i2c-0: Successfully instantiated SPD at 0x50
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain package 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain dram 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: scsi 1:0:7:0: Direct-Access     ATA      WDC WD80EFAX-68L 0A83 PQ: 0 ANSI: 5

 

I happen to have the same card in my server but I have it cross-flashed to IT mode where it becomes a pure HBA (no RAID support) that passes through each disk to unRAID without issues. Note that in IT mode is is loading the mpt3sas driver.

 

For the record, I am also running the latest version of unRAID; 6.9.2

 

Here are my controller details also for a Dell H310:

02:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
    DeviceName:  Onboard SAS
    Subsystem: Dell 6Gbps SAS HBA Adapter [1028:1f1c]
    Kernel driver in use: mpt3sas
    Kernel modules: mpt3sas

 

Upgrading unRAID should have had no effect upon the firmware on your Dell H310, but, it is possible that Linux kernel changes between versions have affected how the driver behaves.

[thejellydude]

1 minute ago, Hoopster said:

Has your Dell H310 always been in RAID mode?  If so, the only way to get drives to show up with the Megaraid driver is if each drive is configured as a separate RAID 0 or in JBOD mode if that is supported.  Drives attached to a controller in RAID mode do not generally show up in unRAID.  Megaraid driver and unRAID is usually a problematic combination.

 

This is your controller (note it is using the MegaRAID driver):

01:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS 2008 [Falcon] [1000:0073] (rev 03)
    Subsystem: Dell PERC H310 [1028:1f78]
    Kernel driver in use: megaraid_sas
    Kernel modules: megaraid_sas

 

Whatever mode your card is in, loading the MegaRAID driver only shows one drive recognized:

ASBOX kernel: megaraid_sas 0000:01:00.0: pci id		: (0x1000)/(0x0073)/(0x1028)/(0x1f78)
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: unevenspan support	: no
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: firmware crash dump	: no
Apr  8 14:33:25 NASBOX kernel: megaraid_sas 0000:01:00.0: JBOD sequence map	: disabled
Apr  8 14:33:25 NASBOX kernel: scsi host1: Avago SAS based MegaRAID driver
Apr  8 14:33:25 NASBOX kernel: i2c i2c-0: Successfully instantiated SPD at 0x50
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain package 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain dram 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
Apr  8 14:33:25 NASBOX kernel: scsi 1:0:7:0: Direct-Access     ATA      WDC WD80EFAX-68L 0A83 PQ: 0 ANSI: 5

 

I happen to have the same card in my server but I have it cross-flashed to IT mode where it becomes a pure HBA (no RAID support) that passes through each disk to unRAID without issues. Note that in IT mode is is loading the mpt3sas driver.

 

For the record, I am also running the latest version of unRAID; 6.9.2

 

Here are my controller details also for a Dell H310:

02:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
    DeviceName:  Onboard SAS
    Subsystem: Dell 6Gbps SAS HBA Adapter [1028:1f1c]
    Kernel driver in use: mpt3sas
    Kernel modules: mpt3sas

 

Upgrading unRAID should have had no effect upon the firmware on your Dell H310, but, it is possible that Linux kernel changes between versions have affected how the driver behaves.

I thought that might be the issue, but that wouldn't affect the drives when plugged directly into the mothrboard, correct? I powered down the system, removed all the drives from the expansion card, and plugged four directly into the motherboard. Those drives did not show up.

[Hoopster]

1 minute ago, thejellydude said:

that wouldn't affect the drives when plugged directly into the mothrboard, correct?

Correct.  Should not affect drives attached directly to the motherboard. 

 

However, if they do not show up in the motherboard BIOS/UEFI, they will not show up in unRAID.  Go into your BIOS and see if the drives show up there.

[thejellydude]

13 minutes ago, Hoopster said:

Correct.  Should not affect drives attached directly to the motherboard. 

 

However, if they do not show up in the motherboard BIOS/UEFI, they will not show up in unRAID.  Go into your BIOS and see if the drives show up there.

Unfortunately I upgraded my board/cpu recently, and I can not for the life of me figure out where the drive information is located. I'll probably research that in a bit.

[Hoopster]

1 minute ago, thejellydude said:

Unfortunately I upgraded my board/cpu recently, and I can not for the life of me figure out where the drive information is located. I'll probably research that in a bit.

OK.  When other issues are resolved, you really should consider cross-flashing that H310 with IT firmware version 20.0.7.0 (the latest).  It will make your life a lot easier.

[thejellydude]

2 hours ago, Hoopster said:

OK.  When other issues are resolved, you really should consider cross-flashing that H310 with IT firmware version 20.0.7.0 (the latest).  It will make your life a lot easier.

Would you possibly have instructions on how to do that?

[Hoopster]

10 minutes ago, thejellydude said:

Would you possibly have instructions on how to do that?

Here is one guide:

https://tylermade.net/2017/06/27/how-to-crossflash-perc-h310-to-it-mode-lsi-9211-8i-firmware-hba-for-freenas-unraid/

 

Here is another:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

And here is a reddit thread from someone who tried both tutorials and ran into a problems which were solved with the help of others:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

There are many threads about cross-flashing a RAID controller to IT mode in the unRAID forums.  Here is a very long one on the subject:

 

[thejellydude]

4 minutes ago, Hoopster said:

Here is one guide:

https://tylermade.net/2017/06/27/how-to-crossflash-perc-h310-to-it-mode-lsi-9211-8i-firmware-hba-for-freenas-unraid/

 

Here is another:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

And here is a reddit thread from someone who tried both tutorials and ran into a problems which were solved with the help of others:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

There are many threads about cross-flashing a RAID controller to IT mode in the unRAID forums.  Here is a very long one on the subject:

 

Thank you, I will look at these in the future.

 

Unfortunately, the drives aren't showing up in Bios when directly plugged into the system. I fear that the combination of the upgrade/troubleshooting may have somehow killed all my drives. I seriously hope that is not the case.

[trurl]

Just now, thejellydude said:

fear that the combination of the upgrade/troubleshooting may have somehow killed all my drives. I seriously hope that is not the case.

I seriously doubt it is the case. Killing all your actual disks would require a serious hardware malfunction, such as incompatible power cables.

[thejellydude]

4 minutes ago, trurl said:

I seriously doubt it is the case. Killing all your actual disks would require a serious hardware malfunction, such as incompatible power cables.

Maybe the word "corrupted" would have been better. Do you have any suggestions? This is the case I have. Originally after the update my second data drive stopped working, so I removed it from the hot-swap bay and troubleshooted it. Eventually I decided to see if the part was dead, so I shuffled around the drives. This caused one of my parity drives to go offline as well. I now suspected that perhaps the backplane inside the case was faulty, so I removed four of the drives from the hotswap bays and connected them directly to the breakout cables from my Dell H310. Now even less drives were showing up. I guessed that maybe my H310 was faulty, so I unplugged my hard drives and plugged them into my motherboard directly. Still not luck.

However, my cache drive is always detected by the bios. If I swap around the sata cables between the drives, the one that the cache drive is connected to always shows up in the BIOS.

[Frank1940]

14 minutes ago, thejellydude said:

so I removed four of the drives from the hotswap bays and connected them directly to the breakout cables from my Dell H310. Now even less drives were showing up.

 

Google  '3.3 volt pin problem'.  That is one 'gotcha' that can bite one's posterior.. 

[Hoopster]

@thejellydude The 3.3v problem to which Frank1940 refers is very often an issue with drives shucked form external enclosures.  They work fine when used in many disk cages that are powered by Molex, Molex to SATA power adapters, etc.  When direct connected to SATA power, they go into a reset state and are not recognized in the BIOS or by unRAID.

 

If your drives are shucked, this could be an issue.  If they were sold as bare or OEM desktop of server drives, this is usually not a problem.  However, even in those cases, some drives could still have the issue even if they were not shucked drives.  It depends on the drive model.

 

The solution is to cover pin 3 of the drive SATA connector with Kapton tape.  I have needed to do this on a couple of shucked drives.

Edited April 9, 2021 by Hoopster

[thejellydude]

33 minutes ago, Frank1940 said:

 

Google  '3.3 volt pin problem'.  That is one 'gotcha' that can bite one's posterior.. 

Good call! My backpane uses a molex connection, so I forgot my 12TB drives have the 3.3v issue. My 8TB drives also don't have the 3.3v issue however and I can use them to test. Right now I can't get my system to post at all though. I see the splash screen for a second, but it doesn't enter BIOS anymore. I regret upgrading so much.

[thejellydude]

I managed to get it to post and now the 8TB drives show up! I'm going to try updating the H310 and see if the drives will show up using that again.

[thejellydude]

1 hour ago, Hoopster said:

Here is one guide:

https://tylermade.net/2017/06/27/how-to-crossflash-perc-h310-to-it-mode-lsi-9211-8i-firmware-hba-for-freenas-unraid/

 

Here is another:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

And here is a reddit thread from someone who tried both tutorials and ran into a problems which were solved with the help of others:

https://techmattr.wordpress.com/2016/04/11/updated-sas-hba-crossflashing-or-flashing-to-it-mode-dell-perc-h200-and-h310/

 

There are many threads about cross-flashing a RAID controller to IT mode in the unRAID forums.  Here is a very long one on the subject:

 

 I just realized you accidentally did the Techmattr thread twice instead of the reddit thread! Would you mind posting that one?

[Hoopster]

57 minutes ago, thejellydude said:

 I just realized you accidentally did the Techmattr thread twice instead of the reddit thread! Would you mind posting that one?

Oops!  Here you go.

 

[thejellydude]

I attempted to folIow the techmattr guide. The link to creating a bootable USB is no longer there, so I simple downloaded Rufus, ticked the "Download FreeDos option" and click "start". I think copied over the files from the guide. However, Ikeep getting an EFI shell instead of an actual DOS prompt. Does anyone have any idea why that might be the case?

 

Edited April 9, 2021 by thejellydude

[thejellydude]

I attempted downgrading and that hasn't fixed the issue. If anyone knows why I'm getting an EFI shell instead of FreeDOS, the help would be appreciated.

[Frank1940]

I don't have an answer but when I am confronted with this type of situation, I turn to Google.  Use terms like 'EFI shell' 'FreeDos' and the name of the program you used to generate a boot disk.

[thejellydude]

1 hour ago, Frank1940 said:

I don't have an answer but when I am confronted with this type of situation, I turn to Google.  Use terms like 'EFI shell' 'FreeDos' and the name of the program you used to generate a boot disk.

I've been trying. Unfortunately it's not really pulling up anything useful. I'm pretty sure that my H310 used to be flashed with IT Mode and I think it somehow got messed up during the upgrade. It now shows up on my motherboard as an H310 device. It should do that if it's flashed with the P20 firmware, correct?

[bonienl]

9 hours ago, thejellydude said:

I attempted to folIow the techmattr guide. The link to creating a bootable USB is no longer there, so I simple downloaded Rufus, ticked the "Download FreeDos option" and click "start". I think copied over the files from the guide. However, Ikeep getting an EFI shell instead of an actual DOS prompt. Does anyone have any idea why that might be the case?

 

 

You need to make the disk bootable.

 

Open a DOS window as administrator and run the make_bootable script. Ensure your USB stick is labeled UNRAID (all capitals)

[thejellydude]

8 minutes ago, bonienl said:

 

You need to make the disk bootable.

 

Open a DOS window as administrator and run the make_bootable script. Ensure your USB stick is labeled UNRAID (all capitals)

Does RUFUS not do this? The drive should be bootable already since it launches the EFI shell, right? And I don't think it should need to be named UNRAID, since this is my FreeDOS boot drive. My UnRAID one works just fine.

[Frank1940]

37 minutes ago, thejellydude said:

I've been trying. Unfortunately it's not really pulling up anything useful. I'm pretty sure that my H310 used to be flashed with IT Mode and I think it somehow got messed up during the upgrade. It now shows up on my motherboard as an H310 device. It should do that if it's flashed with the P20 firmware, correct?

Try including the MB information into the google term mix.  (Getting results out google is usually a matter of using combinations of permutations...)

[bonienl]

21 minutes ago, thejellydude said:

Does RUFUS not do this? The drive should be bootable already since it launches the EFI shell, right? And I don't think it should need to be named UNRAID, since this is my FreeDOS boot drive. My UnRAID one works just fine.

 

Sorry, I was mixing things up