dealbakerjones Posted January 21 Share Posted January 21 Hey everyone, been digging through the threads on getting these cards configured. I have a Dell R620 and have used a PERC H310 so far). I am adding a PowerVault MD1200 and bought an 0TFJRW (LSI SAS2308 / 9206-16e) HBA to get an external SAS port to hook it up to the R620. Everything is connected and powered on (MD1200 drives show ready lights), but the drives in the MD1200 don't appear available for preclear, or anywhere else. 1. the HBA shows in unraid System Devices: IOMMU group 56: [1000:0087] 43:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05) IOMMU group 57: [1000:0087] 45:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT This is also supported when I run lspci: 43:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05) Subsystem: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 Flags: bus master, fast devsel, latency 0, IRQ 41, NUMA node 1, IOMMU group 56 I/O ports at bc00 [size=256] Memory at d0ff0000 (64-bit, non-prefetchable) [size=64K] Memory at d0f80000 (64-bit, non-prefetchable) [size=256K] Expansion ROM at d0000000 [disabled] [size=1M] Capabilities: [50] Power Management version 3 Capabilities: [68] Express Endpoint, MSI 00 Capabilities: [d0] Vital Product Data Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [c0] MSI-X: Enable+ Count=16 Masked- Capabilities: [100] Advanced Error Reporting Capabilities: [1e0] Secondary PCI Express Capabilities: [1c0] Power Budgeting <?> Capabilities: [190] Dynamic Power Allocation <?> Capabilities: [148] Alternative Routing-ID Interpretation (ARI) Kernel driver in use: mpt3sas Kernel modules: mpt3sas 45:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05) Subsystem: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 Flags: bus master, fast devsel, latency 0, IRQ 41, NUMA node 1, IOMMU group 57 I/O ports at ac00 [size=256] Memory at d1ff0000 (64-bit, non-prefetchable) [size=64K] Memory at d1f80000 (64-bit, non-prefetchable) [size=256K] Expansion ROM at d1000000 [disabled] [size=1M] Capabilities: [50] Power Management version 3 Capabilities: [68] Express Endpoint, MSI 00 Capabilities: [d0] Vital Product Data Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [c0] MSI-X: Enable+ Count=16 Masked- Capabilities: [100] Advanced Error Reporting Capabilities: [1e0] Secondary PCI Express Capabilities: [1c0] Power Budgeting <?> Capabilities: [190] Dynamic Power Allocation <?> Capabilities: [148] Alternative Routing-ID Interpretation (ARI) Kernel driver in use: mpt3sas Kernel modules: mpt3sas Running sas2flash shows the card is running in IT mode, and the firmware is pretty up to date (I saw a May 2023 post showing 20.00.07.00 as current, is this right?): ./sas2flash -list LSI Corporation SAS2 Flash Utility Version 20.00.00.00 (2014.09.18) Copyright (c) 2008-2014 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2308_2(D1) Controller Number : 0 Controller : SAS2308_2(D1) PCI Address : 00:43:00:00 SAS Address : 5000d31-0-0083-a41d NVDATA Version (Default) : 14.01.00.08 NVDATA Version (Persistent) : 14.01.00.08 Firmware Product ID : 0x2214 (IT) Firmware Version : 20.00.11.00 NVDATA Vendor : LSI NVDATA Product ID : SAS9206-16e BIOS Version : N/A UEFI BSD Version : N/A FCODE Version : N/A Board Name : SAS9206-16E Board Assembly : H3-25553-01A Board Tracer Number : SV42817207 Finished Processing Commands Successfully. Exiting SAS2Flash. ./sas2flash -c 1 -list LSI Corporation SAS2 Flash Utility Version 20.00.00.00 (2014.09.18) Copyright (c) 2008-2014 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2308_2(D1) Controller Number : 1 Controller : SAS2308_2(D1) PCI Address : 00:45:00:00 SAS Address : 5000d31-0-0083-a425 NVDATA Version (Default) : 14.01.00.08 NVDATA Version (Persistent) : 14.01.00.08 Firmware Product ID : 0x2214 (IT) Firmware Version : 20.00.11.00 NVDATA Vendor : LSI NVDATA Product ID : SAS9206-16e BIOS Version : N/A UEFI BSD Version : N/A FCODE Version : N/A Board Name : SAS9206-16E Board Assembly : H3-25553-01A Board Tracer Number : SV42817207 Finished Processing Commands Successfully. Exiting SAS2Flash. I'm hoping I missed something really silly - can anyone point me in the right direction? jeeves-diagnostics-20240121-1807.zip Quote Link to comment
JorgeB Posted January 22 Share Posted January 22 HBA appears to be initializing correctly, though it's using a strange firmware: Jan 21 16:48:19 jeeves kernel: mpt2sas_cm1: LSISAS2308: FWVersion(20.00.11.00), ChipRevision(0x05), BiosVersion(00.00.00.00) AFAIK latest firmware for SAS2008/2308 is 20.00.07.00, there's also this: Jan 21 18:07:26 jeeves kernel: mpt2sas_cm1: Temperature Threshold flags 0 exceeded for Sensor: 0 !!! Jan 21 18:07:26 jeeves kernel: mpt2sas_cm1: Current Temp In Celsius: 70 Jan 21 18:07:26 jeeves kernel: mpt2sas_cm0: Temperature Threshold flags 0 exceeded for Sensor: 0 !!! Jan 21 18:07:26 jeeves kernel: mpt2sas_cm0: Current Temp In Celsius: 70 Quote Link to comment
dealbakerjones Posted January 22 Author Share Posted January 22 thanks! I'll reflash the firmware with the broadcom latest to be sure - ill check on the temps too, fans have been spinning fine and other temps look ok in the dashboard. Is that temp specific to the HBA ("mpt2sas_cm0")? Quote Link to comment
dealbakerjones Posted January 23 Author Share Posted January 23 9 hours ago, JorgeB said: AFAIK latest firmware for SAS2008/2308 is 20.00.07.00, there's also this: OK I crossflashed updated firmware, checked all the cables, everything is showing solid green/good, checked that I'm in unified mode and connected to EMM0, but still not seeing the drives anywhere. Is there anything else I can check? jeeves-diagnostics-20240122-1927.zip Quote Link to comment
dealbakerjones Posted January 23 Author Share Posted January 23 only other things I've seen online/can think of: - some sas cables are not bi directional, i asked the ebay seller to be sure (cable is new i think) - sometimes the server may not read the card correctly and you need to set the pcie bus to read 2x8 instead of 1x16; this is possible but since the card is recognized correctly, seems unlikely? Quote Link to comment
JorgeB Posted January 23 Share Posted January 23 Everything looks good to me on the HBA side, except for the temps, but that should not cause issues with the disks being detected. Assuming the HBA ports are working problem could be the enclosure, or the cables. Quote Link to comment
dealbakerjones Posted January 27 Author Share Posted January 27 Thanks for your help so far - I'm still stumped! 1. I flashed BIOS to the HBA, but during boot sequence it's still not recognized (BIOS is enabled in the R620, I followed this video https://www.youtube.com/watch?v=ggdbgjm8cRQ) -- but unraid sees the device in PCIe settings (lspci and sas2flash both confirm, and device messages show in unraid system logs) 2. I tried using a known good drive (part of my array) and a new drive (which appears in Tools>Preclear Disk) in the MD1200, but nothing appears anywhere I can find in unraid (lsscsi, Tools>System Devices) [actually the new drive shows up in preclear but doesn't get a green status light in the MD1200, not sure what's up with that) 3. closer look at the MD1200 shows it's actually in an error state, but both EMMs, PSUs, and all HDDs show green/ready, not sure what else I can troubleshoot (replace the backplane? control panel?) Not quite sure where to go from here - if I can get the HBA BIOS to load, I'm hoping I'll be able to tell if the HGST drives need to be formatted to 512 or something (rule out the HBA as the issue), which will confirm that the MD1200 is just dead and I need to return it. jeeves-diagnostics-20240127-1848.zip Quote Link to comment
JorgeB Posted January 28 Share Posted January 28 Drive would still be detected if not using 512B sectors, just wouldn't be assignable, so something else going on. Quote Link to comment
macmann Posted January 28 Share Posted January 28 I have an MD1200 used with an ASUS ROG board and external port controller. I have used it for several years but now the MD1200 is going bananas with fan speed. It still works fine with UNRAID and I have no issues at all other than fan speed. I hope you can figure it out. Quote Link to comment
dealbakerjones Posted February 6 Author Share Posted February 6 OK still digging on this, I bought an external cable that allows me to connect directly to the HBA with known SATA drives (am powering separately, they do spin up), but still see nothing in syslog or unassigned devices . . .starting to think my card just doesn't work Quote Link to comment
dealbakerjones Posted February 7 Author Share Posted February 7 (edited) OH MY GOSH aos saving me again watch this video, solved my problem: Summary: 1. Get lsiutil (good samaritan hosted here: https://github.com/thomaslovell/LSIUtil/), (and save to unraid) 2. ./lsiutil -s to show all devices If you see "SAS2308's links are off, off, off, off, off, off, off, off", keep going! (this is bad) 3. select your device with ./lsutil -p 2 -e (mine is on port 2, my internal card is port 1) 4. 68 shows port state summary, look for these problematic settings: "port stat off,off,off" (bad, as above) Link Enabled: No (bad) SSP Target Enabled: No (bad) 5. 61 sets default values, then 68 to view port state again. this enables the links, but port state will still show "off,off,off" because you need to reset to take effect 6. 99 to reset the chip, then 68 to check that ports with plugs in em say "6.0 G" (connection speed) and if disconnected say "down" -- not "off"! Once I did this, my plugged-in drives immediately appeared in UD! Edited February 7 by dealbakerjones 1 Quote Link to comment
JorgeB Posted February 8 Share Posted February 8 Good find, first time I see this issue. Quote Link to comment
Quadrapole Posted Wednesday at 05:50 AM Share Posted Wednesday at 05:50 AM On 2/7/2024 at 3:40 PM, dealbakerjones said: OH MY GOSH aos saving me again watch this video, solved my problem: Summary: 1. Get lsiutil (good samaritan hosted here: https://github.com/thomaslovell/LSIUtil/), (and save to unraid) 2. ./lsiutil -s to show all devices If you see "SAS2308's links are off, off, off, off, off, off, off, off", keep going! (this is bad) 3. select your device with ./lsutil -p 2 -e (mine is on port 2, my internal card is port 1) 4. 68 shows port state summary, look for these problematic settings: "port stat off,off,off" (bad, as above) Link Enabled: No (bad) SSP Target Enabled: No (bad) 5. 61 sets default values, then 68 to view port state again. this enables the links, but port state will still show "off,off,off" because you need to reset to take effect 6. 99 to reset the chip, then 68 to check that ports with plugs in em say "6.0 G" (connection speed) and if disconnected say "down" -- not "off"! Once I did this, my plugged-in drives immediately appeared in UD! Forgive me because I'm a noob but how exactly do I run lsiutil in unraid? I downloaded the files from the github, and i navigated to /LSIUtil/Binaries/LSIutil_1.70_release_binaries/linux , but when I try to run lsiutil with ./lsiutil , it gives me the error "cannot execute binary file: Exec format error" Can someone point me in the right direction? Quote Link to comment
dealbakerjones Posted Wednesday at 10:50 AM Author Share Posted Wednesday at 10:50 AM @Quadrapole Looked back through my old files, I have a ton of different versions of these utilities. I got the same error as you when I ran `./lsiutil` directly, but running the x86_64 version worked: `./lsiutil.x86_64 -s` Both files should be here: https://github.com/thomaslovell/LSIUtil/tree/master/Binaries/LSIutil_1.70_release_binaries/linux Good luck! 1 Quote Link to comment
Quadrapole Posted Thursday at 04:30 AM Share Posted Thursday at 04:30 AM 17 hours ago, dealbakerjones said: @Quadrapole Looked back through my old files, I have a ton of different versions of these utilities. I got the same error as you when I ran `./lsiutil` directly, but running the x86_64 version worked: `./lsiutil.x86_64 -s` Both files should be here: https://github.com/thomaslovell/LSIUtil/tree/master/Binaries/LSIutil_1.70_release_binaries/linux Good luck! Got my LSI 9206 working after your help. Thanks a lot! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.