fredhoon Posted September 12, 2017 Share Posted September 12, 2017 Has anyone experienced issues with P20.00.07.00 firmware on SAS2008 cards not detecting drives?I've (un)successfully crossflashed a Fujitsu D2067 A11 and Dell H310 to 9211-8i P20-IT via both sas2flsh.exe and sas2flash.efi and both cards are experiencing the same issue: Pre-flash the OEM Bios can see 6of6 SATA drives connected across both channels via fan-out cables. Post-flash both cards can only see the same 2of6 drives across both channels regardless of fan out cable connected port & channel. If I change the port drive connections around the cards will still only see the same 2x drives. Test drives are 2x Seagate 1TB, 1x Samsung 1TB & 3x Seagate 300GB, after P20 crossflash only 1x Seagate 1TB & 1x Samsung 1TB can been seen by the controller Bios At first I thought it might be an issue with the modified D2067 A11/A21 firmware however the exact same issue is present on the H310 card, same drives. I'm going to try again with P19 or P18 firmware and without a Bios, maybe even try a fresh download of the 9210-8i P20 firmware from the Broadcom website. It's a strange issue and I can't logically explain / troubleshoot the symptoms, has anyone experienced this before or able to offer some pointers to help me troubleshoot further? Quote Link to comment
Fireball3 Posted September 12, 2017 Share Posted September 12, 2017 Welcome to the unRAID forums. You have a typo in there "Fujitsu D2067 A11". It's D2607. Haven't heard of this kind of issue yet. P20.00.07.00 is working fine. Maybe one of your old drives is causing trouble? Have you tried increasing drive count step-by-step? Where do you check drive presence? In the card BIOS or fdisk -l? Maybe another cable, just to rule that out? Quote Link to comment
fredhoon Posted September 12, 2017 Share Posted September 12, 2017 2 hours ago, Fireball3 said: Haven't heard of this kind of issue yet. P20.00.07.00 is working fine. I was afraid of that after my googling came up short. I'm checking drive presence in the Card Bios (Ctrl+C config utility), GParted Live CD and left it to boot windows 7 on the desktop I was using for the UEFI shell flash (device manager and HDD Sentinel could only see the two drives). I haven't tried fdisk -I, will give that a bash on a live CD/USB, I haven't tried the card in my UnRAID box yet as I didn't want to tear it down until I confirmed at least one of the cards were working. I suppose I' ll also try querying the card with MegaCLI (or whatever the equivalent is for IT firmware). I've tried a different cable but haven't tried drives one by one as yet, I'll give that a go as well. Cheers for the pointers. Quote Link to comment
nthu9280 Posted September 12, 2017 Share Posted September 12, 2017 Assuming you followed the instructions on the D2607 thread. That card requires correct SBR (A11/A21) and also modified/patched version of sas2flsh (sas2hax) and correct sequence of steps. What perplexes me is that H310 also having the same issue for you. Sbrempty.bin should work fine with this card. You can reflash just the SBR file using megarec and see if rest of the ports are enabled again. Quote Link to comment
Fireball3 Posted September 12, 2017 Share Posted September 12, 2017 2 hours ago, fredhoon said: I was afraid of that after my googling came up short. I'm checking drive presence in the Card Bios (Ctrl+C config utility), GParted Live CD and left it to boot windows 7 on the desktop I was using for the UEFI shell flash (device manager and HDD Sentinel could only see the two drives). I haven't tried fdisk -I, will give that a bash on a live CD/USB, I haven't tried the card in my UnRAID box yet as I didn't want to tear it down until I confirmed at least one of the cards were working. I suppose I' ll also try querying the card with MegaCLI (or whatever the equivalent is for IT firmware). I've tried a different cable but haven't tried drives one by one as yet, I'll give that a go as well. Cheers for the pointers. OK, I guess fdisk will come up with the same story. Presuming the crossflash went OK, I don't believe it's the controller. You could also try another PCIe port and/or another board. Maybe drop it into the unRAID and see what happens. Quote Link to comment
fredhoon Posted September 17, 2017 Share Posted September 17, 2017 (edited) On 12/09/2017 at 8:28 PM, nthu9280 said: Assuming you followed the instructions on the D2607 thread. That card requires correct SBR (A11/A21) and also modified/patched version of sas2flsh (sas2hax) and correct sequence of steps. What perplexes me is that H310 also having the same issue for you. Sbrempty.bin should work fine with this card. You can reflash just the SBR file using megarec and see if rest of the ports are enabled again. The D2607 is an A11 variant, however I did try the A21 SBR (before flashing the H310) just in case, it disabled one channel as expected so I flashed A11 back. Both cards are showing the correct FW and BIOS revisions on boot and in the Ctrl+C config utility so I presume they are flashed correctly. On 12/09/2017 at 10:10 PM, Fireball3 said: OK, I guess fdisk will come up with the same story. Presuming the crossflash went OK, I don't believe it's the controller. You could also try another PCIe port and/or another board. Maybe drop it into the unRAID and see what happens. This morning I've tried the card in another motherboard (S5000 chipset) which resulted in the same 2x drives detected and then started testing drives one by one. After running through a heap of old drives I've confirmed that both the D2607 & H310 with 9211-8i 20.00.07.00 IT mode FW will detect 8 of 8 drives connected, it's just strange that the drives they won't detect were previously detected by the OEM IR mode FW and work fine in other PC's. It appears the certain drives just won't detect with the 20.00.07.00 FW, although they work fine on AHCI motherbord SATA ports and I copied data off them in my Z170 Chipset windows PC. For info, none of my 320G or 200G Seagate drives could be detected and 2of3 1TB Seagate drives wouldn't detect on the SAS2008 with 20.00.07.00 IT mode FW (I'm going to test these on my SAS3008 card next). 2of2 1TB Samsung 7.2k drives, 3of3 1.5TB Samsung 5.4k drives, 1.5TB Seagate 7.2k drive & 150G WD 10k Raptor were all detected OK. Thanks for the help guys, time to tear down the UnRAID box and upgrade the old SAS1068/1078 controllers. And also thanks a heap to everyone in this thread for all the hard work putting the flashing guides together! Edited September 17, 2017 by fredhoon Quote Link to comment
fredhoon Posted September 19, 2017 Share Posted September 19, 2017 (edited) On 17/09/2017 at 10:37 AM, fredhoon said: <snip>...I've confirmed that both the D2607 & H310 with 9211-8i 20.00.07.00 IT mode FW will detect 8 of 8 drives connected, it's just strange that the drives they won't detect were previously detected by the OEM IR mode FW and work fine in other PC's. It appears the certain drives just won't detect with the 20.00.07.00 FW, although they work fine on AHCI motherbord SATA ports and I copied data off them in my Z170 Chipset windows PC. I've just tried the drives on a SAS3008 controller (M1215 flashed to 9300-8i P14 IT-mode) and the drives wouldn't show either. I had originally thought it was a SATA150 limitation but discounted it as the WD Raptor was working at this speed, with this in mind I checked over the drives again and the "force SATA150/300" jumper was in place at 150 for all that't weren't detecting on the 3x HBAs. Jumper removed = drives detected, no issues with P20.00.07 FW, only user error! I could have saved a heap of time by being a bit more thorough and less hasty jumping to conclusions. Edited September 19, 2017 by fredhoon 1 Quote Link to comment
kricker Posted October 10, 2017 Share Posted October 10, 2017 SAS9201-8ISent from my Nexus 5X using Tapatalk Quote Link to comment
Fireball3 Posted October 22, 2017 Share Posted October 22, 2017 @Jonny I just flashed a D2607 A11 with the toolset you put together. This is my result: LSI Corporation SAS2 Flash Utility Version 19.00.00.00 (2014.03.17) Copyright (c) 2008-2014 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2008(B1) Controller Number : 0 Controller : SAS2008(B1) PCI Address : 00:01:00:00 SAS Address : 5003005censored:) NVDATA Version (Default) : 14.01.00.08 NVDATA Version (Persistent) : 14.01.00.08 Firmware Product ID : 0x2213 (IT) Firmware Version : 20.00.07.00 NVDATA Vendor : LSI NVDATA Product ID : SAS9211-8i BIOS Version : N/A UEFI BSD Version : N/A FCODE Version : N/A Board Name : SAS9211-8i Board Assembly : N/A Board Tracer Number : N/A Finished Processing Commands Successfully. Exiting SAS2Flash. Yea, looks good, but unfortunately the controller is not detecting any drive on any port. It says A11-GS1 on the PCB. Attached is the original SBR dump. Any advice what I could try next? Also, could you try the command: sas2flsh.efi -l adapter.txt -c 0 -list My machine is locking up when I try to log. Instead i need to pipe the output into a file. "-c 0 -list" alone ist working for some reason... Could you also show me your output of e.g.: megarec -writesbr 0 SBR-A11.bin The last word is "success" but in between i get: Warning! IO Base address high. Currently not supported. Warning! IO Base address high. Currently not supported. O-SBR.BIN 1 Quote Link to comment
FingerlessGloves Posted October 22, 2017 Share Posted October 22, 2017 (edited) 34 minutes ago, Fireball3 said: @Jonny I just flashed a D2607 A11 with the toolset you put together. This is my result: LSI Corporation SAS2 Flash Utility Version 19.00.00.00 (2014.03.17) Copyright (c) 2008-2014 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2008(B1) Controller Number : 0 Controller : SAS2008(B1) PCI Address : 00:01:00:00 SAS Address : 5003005censored:) NVDATA Version (Default) : 14.01.00.08 NVDATA Version (Persistent) : 14.01.00.08 Firmware Product ID : 0x2213 (IT) Firmware Version : 20.00.07.00 NVDATA Vendor : LSI NVDATA Product ID : SAS9211-8i BIOS Version : N/A UEFI BSD Version : N/A FCODE Version : N/A Board Name : SAS9211-8i Board Assembly : N/A Board Tracer Number : N/A Finished Processing Commands Successfully. Exiting SAS2Flash. Yea, looks good, but unfortunately the controller is not detecting any drive on any port. It says A11-GS1 on the PCB. Attached is the original SBR dump. Any advice what I could try next? Also, could you try the command: sas2flsh.efi -l adapter.txt -c 0 -list My machine is locking up when I try to log. Instead i need to pipe the output into a file. "-c 0 -list" alone ist working for some reason... Could you also show me your output of e.g.: megarec -writesbr 0 SBR-A11.bin The last word is "success" but in between i get: Warning! IO Base address high. Currently not supported. Warning! IO Base address high. Currently not supported. O-SBR.BIN Try this moffied SBR if it doesn't work try the A21 SBR. Had that work on A11 cards before now. O-SBR.BIN I would show you output but, I don't have one free to use. Also make sure the card is the the first PCI slot. EDIT: https://marcan.st/2016/05/crossflashing-the-fujitsu-d2607/ That site might help you also. Edited October 22, 2017 by Jonny Quote Link to comment
Fireball3 Posted October 22, 2017 Share Posted October 22, 2017 2 hours ago, Jonny said: Try this moffied SBR LSI Corporation SAS2 Flash Utility Version 19.00.00.00 (2014.03.17) Copyright (c) 2008-2014 LSI Corporation. All rights reserved No LSI SAS adapters found! Limited Command Set Available! ERROR: Command Not allowed without an adapter! ERROR: Couldn't Create Command -c Exiting Program. 2 hours ago, Jonny said: if it doesn't work try the A21 SBR. LSI Corporation SAS2 Flash Utility Version 20.00.00.00 (2014.09.18) Copyright (c) 2008-2014 LSI Corporation. All rights reserved Adapter Selected is a LSI SAS: SAS2008(B1) Controller Number : 0 Controller : SAS2008(B1) PCI Address : 00:01:00:00 SAS Address : 5003005-..... NVDATA Version (Default) : 14.01.00.08 NVDATA Version (Persistent) : 14.01.00.08 Firmware Product ID : 0x2213 (IT) Firmware Version : 20.00.07.00 NVDATA Vendor : LSI NVDATA Product ID : SAS9211-8i BIOS Version : N/A UEFI BSD Version : N/A FCODE Version : N/A Board Name : SAS9211-8i Board Assembly : N/A Board Tracer Number : N/A Finished Processing Commands Successfully. Exiting SAS2Flash. How I did it, just to make sure I did something wrong. Booted in FreeOS and issued megarec -writesbr 0 SBR-A21.bin While your custom SBR didn't work at all, well at least we know the flashing of different SBR works, the A21.SBR shows the same results. Also no drives recognized. Pulled an unRAID syslog with the A21.SBR. It seems the controller doesn't reset when querried by the OS. To rule out general issues I plugged an H200 controller and ran unRAID. It detects drives as expected. syslog 1 Quote Link to comment
FingerlessGloves Posted October 22, 2017 Share Posted October 22, 2017 (edited) When the PC boots do you see the drives when the raid card starts? I've seen these D2607 cards have issues on the OS level where they won't start. Add the following to your Linux kernel boot parameters pci=realloc=off Looks from the kernel Version your running UNRaid 6.1, I would update your install to the latest 6.3.3. to get the Latest Kernel and Driver for the SAS cards. You can find more info here on the bug in the Linux Kernel, it may be on the Ubuntu bugs report but Linux Kernel is the issue what causes this - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1363313 Edited October 23, 2017 by Jonny Quote Link to comment
bigsing Posted October 24, 2017 Share Posted October 24, 2017 I recently purchased a LSI 9201-16i off eBay. I plugged the card into my test bench last night and verified that each SAS port recognizes 4 drives when using a reverse breakout cable. The drives i tested with included 2x250GB, a 1TB HDD, and 1 SSD. I was going to run a parity check as well, but is there any other testing I should do before I consider moving this card from my test bench into my main server? Also the cards BIOS version is 8.00.00.00 would I gain anything from updating this? Quote Link to comment
uldise Posted October 25, 2017 Share Posted October 25, 2017 9 hours ago, bigsing said: reverse breakout cable Reverse breakout? i think you have Forward breakout cables.. Quote Link to comment
bigsing Posted October 26, 2017 Share Posted October 26, 2017 Reverse breakout? i think you have Forward breakout cables..Yep, I meant forward breakout cables.Sent from my Pixel using Tapatalk Quote Link to comment
Fireball3 Posted October 26, 2017 Share Posted October 26, 2017 On 22.10.2017 at 11:57 PM, Jonny said: When the PC boots do you see the drives when the raid card starts? I've seen these D2607 cards have issues on the OS level where they won't start. Add the following to your Linux kernel boot parameters pci=realloc=off Looks from the kernel Version your running UNRaid 6.1, I would update your install to the latest 6.3.3. to get the Latest Kernel and Driver for the SAS cards. You can find more info here on the bug in the Linux Kernel, it may be on the Ubuntu bugs report but Linux Kernel is the issue what causes this - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1363313 Lets go ahead with this story. I was busy the last few days so I hadn't time to continue fiddling. But now I did some extensive testing. The reason why I'm staying on 6.1 is because I know there is a working console-based preclear script. I use that to preclear my drives. For testing purposes though, I updated my test bench unRAID to the latest 6.3.5. Unfortunately without success on the matter itself. Added the pci=realloc=off to the go. No success. Next step, I've been going through the whole process once again - from scratch (apart from the card not being on it's original firmware). Then, I ensured to get the cards POST screen. I had a setting in that boards BIOS called "Option ROM messages". It was set to "keep current" (whatever that means). After setting it to "Force BIOS" (the only other option to choose) it shows the Avago BIOS POST screen. On that screen, all drives are shown. Success. But with unRAID loaded, no drives are availabe. So, partial success. Well, at least now I know the card is working. Now, let's plug that thing into another PC. Tadaaaa, Success. That works down to unRAID level. So there must be something in that ASUS M5A99X EVO board that keeps the controller from running nicely. Overall, not so nice but if it's just this board, I don't mind. Another thing that I don't like is the boot delay caused by the controller's BIOS we flashed. So, repeat the procedure and skip the BIOS. Note: When flashing both, firmware and BIOS to the card (7.NSH), I noticed that the "adapter reset" after the firmware is flashed will be successful. The firmware flash alone instead (6.NSH), will result in a reset error. Somewhat confusing...but, meh. OK, so I went back to the ASUS test rig, repeated the whole thing but without the BIOS. Then switched to the other PC again. Tested both fully populated SAS ports subsequently. All fine. The SBR I eventually used was A11. I'm attaching the syslog of the not working PC and the working one. Both running unRAID 6.3.5 Maybe somebody more knowledgeable has an idea how to get the ASUS working, or maybe somebody can use this for troubleshooting when having similar problems. For the moment I will leave the card as it is and see if I can use it in my productive system. After all, it seems the Fujitsu cards are xflashable, but this example proves that they are not that straight forward than the DELL or IBM cards. The flash may have run perfect, but you still don't know if incompatibilities stop you from going productive. I flashed a couple of DELL cards and never had similar problems. syslog_6.3.5_mptsas_reset_failed.log syslog_6.3.5_working_pc.log Quote Link to comment
FingerlessGloves Posted October 26, 2017 Share Posted October 26, 2017 Hey, Its very odd how it doesn't work in the Asus Motherboard. You could try updating the BIOS. Some boards can act differently on the PCI Card side of things. I tried my D2607 on 3 different ASUS boards and they always work AM1, AM3 and a Intel 1156. I do most of my flashing with a AM1 Asus board. Try the bios update and see if that helps. What board you using? Also did you try different slots? Quote Link to comment
Fireball3 Posted October 26, 2017 Share Posted October 26, 2017 It's an ASUS M5A99X EVO. Tried all PCIe slots - forgot to mention that. I need to check for a BIOS Update. Maybe there is something available. Quote Link to comment
FingerlessGloves Posted October 26, 2017 Share Posted October 26, 2017 My AM3 system was a M5A97 so bios update could do the trick. Its not over clocked is it? Quote Link to comment
Fireball3 Posted October 26, 2017 Share Posted October 26, 2017 Updated to the latest BIOS from 2013. Changes were related to USB. Tried the controller with the new BIOS but, no joy. No, nothing overclocked. Quote Link to comment
FingerlessGloves Posted October 29, 2017 Share Posted October 29, 2017 Not sure, whats happening there. Could try restore defaults on Motherboard but I guess it won't do anything. Very odd. Quote Link to comment
opentoe Posted November 15, 2017 Share Posted November 15, 2017 Does unraid have built in TRIM support for SSD drives? I could of sworn I saw TRIM DISABLED in my syslog tonight. 3AM in the morning now, need to get up for work in 4 hours, so I could be a little delusional. Quote Link to comment
Fireball3 Posted November 15, 2017 Share Posted November 15, 2017 iirc, drives need to be on the mainboards controller for TRIM support. @johnnie.black can tell us more. Quote Link to comment
JorgeB Posted November 15, 2017 Share Posted November 15, 2017 59 minutes ago, Fireball3 said: iirc, drives need to be on the mainboards controller for TRIM support. @johnnie.black can tell us more. SAS2008 based controllers only support trim on SSDs with deterministic read after trim / read zero after trim, most SSDs don't support that, SAS2308 and newer support trim on all SSDs. 1 Quote Link to comment
Fireball3 Posted November 15, 2017 Share Posted November 15, 2017 2 hours ago, johnnie.black said: SSDs with deterministic read after trim / read zero after trim I had to google that...never heard of. Interesting read https://drbobtechblog.com/ssd-makes-legal-hold-nightmare-process/ Seems the enterprise grade SSD's do the DZAT https://www.win-raid.com/t1366f25-Nightmare-to-find-a-SSD-with-functional-TRIM-on-LSI-SAS.html Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.