IBM ServerRaid Br10i FRU PCI-e 8x SAS HBA's


Recommended Posts

  • Replies 119
  • Created
  • Last Reply

Top Posters In This Topic

I'll experiment in a pure base flash-drive unRAID 5.0b4 system with the flash program to see what's required. It doesn't look to require anything from the linker perspective, however with that said it might need an option or two enabled in the Linux Kernel.

 

I'll also try to dig out 2 of my 74GB Raptors and setup a RAID-1 volume to see how the system behaves, but that might take more time to investigate and work out.

 

As for the SAS to SATA cables, these cards use the exact same cables as the SuperMicro AOC SATA card. Since I do not have any SAS backplanes in my case (as some NORCO models might), I'm using 0.5 meter 3Ware CBL-SFF8087OCF. It is an internal forward break out cable.

Link to comment

So I was going to buy a SASLP MV8 but I'm very tempted by these, would it be safe to pull the trigger on one now?

 

They have not been vetted nearly as much as the SASLP's and all the reports show that drive temps are not working on them yet.

 

Unless you can deal with those things I would hold of on them for right now.

Link to comment

Yes, the controller supports the full blown smartctl tests. Is there anything specific other than smartctl -A to report only temp?

 

As far as I can tell, hdparm -C and hdparm -Y and hdparm -y function fine. It's just a matter of getting the right options set inside of unRAID to do spindown/spinup. I believe that SK had a set of patches which seemed to do that. If not, perhaps unRAID needs an option on how to control spindown, internally or by using hdparm.

 

hdparm -C /dev/sdg

 

/dev/sdg:

drive state is:  active/idle

root@reaver:/cache/.downloads/extracted# hdparm -Y /dev/sdg

 

/dev/sdg:

issuing sleep command

 

# A clicking noise like the hard drive parking is heard

 

root@reaver:/cache/.downloads/extracted# hdparm -C /dev/sdg

 

/dev/sdg:

drive state is:  standby

 

root@reaver:/cache/.downloads/extracted# smartctl -A /dev/sdg

 

# A clicking noise and a motor spinning noise is heard

 

root@reaver:/cache/.downloads/extracted# hdparm -C /dev/sdg

 

/dev/sdg:

drive state is:  active/idle

 

hdparm -y /dev/sdg

 

/dev/sdg:

issuing standby command

 

# a slight clicking noise is heard

 

root@reaver:/cache/.downloads/extracted# hdparm -C /dev/sdg

 

/dev/sdg:

drive state is:  standby

 

root@reaver:/cache/.downloads/extracted# smartctl -A /dev/sdg

 

# A clicking noise and a motor spinning noise is heard

 

root@reaver:/cache/.downloads/extracted# hdparm -C /dev/sdg

 

/dev/sdg:

drive state is:  active/idle

Link to comment

Please correct me if I am wrong, as I don't really understand all the ins and outs of this problem. But, I have been following this discussion in the various threads.

 

From what I understand SK's patch enabled unRAID to identify the drives connected to the LSI1068E card (i.e. workout serial number, model number etc), such that the drive could be used as an unRAID array drive. However, temperature and spin up/down commands were not working. This was because the unRAID driver was not using the same code as smartctl to communicate with the drives.

 

Tom's comment in the 5.03b thread seems to answer why with 5.04b unRAID correctly identifies drives on the LSI1068E card, but why its not getting the temperature and spin up/down commands.

I'm trying to see here if 'smartctl' command can enable/disable SMART on the drive.  Probably it's going to work and probably what's happening is a different interface is being used by smartctl than what the unraid driver is using.  I thought eventually this might happen - there's code in the driver to turn on SMART because at the time it was written (early days of SATA support in linux via libata) the utilities were not updated to work completely with SATA, so I added the code directly in the driver to handle this.  Today, of course utilities work fine, and so has code in driver (let sleeping dogs lie), but apparently the issue must be addressed now to support some of these SAS controllers...

 

 

Link to comment

That's the story how I gather it. However, unRAID 5.0b4 natively supports identifying and adding the drives to the array, so there's no need for patches for that. The 5.0b4 just doesn't as of yet support temps or spindown/spinup.

 

Unfortunately drive temps is internal to the emhttp module, so only Limetech can modify that.

 

I haven't looked that closely at SK's final patches to see if they enable spinup/spindown on the LSI1068E cards directly or if it's only within a VM instance. I'm hoping Limetech is able to get things working before I have time to get into this hacking.  ;)

Link to comment

That's the story how I gather it. However, unRAID 5.0b4 natively supports identifying and adding the drives to the array, so there's no need for patches for that. The 5.0b4 just doesn't as of yet support temps or spindown/spinup.

 

Unfortunately drive temps is internal to the emhttp module, so only Limetech can modify that.

 

I haven't looked that closely at SK's final patches to see if they enable spinup/spindown on the LSI1068E cards directly or if it's only within a VM instance. I'm hoping Limetech is able to get things working before I have time to get into this hacking.  ;)

 

If you can get at the temperature through smartctl, and you can get the spin status of the drive through hdparm, then Tom has what he needs to make the emhttp do its thing with this controller.  I expect he will make this work if you keep reporting it in the beta forums. 

 

Seriously, anyone who wants something to be supported and it isn't working, these beta cycles are the time to be vocal.  Become a beta tester and start testing, posting syslogs, running commands that Tom requests, and generally being the most helpful PITA you can be!

 

;)

Link to comment

I'll experiment in a pure base flash-drive unRAID 5.0b4 system with the flash program to see what's required. It doesn't look to require anything from the linker perspective, however with that said it might need an option or two enabled in the Linux Kernel.

 

I'll also try to dig out 2 of my 74GB Raptors and setup a RAID-1 volume to see how the system behaves, but that might take more time to investigate and work out.

 

As for the SAS to SATA cables, these cards use the exact same cables as the SuperMicro AOC SATA card. Since I do not have any SAS backplanes in my case (as some NORCO models might), I'm using 0.5 meter 3Ware CBL-SFF8087OCF. It is an internal forward break out cable.

 

BRiT - Any update on this.  Particularly interested in the first part on flashing.  Can I flash with 4.7?  Or 5.04?  Or need to do something more exotic?

 

Thanks!

Link to comment

I tried updating in a base unRAID 5.0b4 system and ran into an issue early on. Even though the drives show up and the adapter is clearly enabled and working, it doesn't detect it.

 

My next attempt will be trying from a Slackware LiveCD.

 

./lsiIRupdate ctlr-info.txt

 

**************************************************

LSI IR Update Utility for use with IBM hardware

      Version 2.02 - Release Date 1/21/10

**************************************************

 

This update is for the following controllers:

- ServeRAID-BR10i(B3) in any System x server

- ServeRAID-BR10i(C0) in any System x server

 

sh: dmidecode: command not found

The platform ID for the host system is

 

/bin/mknod: `/dev/mptctl': File exists

There is/are 0 controller(s) in the system.

No controllers were found for update

Link to comment

If you already have a DOS environment, then you merely need the IBMBR10i.zip files to execute. The zip file contains the DOS-based sasflash program, firmware 1.30.05, and mptsas bios 6.30.00. Unfortunately it was too large to attach to this message, so here's  (HotFile Host of IBMBR10i.zip).

 

To see details on your IBM BR10i adapter:

cd C:\IBMBR10I
sasflash -list

 

To flash your IBM BR10i adapter to Firmware 1.30.05 and MPT Bios 6.30.00:

cd C:\IBMBR10I
sasflash -f 013005B3.fw -b 63000mpt.rom

 

I renamed the firmware and the bios to be 8.3 dos friendly, but they were taken from the

 

To create the FreeDOS USB boot device I downloaded this bootstick file then followed these directions which are very similar to unRAID USB Flash prep.

 

Make sure you're working as administator and assume X: is the usb sticks drive letter

  • Format your USB flash drive using FAT32
  • From a Administrator Dos Prompt invoke syslinux\syslinux.exe -fma X:
  • copy the files underneath "usb-root\" to the root of "X:"

 

I copied the IBMBR10I directory to the flash device.

 

On the system with the SAS cards, I then opted to boot from this new USB flash device. On my MSI H55 motherboard I hit F11, then selected the USB Flash device once the menu came up. When prompted at the "boot:" prompt, simply hit enter. When prompted for the date and time, simply hit enter. You are now at the freeDOS prompt.

 

 

Link to comment

The IR software defaults to JBOD anyways and it's functioning fine for me. More importantly, I could NOT find the IT mode firmware that was as recent as the IR firmware. If you can locate a newer version than 1.30.05 / 6.30.00 IT for "B3" cards then let me know.

 

Also, if you want to be able to flash the LSI1068E or similar cards in Linux, you definitely need the Kernel config option "Fusion MPT misc device (ioctl) driver" (CONFIG_FUSION_CTL) set as a module. This creates the /dev/mptctl device.

Link to comment

Alright, so I felt really adventurous tonight, so I'm trying out the newest firmware I could find. I even flashed it with "IT" mode.  :o

 

The firmware version is 01.32.00.00 (December 8, 2010). The BIOS version is 06.34.00.00 (December 7, 2010).

 

I downloaded from the general LSI SAS3081E support page this firmware package SAS3081ER P20 Supports SAS_1-5G_3G_6G and SATA_1-5G_3G_HDD. As the name indicates it only supports SATA 1.5G/S and 3G/S drives. There is another firmware package that supports only SATA 3G/S and 6G/S drives, but I opted not to use it since it does not support/negotiate with 1.5G/S drives.

 

Both firmware packages contain a dos-based flash utility (sasflash.exe). In order to flash the IBM BR10i with this firmware, you have to force it since the initial vendor id does not match. The sasflash option to force an update is -o and must be supplied with the -f and -b options. If asked if I wanted to continue a couple of times and responding with a "y" both times it then proceeded with the firmware update.

 

sasflash.exe -o -f 3081ETB3.fw -b mptsas.rom

 

If you want to remain with the IR firmware use this command:

 

sasflash.exe -o -f 3081ERB3.fw -b mptsas.rom

 

After this was done, I powered cycled the server and upon reboot, unraid detected the drives fine.

 

Most of the info was gathered from ServeTheHome (Article 1, Article 2, Forum Thread), but I did not see any clear indications the 3081-based firmware would work on the IBM BR10i since it is a 3082-based card.

 

I only suggest doing the update if you feel equally as adventurous.  ;D

Link to comment

For reference, here is what the Linux flash shows under a listing:

 

root@reaver:/flash/LSI1068E# lsiflash/sasflash -list

****************************************************************************
    LSI Corporation SAS FLASH Utility.

    SASFlash Version 1.28.00.00 (2010.12.08)

    Copyright (c) 2006-2010 LSI Corporation. All rights reserved.
****************************************************************************

        Adapter Selected is a LSI SAS 1068E(B3):

Controller Number:              1
Controller:                     1068E(B3)
PCI Address:                    00:01:00:00
SAS Address:                    500605b-0-014b-c050
NVDATA Version (Default):       0x2d03
NVDATA Version (Persistent):    0x2d03
Product ID:                     0x2204
Firmware Version:               01.32.00.00
NVDATA Vendor:                  LSILogic
NVDATA Product ID:              SAS3081E
BIOS Version:                   06.34.00.00
BIOS Alternate Version:         N/A
EFI BSD Version:                03.16.00.05
FCODE Version:                  N/A

Finished Processing Commands Successfully.
        Exiting SASFlash.

 

Link to comment

Thanks so much BRiT!!

 

Unfortunately my Br10i shipping was delayed by a day - will arrive Monday.  (Snow in Menphis, can you believe it!)

 

Anyway, with your instructions I'm ready when it comes.

 

What's the difference between the IT and IR firmware?  Is IT stricly JBOD?  Should I stick with the IR firmware to get a RAID-1 cache disk + 6 independent disks?  

 

See THIS POST.  Am hoping that this board will work like the ARC-1200.

 

I am really not that interested in the RAID-1 for true cache purposes - I am able to copy directly to the array most of the time anyway, and don't bother to use cache.  But a RAID-1 disk, independent from the array, would be useful for storaging databases, local documents, and other dynamic files that I want instantly protected and at the best performance.  RAID-1 replication provides even stronger protection than the array for permanent storage of my most important documents.

Link to comment

See post HERE.

 

As I explain, I have created a 2 disk RAID-1 array and trying to get it to spin down and report temperature without success.  The syslog indicates that it has detected the 2 physical disks that make up the array.  I wonder if Linus is providing some access to the underlying disks that I could use for spindowns and smart reports.

 

Any ideas?

Link to comment

I haven't had a chance to play around with RAID1 setup on the cards yet, but I did stumble upon this "LSIUtil". Here's the source package but it also includes an executable so no compiling was needed at least on my system. [ ftp://ftp.lsil.com/HostAdapterDrivers/linux/lsiutil/lsiutil.tar.gz ]

 

Just be careful with what you opt for it to do. There are a lot of options available and seems to offer full control over the card.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.