BetaQuasi

I know I'm derailing my own thread, but I just set up OpenIndiana/Napp-it in 30 minutes. The guy's PDF instructions were spot on and so far, I really like the look of it. I am going to back up the contents of my ZFS datastore overnight (there's a little under a TB to do), kill my FreeNAS install and migrate the disks to OI/Napp-it I think.

No, everyone else uses passed through controllers (Ok so that's a fairly broad brushstroke, but I'd suggest at least 90% of us do.)

Because FreeNAS was a bit of a pain to set up - was several hours of stuffing around, and I encountered some strange issues (not that I remember much of it now.) It seems with napp-it/openindiana, installation is far more straight forward: http://www.napp-it.org/downloads/index_en.html In fact, I might just test it now for laughs!

FreeNAS at the moment, but I kinda wish I'd gone the napp-it/openindiana route. Thinking of migrating it as another 'project'.

Yeah works fine with passthrough, all my drives spin down properly (except the .vmdk cache drive, but then if that ZFS array spun down, I'd have a lot of other issues haha)

Yeah a .vmdk is probed for status fairly regularly, so I think you will see the same occur for a RDM drive and it not being spun down unfortunately. I think there was a discussion about this a while back.. buggers me if I can find it though!

You can't get spindown with RDM'ed drives - you need to go with the controller passthrough option, something like a M1015 or Supermicro SASLP-MV8. I wouldn't be surprised if you can't get SMART either, as with RDM, you are essentially putting an extra layer between the VM and your drives. Also, while the E1000 adapter certainly works, the VMXNET3 adapter is even better, as it allows (theoretically) 10Gbps traffic internally between VM's. Lastly, the '-flat' file is the actual virtual hard disk itself - the 1kb file is just a configuration file for it. You need both, though you'll only ever see the 1kb file in the datastore when you go looking (as that's what VMware needs to look at to mount the virtual hard disk.)

Sounds like the virtual BIOS is taking a while to POST.. no idea why that would be. Here's a screen recording of my VM booting from the 5.0rc12a .vmdk: http://goo.gl/NV4Ws I wonder if it's something to do with the multiple different types of adapters you have passed through.. or maybe that 'unknown' adapter that I can see. That's the only obvious difference between our VM's.

I'm afraid I have no idea - I don't see anything wrong with your config. That's downright weird!

That's all you should have to do - really no idea why it boots slowly. What type of SCSI controller have you used? Also, what profile have you used to create your VM? (i.e. ubuntu linux, other linux etc.) Maybe post a screenshot similar to this:

No, the boot menu should appear instantly and disappear even quicker. Not sure what you have done wrong - what do you have your primary datastore on? Is it in the same host?

I'm afraid I don't know.. google would suggest it is possible, but there doesn't appear to be any clear information out there. If you try it, it would be at your own risk!

X9SCM IPMI Firmware Update Another easy one - boot off the USB stick and use the following commands: cd SMT_X9_186 DUPDATE -f SMT_X9_186.bin Be patient, it takes a few minutes to update and then restart the BMC.

X9SCM BIOS Updating This one is nice and easy! Boot from the USB stick If you are currently on 1.1c or lower, type "AMI_1.BAT x9scm2.917" and hit enter. Be patient, it takes a few minutes to complete. After it has completed, reboot the PC. After reboot, type "AMI2.BAT" and hit enter. Note the lack of underscore, i.e. do NOT run AMI_2.BAT. AMI_1.BAT tells you this before you reboot in the previous step. If you are currently on 2.0a, type "AMI_2.BAT x9scm2.917" and hit enter. Be patient, it takes a few minutes to complete. After it has completed, reboot the PC. Enter the BIOS by pressing DEL at the POST screen and check all configurations as required. In my case, going from 1.1c to 2.0b disabled Intel VT-d and I had to re-enable it. I also had to re-disable all the PXE booting crap that I don't want and fix the boot order to boot off my ESXi USB stick. DONE!

M1015 Cross Flashing from EFI Shell Some motherboards, including the popular Supermicro X9SCM-F and X9SCM-IIF, require flashing to be done in the built-in EFI shell. All switches and parameters are identical to the dos-based SAS2FLSH.EXE, but instead you need to call SAS2FLASH.EFI (note the 'A' is included in the EFI version.) Follow the first couple of steps from the previous post to prepare the M1015 and then reboot (i.e. the two 'megarec' commands) When booting your PC/Server, choose the 'Boot Menu' option (on a Supermicro X9 board, this is F11.) Select 'EFI Shell' from the list and hit enter Once booted into the EFI shell, we need to access the USB stick. It will be fsx, where x may be 0, 1, 2 depending on how many sticks/drives are attached. Type "fs0:" and hit enter Type 'ls' to see what files are shown. If they match what you expect on your USB stick, then you're good to go. Otherwise, try fs1: etc until you find it Type "Shell_Full.efi" and hit enter. This will start up a more complete UEFI shell Use the same commands as for SAS2FLSH.EXE to flash your card, but substituting in SAS2FLASH.EFI: sas2flash.efi -o -f 2118it.bin -b mptsas2.rom sas2flash.efi -o -sasadd 500605bxxxxxxxxx (replace the xx's with the SAS address from your card) DONE! Cross-flashing Mutiple Adapters When multiple adapters are present, you can either attempt to flash all of them at once (I don't recommend this - grab the LSI SAS2Flash Utility reference guide from their site if you want to try it) or flash them individually by telling the utility which card you want to flash. Some commands to assist with this follow: sas2flash.efi -o -listall (This will show you a list of all your adapters and give them a reference number) sas2flash.efi -o -c x -listsasadd (Replace 'x' with the number of the relevant card gleaned from the previous command - this is another way of getting the SAS address of the card) Once you have the card number, you need to alter the previous commands slightly to select the controller you want to flash. The alteration involves adding in "-c x" where x is the controller number. Note that the sequence of parameters is important as this tells the utility what sequence to run the commands in. As an example, if you were flashing controller #2 with IT firmware, it would look like this: sas2flash.efi -o -c 2 -f 2118it.bin -b mptsas2.rom sas2flash.efi -o -c 2 -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Upgrading firmware Upgrading firmware, for example from P14 to P15, is a simple matter of flashing the BIOS and firmware. No need to reset the SAS address, eg: sas2flash.efi -o -f 2118it.bin -b mptsas2.rom The above will flash the latest (P15) IT firmware from this collection of utilities to your card. No need to do anything else (don't erase the existing firmware/BIOS and don't reset the SAS address).

M1015 Cross Flashing Disclaimer - ALL firmware flashing has the potential to go wrong. You can brick your devices (make them unuseable) if you are not careful and follow the steps EXACTLY. Also note that a power outage will also typically cause your hardware to be bricked, so wherever possible use a UPS! NB: If you try the below procedure on a X9SCM motherboard, it will likely not work and you'll get a "PAL" error. You can still use your X9SCM to flash, but I would recommend updating the X9SCM BIOS to the latest version first prior to doing this. You will need to boot into your motherboard's EFI Shell as per the instructions in the next post. Before doing any of the following (except firmware updates, eg P14 IT to P15 IT), get the SAS address of the card. It's on the back of the card and is a green sticker like the following: M1015 -> 9211-IT Boot from the USB drive you created as per the first post in this thread Enter the following at the command prompt megarec -writesbr 0 sbrempty.bin megarec -cleanflash 0 Reboot your PC (CTRL-ALT-DEL) and boot back onto the USB stick sas2flsh -o -f 2118it.bin -b mptsas2.rom sas2flsh -o -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Reboot your PC (CTRL-ALT-DEL) DONE! M1015 -> 9211-IR Boot from the USB drive you created as per the first post in this thread Enter the following at the command prompt megarec -writesbr 0 sbrempty.bin megarec -cleanflash 0 Reboot your PC (CTRL-ALT-DEL) and boot back onto the USB stick sas2flsh -o -f 2118ir.bin -b mptsas2.rom sas2flsh -o -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Reboot your PC (CTRL-ALT-DEL) DONE! 9211-IT -> IR Boot from the USB drive you created as per the first post in this thread Enter the following at the command prompt megarec -cleanflash 0 Reboot your PC (CTRL-ALT-DEL) and boot back onto the USB stick sas2flsh -o -f 2118ir.bin -b mptsas2.rom sas2flsh -o -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Reboot your PC (CTRL-ALT-DEL) DONE! 9211-IR -> IT Boot from the USB drive you created as per the first post in this thread Enter the following at the command prompt megarec -cleanflash 0 Reboot your PC (CTRL-ALT-DEL) and boot back onto the USB stick sas2flsh -o -f 2118it.bin -b mptsas2.rom sas2flsh -o -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Reboot your PC (CTRL-ALT-DEL) DONE! Cross-flashing Mutiple Adapters When multiple adapters are present, you can either attempt to flash all of them at once (I don't recommend this - grab the LSI SAS2Flash Utility reference guide from their site if you want to try it) or flash them individually by telling the utility which card you want to flash. Some commands to assist with this follow: sas2flsh -o -listall (This will show you a list of all your adapters and give them a reference number) sas2flsh -o -c x -listsasadd (Replace 'x' with the number of the relevant card gleaned from the previous command - this is another way of getting the SAS address of the card) Once you have the card number, you need to alter the previous commands slightly to select the controller you want to flash. The alteration involves adding in "-c x" where x is the controller number. Note that the sequence of parameters is important as this tells the utility what sequence to run the commands in. As an example, if you were flashing controller #2 with IT firmware, it would look like this: sas2flsh -o -c 2 -f 2118it.bin -b mptsas2.rom sas2flsh -o -c 2 -sasadd 500605b0xxxxxxxx (replace the xx's with the SAS address from your card) Upgrading firmware Upgrading firmware, for example from P14 to P15, is a simple matter of flashing the BIOS and firmware. No need to reset the SAS address, eg: sas2flsh -o -f 2118it.bin -b mptsas2.rom The above will flash the latest (P15) IT firmware from this collection of utilities to your card. No need to do anything else (don't erase the existing firmware/BIOS and don't reset the SAS address).

Hi all, This is another one of those things that I see a lot of queries about and there is a lot of information around various posts/forums etc going through this. This is an effort to consolidate all of that into one useful download that you can put onto a bootable USB stick and use immediately. I will keep it up to date with new firmware revisions as they happen. This set of files contains the following: (note that you can just use a subset of the files (you might not have a X9SCM-F motherboard, but you need to flash a M1015 for example. That's fine, you can still make use of this.) LSI 9211 firmware for cross-flashing a M1015 into IR or IT mode P15 (2012-11-15) M1015 original firmware for cross-flashing back (not sure why you ever would but hey!) LSI drivers for most MegaRaid SAS adapters and various OS X9SCM firmware and accompanying files v2.0b (2012-09-17) X9SCM IPMI firmware 01.86 (2012-11-14) NB: X9SCM-iiF users, do NOT use the BIOS files here to upgrade your motherboard. You can replace the ones in the archive with the correct ones from the Supermicro site if you really want to. I don't have a iiF, so I don't intend including those files in the archive. Credit where credit is due: mobilenvidia @ laptopvideo2go.com for his thread on M1015 flashing and files - http://forums.laptopvideo2go.com/topic/29059-sas2008-lsi92409211-firmware-files/ newbie_dude @ these forums for his notes on flashing via UEFI on a X9SCM - http://lime-technology.com/forum/index.php?topic=25891.msg225711#msg225711 madburg @ these forums for his original mammoth thread on LSI SAS cards - http://lime-technology.com/forum/index.php?topic=12767.0 Probably some others that I forgot - thanks to you guys too! I will also update the unRAID wiki with a link to this post. Download here: Date: | LSI/X9/IPMI ver | Download Link 2013-03-29: P15/2.0b/186 https://www.dropbox.com/s/pzqex4p3jgbsdug/M1015_X9SCM_BOOT_P15.zip?dl=0 2013-04-18: P16/2.0b/186 https://www.dropbox.com/s/k5r8l05nkvlmu1o/M1015_X9SCM_BOOT_P16.zip?dl=0 2013-11-01: P17/2.0b/238 https://www.dropbox.com/s/7lfo79fb2gfvvrw/M1015_X9SCM_BOOT_P17.zip?dl=0 Creating a bootable USB stick Download the excellent bootable USB utility 'Rufus' from here: http://rufus.akeo.ie/ Insert a USB stick into your PC (note that the contents WILL be wiped, so back them up if you don't want that to happen) Set up the utility as follows, ensuring that the device selected in the first dropdown is the USB stick you want to use (note you can make the volume label anything you like): http://i.imgur.com/5xbkWkJ.jpg Click Start and wait for your USB stick to be created Unzip the contents of the M1015_X9SCM_BOOT.zip file directly onto the USB stick Follow one of the subsequent posts in this thread depending on what you are wanting to flash

Yes, my .vmdk's will only work in ESXi 5.0/5.1. I've edited the OP to clarify that.

I'm just doing what I'm doing guys, not really interested in making changes/modifications, as there is so many different ways to do this. Also, at 100mb a pop, I'd like to limit the amount of these things that are being hosted as I really don't want to start using a service like mega or 4shared or whatever as they are a pain. Perhaps you'd like to look at implementing/hosting these versions yourselves if you think it's warranted. I think the VM that another user has posted here ticks some of these boxes: http://lime-technology.com/forum/index.php?topic=26634.0

No, the volume label of the (virtual) disk itself - edited to make that clearer.

OP updated with 5.0rc12a

EDIT: I am no longer maintaining this as I have moved to unRAID bare metal/KVM. I will continue to add .vmdk's to this first post if people want to upload/host them. I won't be checking their content however, so please do find the person in the thread that has added it and speak to them if there are any issues. ________________________ I have been intending to do this for a while, since it often comes up as a requirement for various users. I will maintain this thread with each update of unRAID - essentially these are pre-built VMware virtual disks containing what is required to boot unRAID directly, and more quickly than booting entirely from a USB stick. 5.0rc11: https://app.box.com/s/8un6bssqjy8d1z761xputhhu4capgym4 5.0rc12: https://app.box.com/s/uysgq9ij5g3m0823vuio7isrpr4l7wpu 5.0rc12a: https://app.box.com/s/hs5w6ry8nimkt943u5iglobszpv47m90 5.0rc13: https://app.box.com/s/zujn4kj1yc1g5q0plu0tafxk7m38nhh4 5.0rc15: https://app.box.com/s/s82fgzzhp3uw3r2kg89pkjq6ntx27p6z 5.0 final: https://app.box.com/s/in501e2qda18p2k62x3pocnvx1v52698 6.0 b12: https://drive.google.com/open?id=0B3cXNKF8ZrHsR0h3ZVFkVEtyZlk&authuser=0 6.0 b14b: http://goo.gl/T47Mh8 6.0: https://goo.gl/4oeTnG 6.0.1: https://goo.gl/vwg2Yr 6.1.9: https://drive.google.com/open?id=0B4uhfP2afCgiYm5IMXNOTmg1bUk 6.2: https://drive.google.com/open?id=0B4uhfP2afCgieklzUjc3N2VKREE 6.2.1: https://drive.google.com/file/d/0B4uhfP2afCgiRXFENXdxSks3bWc/view?usp=sharing 6.2.2: https://drive.google.com/open?id=0B4uhfP2afCgieFBLTU1uQ24tWW8 6.2.3: https://drive.google.com/open?id=0B4uhfP2afCgiLVhPeDFRNVNKRk0 - 1Gb .vmdk with 100mb partition containing bzroot/bzimage etc - Upgrade/downgrade by mounting the .vmdk in another VM/direct to your PC/via ESXi (ask google) and replacing the bzroot/bzimage with the new version of these files, or simply download the new version from here as they are updated. - If you download the new vmdk from here, simply shut down your unRAID VM, upload the .vmdk over the top of the old one and restart your VM. - Use of these boot .vmdk's is quite simple (I will refine these instructions with more detail, or you can refer to Johnm's Atlas thread in the unRAID Compulsive Design forum: Create a new VM in ESXi, use the Ubuntu 32-bit profile When prompted to add a virtual disk, choose 'Do not create disk'. Upload the .vmdk into your datastore, preferably into the newly-created folder for the VM you just created Edit the new VM and add the disk - choose the .vmdk you just uploaded Map your unRAID USB stick through to the newly-created VM and boot your new VM up NB: Do not change the volume label on the virtual disk to UNRAID - your USB stick should be the only thing with the volume label of UNRAID. The USB stick is still required and holds your unRAID configuration files, all plugin files and your license key. NB2: These .vmdk's will only work in ESXi 5.x I also recommend coupling an ESXi install of unRAID with Zeron's VMware tools plugin, available here: http://lime-technology.com/forum/index.php?topic=11449.0 EDIT 23/03 - Added 5.0rc12 EDIT 24/03 - Added 5.0rc12a EDIT 12/01/15 - Added 6.0 b12 - note I am no longer running ESXi, now running bare metal unRAID with KVM VM's, so I won't be maintaining these any longer. I'll update this post to include vmdk's from others that are up to date however, as per above. EDIT 01/03/15 - Added 6.0 b12 EDIT 29/06/15 - Added 6.0.1 EDIT 03/08/16 - Updated links EDIT 01/12/16 - Updated to include links to 6.1.9 through 6.2.3

Oh sure make the rest of us look like deadbeats Kryspy Subtle encouragement?

Thanks Tom, that's great - will order a couple shortly. I am going to pay $5 each by the way.. I know it's not much, but the rest can go towards assisting your efforts.

I'd jump on this and buy some badges if they were available.