ATLAS My Virtualized unRAID server

[Johnm]

Hey Johnm,

Remember your "lovely girl friend's sock" that you had stuffed in one of the empty drive bays? Right now, on my 4220, I only have 7 drives. Should I too use similar device to maximize airflow over the hard drives by blocking airways in empty bays? I am currently preclearing 2 2Tb drives -- a Hitachi coolspin and a WD EURS drive - and the temp during preclearing was always between 42-44 degree C. Right now, the top of the case is open since I am still waiting on my M1015 cards to arrive over the weekend. I will then close the case, tape all the holes in the sides, etc.

 

LOL.. No. i only did that because I stole a drive for for a few days. with no caddie at all, that would be a major source of air leakage.

 

The 4224's have vents to open or close bays on the font of the caddies for bays without drives.

rumor is the latest model with the yellow backplanes do NOT have this feature. in that case, I am not sure how the cooling will perform.

I would never close more then say 12-16 bays anyways. you need some cool air to get into the motherboard area to cool the other warm bits.

 

If you have the newest version, i would just see how it performs out of the box.

if you need make adjustments, there are many ways to dampen airflow.

 

[Johnm]

So I'm thinking I might order (3) M1015 cards currently instead of using an expander.  I'm not positive I will be adding another unRaid server in the future so I'd like to get the best performance possible and save a little money. If I wanted to add another unRaid guest in the future I could still go this route by using (2) of the M1015 cards that i buy now and buying (2) of the chenbro CK23061 (or similar), this way i only lose one of the M1015 cards (which i could use as a backup).  How hard would it be to change my configuration in my first unRaid to support this new configuration down the road if I have a ton of drives with data in it?

 

Also, will there be any issues with the 4x PCI-E limitation on the 3rd slot on the X9SCM-F-O for the 3rd M1015 card?

 

Thanks for everyone's input so far.

 

3 M1015s will be fine. the one in the 4x slot will be fine with 8 SATA drives on it. the 50% speed hit wont be noticeable. it will perform more like an SASLP-MV8 for speed..

I would not use the 4x port for "expanding" with the M1015  though (unless you have to). the card will be running at 1/2 the possible bandwidth.

 

as far as expanders, running a single chenbro card to add 24 drives on a single port in the second DAS might take quite bit of a performance hit when doing parity.

 

take a look at this thread for some earlier thoughts http://lime-technology.com/forum/index.php?topic=14521

[kman321]

Thanks so much.  can you answer the other part of my question: how hard would it be to re-configure the cards later if i decided to add another unRaid guest.  (i.e. remove one of the M1015 and add (2) expanders using dual-link)?  Doing this would require moving drives to a different controller so I'm not sure if this is possible in unRaid without wiping the data in the array.

 

Also, do you still think the X9SCM-F-O & Xeon 1240 is the way to go for price/performance/compatibility?  Or are there new/better kids on the block?

 

thanks again.

[queeg]

Hmm, I'm not clear on which firmware to flash the LSI SAS9220-8i with.  Can someone link to it or specifiy exactly which/where it is?

 

Yes, I know about the sticky in the HardDrives and Controllers section.  It doesn't mention the SAS9220-8i while it does mention the M1015 (but they all have different chipsets).

 

Thanks.

 

As the LSI SAS9220-8i, you just update to latest from LSI (12 from Nov2011).

 

Do I still need to do the magarec clearing first?  I'm asking that because the sas2flsh.exe -listall finds no adapter in two mobo's so far.  Even though I see the cards bios during boot up and can go into it. 

[c3]

Hmm, I'm not clear on which firmware to flash the LSI SAS9220-8i with.  Can someone link to it or specifiy exactly which/where it is?

 

Yes, I know about the sticky in the HardDrives and Controllers section.  It doesn't mention the SAS9220-8i while it does mention the M1015 (but they all have different chipsets).

 

Thanks.

 

As the LSI SAS9220-8i, you just update to latest from LSI (12 from Nov2011).

 

Do I still need to do the magarec clearing first?  I'm asking that because the sas2flsh.exe -listall finds no adapter in two mobo's so far.  Even though I see the cards bios during boot up and can go into it.

 

You sure it is LSI SAS9220-8i? I got one from NationSurplus labeled like that, but it was really an IBM M1015. Look above port 0 for sticker with FRU number.

[ashaneil]

Ummm I just got mine from Nationwide Surplus - [urlhttp://www.ebay.com/itm/400271458542?ssPageName=STRK:MEWNX:IT&_trksid=p3984.m1439.l2649#ht_1644wt_1396]IBM ServeRAID M1015 SAS RAID Controller FRU 46M0861 LSI PN# SAS9220-8i.[/url]

 

It says FRU 46M0861 above Port0 and SAS9220-8i on the back. On the back, there is also a sticker that says "LS2-SAS9240-8i".

 

C3, if that is what you got, what firmware did you use? Can you share the link please? Thanks

[c3]

 

It says FRU 46M0861 above Port0 and SAS9220-8i on the back. On the back, there is also a sticker that says "LS2-SAS9240-8i".

 

C3, if that is what you got, what firmware did you use? Can you share the link please? Thanks

 

If you watch the BIOS closely, you'll see IBM ServeRAID. Just follow the sticky post in Disk and Controllers for the M1015. It is a multi step process to clear and convert to IT mode. The download file includes a README which covers it nicely.

 

http://lime-technology.com/forum/index.php?topic=12767.msg121131#msg121131

[ashaneil]

 

Please let me know (PM or post here) what Card you have successfully flashed with the LSI MegaRAID to SAS2008 with the provided zip.

 

LSI MegaRAID to SAS2008(P10).zip - 6.38 MB (DOS, via bootable usb key)

LSI MegaRAID to SAS2008(P11).zip - 5.87 MB (DOS, via bootable usb key)

 

 

Ok will do. Last question - Do you know the difference between P10 and P11? Waiting on daughter to finisher her homework before I start the flashing process on her machine.

[c3]

Ok will do. Last question - Do you know the difference between P10 and P11? Waiting on daughter to finisher her homework before I start the flashing process on her machine.

 

About 3-6 months, those are version numbers, 11(Aug2011) replaced 10, and now 12 is out (Nov2011).

[queeg]

Hmm, I'm not clear on which firmware to flash the LSI SAS9220-8i with.  Can someone link to it or specifiy exactly which/where it is?

 

Yes, I know about the sticky in the HardDrives and Controllers section.  It doesn't mention the SAS9220-8i while it does mention the M1015 (but they all have different chipsets).

 

Thanks.

 

As the LSI SAS9220-8i, you just update to latest from LSI (12 from Nov2011).

 

Do I still need to do the magarec clearing first?  I'm asking that because the sas2flsh.exe -listall finds no adapter in two mobo's so far.  Even though I see the cards bios during boot up and can go into it.

 

You sure it is LSI SAS9220-8i? I got one from NationSurplus labeled like that, but it was really an IBM M1015. Look above port 0 for sticker with FRU number.

 

It is a LSI SAS9220-8i rebadged as an M1015.  It was sold as an M1015 from ServerSupply.com

[c3]

It is a LSI SAS9220-8i rebadged as an M1015.  It was sold as an M1015 from ServerSupply.com

 

Wow, I'm so sorry. We just follow the reflash as documented. I have no idea what you will do. Good luck!

[ashaneil]

Queeg,

It if helps, my card looks exactly like yours. I followed the steps as described for the SAS2008 chipset and it worked. However, I had to try it on 2 different machine before it was detected. I tried p11 file. If you go to the original thread, on the last page, I think there was someone else with silimar issue where he had to use a computer at a gas station. So you may want to try another machine if possible.

[msobon]

I've followed your thread and have pretty much got a very similar build to yours:

 

 

• Intel Xeon E3-1240 Sandy Bridge 3
  SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard
  16GB 2x (2 x 4GB) 240-Pin DDR3 SDRAM ECC Unbuffered
  2x AOC-SASLP-MV8
  
• 
  

 

I've already had esxi 4.1 going on another box but decided to consolidate however I've found that I get the purple screen of death on my esxi as soon as unraid decides to start to mount the drives in unraid. I'm using VTM passthrough for the MV8's and I'm quite lost as to what seems to be the problem. Any ideas?

[ashaneil]

I've followed your thread and have pretty much got a very similar build to yours:

 

 

• Intel Xeon E3-1240 Sandy Bridge 3
  SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard
  16GB 2x (2 x 4GB) 240-Pin DDR3 SDRAM ECC Unbuffered
  2x AOC-SASLP-MV8
  
• 
  

 

I've already had esxi 4.1 going on another box but decided to consolidate however I've found that I get the purple screen of death on my esxi as soon as unraid decides to start to mount the drives in unraid. I'm using VTM passthrough for the MV8's and I'm quite lost as to what seems to be the problem. Any ideas?

 

Did you perform the MV8 hack as linked here or on page 1 - VM #3? And I believe that you have to set both cards to false:

 

pciPassthru0.msiEnabled = "FALSE"
pciPassthru1.msiEnabled = "FALSE"

 

I don't have 2 cards in yet but someone else made a post where he had 2 MV8 cards.

[prostuff1]

I have 2 MV8-SASLP cards in my Andromeda build

[msobon]

I've followed your thread and have pretty much got a very similar build to yours:

 

 

• Intel Xeon E3-1240 Sandy Bridge 3
  SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard
  16GB 2x (2 x 4GB) 240-Pin DDR3 SDRAM ECC Unbuffered
  2x AOC-SASLP-MV8
  
• 
  

 

I've already had esxi 4.1 going on another box but decided to consolidate however I've found that I get the purple screen of death on my esxi as soon as unraid decides to start to mount the drives in unraid. I'm using VTM passthrough for the MV8's and I'm quite lost as to what seems to be the problem. Any ideas?

 

Did you perform the MV8 hack as linked here or on page 1 - VM #3? And I believe that you have to set both cards to false:

 

pciPassthru0.msiEnabled = "FALSE"
pciPassthru1.msiEnabled = "FALSE"

 

I don't have 2 cards in yet but someone else made a post where he had 2 MV8 cards.

 

I did do the Hack however i didn't add the second card in the set for the second card, that was key! thank you! I was going crazy!

[queeg]

Queeg,

It if helps, my card looks exactly like yours. I followed the steps as described for the SAS2008 chipset and it worked. However, I had to try it on 2 different machine before it was detected. I tried p11 file. If you go to the original thread, on the last page, I think there was someone else with silimar issue where he had to use a computer at a gas station. So you may want to try another machine if possible.

 

I'll give it a try.  I just wasn't sure whether to follow the SAS2008 chipset steps since I have a SAS9220-8i.  In this situation I felt it better to get explicit confirmation that it has been tried first.  So thanks for that.  I'll report my results.

[c3]

I'll give it a try.  I just wasn't sure whether to follow the SAS2008 chipset steps since I have a SAS9220-8i.  In this situation I felt it better to get explicit confirmation that it has been tried first.  So thanks for that.  I'll report my results.

 

You have an IBM M1015 as indicated by the FRU.

 

http://www.redbooks.ibm.com/abstracts/tips0740.html

 

It has the LSI SAS2008 6 Gbps RAID on Chip (ROC) controller. No mystery here, well documented.

[Shigo_Naito]

Johnm,

First off great Howto, I followed it and my server runs great. 

 

I was wondering in ESXi 5 the X9SCM motherboard has 2 USB controllers you can set as passthrough, have you try setting them up and if they worked what port where on which controller?  I would like to give one of my VMs direct accesses to a USB port but not sure what controller has the ESXi and the UNRAID USB sticks on them.

 

Thanks for the help

[tibonhom]

Hi Guys,

 

I need your help.

 

I just built a system using the Supermicro X9SCM-F-O MB  and the intel E3-1240 CPU.  I installed ESXi 5.0 and I created a windows 7 and an unRAID VM on it and everything went fine.  It is only when I received my SAS controller and I started to look at the “Passthrough” functionality that I realized I had something not working.  In the passthrough screen I have a message saying “host does not support passthrough configuration”.  I saw in Johnm screen shots he could see the NICs in the passthrough screen but I don’t.

 

Verified that my board and CPU supports VT-d.  The virtualization is enabled in the BIOS.  What did I miss?  Is it possible I have a defective MB or CPU?  Should I try to reflash the BIOS?

 

I am out of ideas.  Help or ideas would be much appreciated.

 

I attached some screen shots to show you my setup configs (sorry for the french but for some reason I can't force it to install in english on on french windows)  The options are located at the same place as the english gui.

 

Thanks!

 

tibonhom

[WeeboTech]

There's two parts in the bios from what I remember on my board.

 

Virtualization and then VT-D

I happen to have a X7SBA with both enabled and I can't see any pass through either, but I'm sure my motherboard is old.

Yours is recent, so it should work if both are enabled.

 

[c3]

It also takes a power cycle, not reboot.

[Shigo_Naito]

Your right there are two spots in the BIOS that need to be enabled before you can enable pass through.  It took me a few hours of searching the Internet before I tried looking in the BIOS.

[tibonhom]

There's two parts in the bios from what I remember on my board.

 

Virtualization and then VT-D

I happen to have a X7SBA with both enabled and I can't see any pass through either, but I'm sure my motherboard is old.

Yours is recent, so it should work if both are enabled.

 

Your right there are two spots in the BIOS that need to be enabled before you can enable pass through.  It took me a few hours of searching the Internet before I tried looking in the BIOS.

 

Guys thank you!

 

I don't know why but I reviewed every single options of the BIOS several times and always missed this VT-D option.  I turned it on and the passthrough is now enabled.

 

@Johnm - Do you think it would be a good thing to add a step to enable VT-d in BIOS in your procedure? I would place it with the Enable "Intel Virtualization Technology" step for blind people like myself

 

It is locate in Advenced -> Chiset configuration -> Integrated IO Configuration -> VT-d for those who were in my situation.

 

By the way thanks for this thread.  It gave me the confidence I needed to go ahead with this project.

 

tibonhom

[kaiguy]

Hey Johnm, I just followed your guide for setting up apcupsd and (without testing the shutdown) things appear to be set up correctly.  However, I keep getting COMMLOST errors followed by ONLINE pretty consistently even if I up the polling time.  Considering unRAID and the Windows VM are on the same host, I would assume that communication errors shouldn't be an issue.  Did you ever experience anything like this before? 

 

Thanks!!