Finally... Unraid on ESXi (AMD Build)


Recommended Posts

Another AMD Esxi 5 build attempt,

 

Just received a new Gigabyte 990FXA-UD5 board.  Have been testing out with esxi 5.1 and unraid 5.0rc8a.

Based on info in the forums have gotten all drives recognized via raw device mapping but with no spindown or temperature readings.

 

Been doing ALOT of reading to the point of saturation but there seems to be comparatively little documentation of amd systems

My questions:

Is this the norm using RDM?

Can any of the mobo onboard sata ports be utilized for normal unraid operation or is it necessary to use an addon card like the M1015?

Can any of the mobo devices recognized by esxi be passed through?

 

thanks

 

Link to comment
  • Replies 112
  • Created
  • Last Reply

Top Posters In This Topic

Nope, that is not the norm, nor is it ideal.

 

You need to buy a mobo and CPU that both support IOMMU.  There aren't many consumer boards that do.

 

Once you get one that does, you simply passthrough a hardware controller (such as the M1015) to the Unraid VM.

 

So, your first step would be to see if the current CPU and mobo you have support IOMMU.  If they do, enable it in the BIOS.  Then in vSphere, go to Configuration->Advanced Settings and see if it will allow you to passthrough hardware.  If it won't, it will say advanced settings not supported, or something along those lines. 

 

You can also check on the Summary page.  Look at the bottom of the General settings... it has an option for "DirectPath I/O:".  That will need to say "Supported" in order to pass hardware through.

 

Hope this helps.

 

Brandon

Link to comment

I have built an amd based esxi server, with passthrough working!

 

Amd A10-5800k

MSI FM2-A85XA-G65

 

There are no options in the bios to enable but luckily it worked without any problems.

 

The onboard nic does not work with esxi however i have a spare broadcom card which works fine.

Link to comment
  • 2 weeks later...

Am I correct that I have to have the IBM card for virtualization to work?  Do the SATA ports on the motherboard not work for this?  I'd like to have about 10 drives on my unRAID server build virtualized, but I can get by with only having 4 or 5 right now.  If I have to add an IBM card for every two hard drives wouldn't that eat away at my savings of using AMD hardware?

 

 

Link to comment

I'd just like to say thanks to OrangePeel for starting this thread as well as the other folks on the forum for posting their builds and experience.  Through this post (and others) I built an AMD ESXi machine using the info provided here.  It went live over the weekend.  AMD FX 6100 processor, ASRock 970 Extreme 4, M1015 all stuffed into an iStar Rackount case.  The only hiccup was flashing the M1015 card where I had to use another MB as the ASRock gave me a PAL error. 

Link to comment

mmalat,

 

Can you tell me for sure if the a M1015 card is required?  I have purchased but not opened a Sabertooth 990FX R2, 16GB of memory & a FX 8120 hoping to be able to use the 8 SATA ports on the motherboard instead of the M1015.  According to my research it will run the latest ESXi but am wondering it I will be able to run unRAID.

Link to comment

Yes, you need a card to be able to passthrough control of the harddrives to Unraid.  It doesn't have to be an M1015, but the M1015 is a great option as it can be used with an expander to do 24 HDDs and is fairly readily available.  There are certainly others, though.

 

You cannot passthrough the HDD controller of your mobo to Unraid as  you would then have nothing for datastores in ESXi.

 

Its possible to pass the hdds through to unraid, but you won't get smart data and it can't spin them down.  Google it.

 

Brandon

Link to comment

I'd just like to say thanks to OrangePeel for starting this thread as well as the other folks on the forum for posting their builds and experience.  Through this post (and others) I built an AMD ESXi machine using the info provided here.  It went live over the weekend.  AMD FX 6100 processor, ASRock 970 Extreme 4, M1015 all stuffed into an iStar Rackount case.  The only hiccup was flashing the M1015 card where I had to use another MB as the ASRock gave me a PAL error.

 

Awesome!  Glad you got your system up and running and glad my post helped.  Its interesting you can't flash the m1015 with a 970 extreme.  I used a different spare mobo to flash mine at work.  :)

 

Brandon

Link to comment

@140.6  I decided to use the M1015 as a dedicated pass through controller for the UnRaid Guest.  Is it a requirement, I would not think so as RDM'ing would work for any drives connected to the on-board SATA Controller.  I went with the M1015 so I could have the onboard connections avail for other VM's. 

Link to comment

OK, so i cheated here so don't murder me over it as I'm  still posting to this thread. I returned my FX8120 and Sabertooth 99FX R2 last night for an i5 3470 and an ASRock Extreme 6.  The prices ended up being within a few dollars ($99/$189=$288 vs $149/$144=293) so I chose Intel as others including OrangePeel have mentioned if the prices were the same they would go this route.  I'm getting on Ebay tonight to buy the IBM M1015 to hopefully allow me to finish my system as I have been able to get ESXi 5.1 installed (with on board ethernet support by making a few tweaks)and have been able to start up a few VM's just to play around.  My main questions from here are how many drives does one M1015 support.  My initial research tells me that it supports 8 drives.  Is this correct?  I only see two ports on the card so I'm guessing that maybe I can get two cables with 4 breakouts each?  If so where do I find these cables?  Lastly, is it going to be possible to run unRAID off of my SSD if it's not connected to the M1015?  I'd like to use as many of my drives as possible for storage without having to buy another card or adapter.

 

I really appreciate all the advice you guys have given me and I just hope that because I went Intel you won't hold a grudge.

Link to comment

That mobo and proc should work really well.  Just be sure to enable VT-D on the mobo.

 

See the first post of this thread for the correct cables for the M1015.  I put a link there.

 

The M1015 supports 8 drives out of the box at SATAIII speeds.  However, it can support up to 24 drives with a SAS expander.  This does divide up the bandwidth available to each drive (from 6Gb/s to 2Gb/s), BUT its still higher than the HDDs can actually output, so its not really a big deal at all.  You'll still get full speeds.

 

Brandon

Link to comment

Thanks for the advice, so if I use up 8 drives with unRAID will I be able to have the OS installed on my SSD?  Sorry for all of the beginner questions but this is my first crack with ESXi.  My SSD is currently connected to a sata port on my MB and I'd like to keep it there if possible.

Link to comment

Yes and no.  You are still going to have to boot Unraid from a USB drive passed through to the VM, but you're going to need to install a bootloader onto a datastore.

 

Your SSD can and should stay on your motherboard to be a datastore.  That M1015 is ONLY going to be available to the Unraid VM once you pass it through.

 

See JohnM's guide for installing Unraid on ESXi, if you've not already.

 

Brandon

Link to comment

Thanks for the advice, so if I use up 8 drives with unRAID will I be able to have the OS installed on my SSD?  Sorry for all of the beginner questions but this is my first crack with ESXi.  My SSD is currently connected to a sata port on my MB and I'd like to keep it there if possible.

 

Technically,  you can install the unRAID boot from a small virtual drive that is on your SDD if it is a datastore.. BUT.. you still need your unRAID usb flash inserted in the server and passed-through to the unRAID for the license key (plus for other things the flash does like log files).

This is a good alternative to PLOP booting.. the usb boot in ESXi is pretty darn slow.. ~yawn~...

 

 

 

Link to comment
  • 1 month later...

I'm interested in doing this for myself..

 

I have a spare processor (1090T) to use, so all I need is a motherboard.

 

I hope someone can answer some of my question..

 

  • is ASRock 970 EXTREME4 really best one out there to do this? Do you guys recommend any other boards? I have Asus M4A88TD-V EVO/USB3.. I wonder if this will work.. but then kinda dont trust this board. Used to have random freeze once in awhile lol
  • Is there any guide on how to get this to work? I have no idea how to set this up...
  • You guys talk about M1015.. I have AOC-SASLP-MV8. Would this work?

 

Please let me know!

 

 

 

Also,

Link to comment

Any motherboard that supports IOMMU will work.  The ASRock extreme4 has been proven to do so, so many people just keep using it because it works well and has plenty of PCI-E.  :)

 

Check out JohnM's Atlas build for more info on setting Unraid up in VM Ware... that's where I got my info.

 

Yes, the MV8 would work fine.

 

HTH!

 

Brandon

Link to comment
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.