Jump to content
OrangePeel

Finally... Unraid on ESXi (AMD Build)

113 posts in this topic Last Reply

Recommended Posts

Hi all,

 

I'm finally getting around to doing an Unraid machine.  I've been waiting to do Unraid until I decided on either virtualizing it or buying new server hardware that would do hardware passthrough.  That time has finally come.

 

I've still got research to do and things to figure out, but my preliminary build is outlined below.

 

OS at time of building:  5rc5, unless something new comes out next week.  :)

 

CPU:  AMD FX-6100  I got a great deal on one of these on eBay, otherwise I might have went for something with a bit more horsepower.

 

Motherboard: MSI 970A G46 ASRock 970 extreme4

 

RAM: 2x8GB Corsair XMS3 and 2x4GB Corsair XMS3 for 24GB total

 

Case: Antec Three Hundred

 

Drive Cage(s): None

 

Power Supply: Antec Earthwatts 430w

 

SATA Expansion Card(s):  IBM M1015 passed through.

 

Cables: 2x Monoprice 8087 Cables

 

Fans: Stock Antec 140mm, stock Antec 120mm, 3 Yate Loon 120mm low speed fans

 

 

Parity Drive: Seagate 3TB Barracuda

 

Data Drives: 2TB WD Green EARX, 1TB WD Black

 

Cache Drive: None yet.  Only running free version of Unraid until November or so.

 

Total Drive Capacity: 3TB (I'll be expanding in time :) )

 

Primary Use: All around file storage for media, VMs, documents, music, pictures, etc.

 

Likes: I love Unraid in general.  The idea of data protection that you can grow and if more than one harddrive dies, you only lose part of your data is just awesome.  Nothing can come close to that, and after ripping my DVDs and BluRays, I can't imagine doing anything less.

 

Dislikes: I'm not fond of paying for it, but I'm glad to, anyway.  :)

 

Add Ons Used: Won't be using any add-ons most likely.  I will have VMs to do other work for me.

 

Future Plans:  I will be expanding the data for sure, as I rip more of my movies.  I'm still working on the logistics of it.  A buddy of mine and I are also storing data at each others' houses using Cubby.  Right now they're just HD's attached to a VM, but we're soon going to be adding them to Unraid and only sharing them out to the VMs for parity protection.  We are planning on doing this in November.  Also, I'll be adding SSDs for datastores before too long to really kick up my VMs. 

 

No photos or power info just yet.  The hardware I do have is at work, and I'm not sure if I'll be getting a Kill a Watt or not just yet.  I will add pics as soon as I'm able to take them.  I'll be traveling a lot soon, but I'm hoping to work on this machine and maybe even get it up and running on Wednesday or Thursday.  I'll update then!

 

EDIT:  Don't be afraid of doing ESXi on AMD hardware!  Its cheaper and runs great for a home machine.  Yes, I'd rather have Intel and Supermicro, simply because of IPMI and efficiency, but this AMD machine will work fine for me for probably at least several years and only cost about $360 for the CPU, mobo, and 32GB of memory.  Not bad at all and works great.

 

Brandon

Share this post


Link to post

Hi Brandon,

 

at last someone who's finally decided on running ESXi on an AMD platform  ;D I've been waiting ages for someone to try this as everyone else seems to have gone the Intel Xeon/Sumpermicro route.

 

Post some pics of your setup when you get back  8)

Share this post


Link to post

I know!  I'm not 100% sure I've seen anyone do an AMD (FX series, anyway) ESXi/Unraid setup.  Its just A LOT cheaper for me since I already have the correct RAM.  The Xeons are much more efficient chips and the Supermicro boards are just awesome (I have Supermicro chassis I use for testing), but I can't justify spending around $300 to $400 more for one.  The WAF (wife acceptance factor) just isn't there.  :)

 

I will definitely post some pics up.  My HDDs and processor are in the mail.  8)

 

Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform.  :)

 

Brandon

Share this post


Link to post

Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform.  :)

 

Brandon

 

We bought some Cisco units to test. We ran them for about a year. We ultimately decided to stay with the HP AMD platform.

Share this post


Link to post

Nice to see an AMD build.. Nice amount of ram off the bat..

 

Yes I prefer intel these days... but i love the guys that say "stuff it" and do things the way they want..

and for less is always a bonus..

Share this post


Link to post

I've been thinking of doing a AMD ESXi build myself. I bounce back and forth between AMD and Intel depending on what I'm building/working on. Intel right now has a great product for a powerhouse that doesn't break the bank with power usage. I also prefer Intel for video encoding and editing with their iGPU's. They are great for computer animation as well.

 

If I am going for a budget home build for family members or friends, I usually lean more towards AMD. Their A-series are great and versatile.

 

At the end of the day, a lot of standards are based off of Intel, so you'll usually get the best implementation from them.

 

All that being said, only reason I'm thinking of a AMD ESXi build is I have the hardware here for that. If I were to purchase parts for a new build I'd probably go with Intel.

Share this post


Link to post

Johnm, I definitely prefer Intel, too, but the ROI on the increased power consumption is probably 15 years... not worth it at all.  :D  I'd rather have a Xeon, but this will work fine for my purposes.  I'll be encoding video while I'm at work and asleep, anyway, so an extra hour or two (or less) just doesn't matter.  :)  At idle, most modern processors idle down to about the same levels, anyway.

 

Influencer, that is the same reason I went with AMD... I already have the RAM in my current computer, and its standard DDR3.  Not only do I already have it, its half the price of ECC unbuffered.  If it wasn't for the memory thing, I probably would have sprung for a Xeon/Supermicro build, but I just can't justify it.

 

I'm at the airport now headed back home.  I'm not sure that I'm going to be able to work on my server tonight or not, but I'm hoping to.  I've only been gone since Sunday night, but my wife may not allow me to work on projects tonight... lol  I will try to remember to take pics.

 

Brandon

 

 

Share this post


Link to post

Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware.  >:(

 

I'll return this one to Amazon and pickup an Asrock 970 Extreme4... it has more PCI-e slots, anyway.  :D

 

To be continued...

 

Brandon

Share this post


Link to post

Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware.  >:(

 

This is the reason I stick to Intel based platform. There is not enough information about IOMMU usage in consumer motherboard.

Share this post


Link to post

Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware.  >:(

 

This is the reason I stick to Intel based platform. There is not enough information about IOMMU usage in consumer motherboard.

 

Yep... but I knew this would be challenging going in, I just need to save the money for now.  :)  I also could have saved myself some headache by doing better research, but its all good.  I'll get it eventually.  :)

 

Brandon

Share this post


Link to post

I've been looking into building an ESXi system based on an AMD processor ...

 

so I'm really curious how things are working for you ...

 

I keep on looking into the supermicro motherboards ...

but I really try to avoid intel processors

 

 

These are the AMD motherboards I have been trying to research and that list iommu in BIOS

 

ASRock 990FX EXTREME4 Motherboard ATX AM3+ DDR3 3PCI-E16 2PCI-E1 2PCI SATA3 USB3.0 SLI CrossFireX

http://www.ncix.com/products/?sku=65339&vpn=990FX%20EXTREME4&manufacture=ASRock

 

ASUS Crosshair V Formula AM3+ DDR3 ATX AMD 990FX 4PCI-E16 1PCI CrossFireX 6 SATA 6GB/S Motherboard

http://www.ncix.com/products/?sku=61991&vpn=CROSSHAIR%20V%20FORMULA&manufacture=ASUS

 

ASUS Sabertooth 990FX AM3+ DDR3 4PCI-E16 1PCI-E1 1PCI SATA3 SLI CrossFireX USB3.0 GBLAN Motherboard

http://www.ncix.com/products/?sku=61765&vpn=SABERTOOTH%20990FX&manufacture=ASUS&promoid=1360

 

Gigabyte 990FXA-UD5 AMD990FX ATX AM3+ DDR3 5PCI-E16 1PCI-E1 1PCI SLI SATA3 USB3.0 Motherboard

http://www.ncix.com/products/?sku=61546&vpn=GA-990FXA-UD5&manufacture=Gigabyte&promoid=1360

 

Gigabyte 990FXA-UD7 AMD990FX ATX AM3+ DDR3 6PCI-E16 1PCI CrossFireX SLI SATA3 USB3.0 Motherboard

http://www.ncix.com/products/?sku=61755&vpn=GA-990FXA-UD7&manufacture=Gigabyte&promoid=1360

 

MSI 990FXA-GD80 ATX AM3+ 990FX DDR3 4PCI-E16 2PCI-E1 1PCI SLI CrossFire SATA3 USB3.0 Motherboard

http://www.ncix.com/products/?sku=61886&vpn=990FXA-GD80&manufacture=MSI%2FMicroStar&promoid=1360

 

 

I read on one website that someone got the Sabretooth working with ESXi ... but it's hard to tell if they did anymore that boot it up ... and it didn't crash.

 

Video Card/PCI Passthrough seems to work most easily with AMD Graphics cards

 

one person said "In case of AMD i found out that SABERTOOTH 990FX got IOMMU fixed, but the support for the rest of the boards are still questionable..."

 

I'm really interested if you get this working ... I'm debating waiting for Octobers processors releases ... or starting now ...

 

I'm torn between starting with the ASUS Sabertooth 990FX, the Gigabyte 990FXA-UD7 or the MSI 990FXA-GD80

 

 

Bobby

 

here's a link to a Sabretooth system running on Zen ... although I'm assuming to get the video cards working (if it's real) he would have had to get passthrough working ...

 

 

Share this post


Link to post

I know!  I'm not 100% sure I've seen anyone do an AMD (FX series, anyway) ESXi/Unraid setup.  Its just A LOT cheaper for me since I already have the correct RAM.  The Xeons are much more efficient chips and the Supermicro boards are just awesome (I have Supermicro chassis I use for testing), but I can't justify spending around $300 to $400 more for one.  The WAF (wife acceptance factor) just isn't there.  :)

 

I will definitely post some pics up.  My HDDs and processor are in the mail.  8)

 

Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform.  :)

 

Brandon

 

I have not posted my setup but i have unraid running on ESXi with an AMD FX-4100 chip, it has been running solid for ~5 months. I had major issues upgrading to 5.1 that just came out, ended up having to back out and go back with 5.0U1, it works great. The AMD FX series is a very cost effective platform for 4+ cores. Especially if you need PCI pass through. Try finding an Intel Motherboard + CPU that support Intel VT-d for $120 (that is what i paid for my motherboard and processor).  I have an IBM M1015 that i pass through to vmware.

Share this post


Link to post

Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware.  >:(

 

I'll return this one to Amazon and pickup an Asrock 970 Extreme4... it has more PCI-e slots, anyway.  :D

 

To be continued...

 

Brandon

 

That is the same motherboard i have. I picked it up for free at Microcenter when i bought the AMD processor (FX-4100). PCI passthrough works great.

 

Brent

Share this post


Link to post

Nice, Brent.  And my point exactly on the AMD vs Intel pricing. 

 

Glad to know the ASRock extreme4 does pass through... it won't hurt to have 3 PCI-e x16 sized slots too. Gotta pickup a dual NIC in the x4.

 

Brandon

Share this post


Link to post

...very interesting topic...

...but I am a bit confused on what mobo exactly is confirmed to work with passthrough.

 

Is t the "ASROCK 970 Extreme 4" or the "ASROCK 990FX Extreme 4" ?

 

TIA,

    ford

Share this post


Link to post

Nice, Brent.  And my point exactly on the AMD vs Intel pricing. 

 

Glad to know the ASRock extreme4 does pass through... it won't hurt to have 3 PCI-e x16 sized slots too. Gotta pickup a dual NIC in the x4.

 

Brandon

 

I was wrong actually, i have a ASRock Extreme3, not 4. I had to re-check my board after you said yours had 3 PCI-e x16 slots. I wish i had the 4 not as mine only has two x16 slots.

Share this post


Link to post

...very interesting topic...

...but I am a bit confused on what mobo exactly is confirmed to work with passthrough.

 

Is t the "ASROCK 970 Extreme 4" or the "ASROCK 990FX Extreme 4" ?

 

TIA,

    ford

 

I believe its the 970.  I am ordering mine on the 3rd when my eBay bucks are released.  :)

 

Then I will be getting Unraid going!  Can't wait to get more of my media virtualized.

 

Brandon

Share this post


Link to post

Well, great news and bad news.  The great news is the I was easily able to passthrough my M1015 with my new ASRock 970 Extreme4.  The bad news is that I'm probably not going to use Unraid.  I'm trying out FlexRAID right now and, overall, I like it better.  :)

 

I still have testing to do, but thought I'd mention it.

 

However, the ASRock 970 Extreme4 is awesome.  All I did was enable IOMMU, reinstall ESXi 5.1.0, and rebuilt my VMs, which just took a few minutes since I already had the virtual disks.  :)  After enabling the M1015 for passthrough, I rebooted the host and was able to add it to my Server2008 VM when it came back up and everything showed up as it should have, even my 3TB Seagate.  Beautiful!  Now to test FlexRAID and compare it to Unraid and see which I'll be choosing.  :)

 

Brandon

Share this post


Link to post

Thanks for the update, much appreciated!

 

Are you able to put a kill-a-Watt in between and measure idle power consumption (w/o drives)?

I have a major concern that the AMD based build will eat a lot more than the intel based ones.

 

TIA,

          Ford

 

Share this post


Link to post

I don't have a Kill a Watt, but according to Anandtechs review of the 6100, all modern processors pretty much turn themselves almost off at idle.  It won't pull any more than an Intel.  I was concerned about that, too, and loomed some stuff up.  :)

 

Brandon

Share this post


Link to post

I don't have a Kill a Watt, but according to Anandtechs review of the 6100, all modern processors pretty much turn themselves almost off at idle.  It won't pull any more than an Intel.  I was concerned about that, too, and loomed some stuff up.  :)

 

Brandon

 

Several years ago there was a program that gave public libraries in most states (usa) kill-a-watt meters that could be signed out just like a book. May want to check it out. Was surprising how much power things were pulling even when they were turned 'off'.

Microcenter has a combo deal, 6100 on sale for $100, plus $40-50 off a motherboard.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.