OrangePeel Posted September 15, 2012 Share Posted September 15, 2012 Hi all, I'm finally getting around to doing an Unraid machine. I've been waiting to do Unraid until I decided on either virtualizing it or buying new server hardware that would do hardware passthrough. That time has finally come. I've still got research to do and things to figure out, but my preliminary build is outlined below. OS at time of building: 5rc5, unless something new comes out next week. CPU: AMD FX-6100 I got a great deal on one of these on eBay, otherwise I might have went for something with a bit more horsepower. Motherboard: MSI 970A G46 ASRock 970 extreme4 RAM: 2x8GB Corsair XMS3 and 2x4GB Corsair XMS3 for 24GB total Case: Antec Three Hundred Drive Cage(s): None Power Supply: Antec Earthwatts 430w SATA Expansion Card(s): IBM M1015 passed through. Cables: 2x Monoprice 8087 Cables Fans: Stock Antec 140mm, stock Antec 120mm, 3 Yate Loon 120mm low speed fans Parity Drive: Seagate 3TB Barracuda Data Drives: 2TB WD Green EARX, 1TB WD Black Cache Drive: None yet. Only running free version of Unraid until November or so. Total Drive Capacity: 3TB (I'll be expanding in time ) Primary Use: All around file storage for media, VMs, documents, music, pictures, etc. Likes: I love Unraid in general. The idea of data protection that you can grow and if more than one harddrive dies, you only lose part of your data is just awesome. Nothing can come close to that, and after ripping my DVDs and BluRays, I can't imagine doing anything less. Dislikes: I'm not fond of paying for it, but I'm glad to, anyway. Add Ons Used: Won't be using any add-ons most likely. I will have VMs to do other work for me. Future Plans: I will be expanding the data for sure, as I rip more of my movies. I'm still working on the logistics of it. A buddy of mine and I are also storing data at each others' houses using Cubby. Right now they're just HD's attached to a VM, but we're soon going to be adding them to Unraid and only sharing them out to the VMs for parity protection. We are planning on doing this in November. Also, I'll be adding SSDs for datastores before too long to really kick up my VMs. No photos or power info just yet. The hardware I do have is at work, and I'm not sure if I'll be getting a Kill a Watt or not just yet. I will add pics as soon as I'm able to take them. I'll be traveling a lot soon, but I'm hoping to work on this machine and maybe even get it up and running on Wednesday or Thursday. I'll update then! EDIT: Don't be afraid of doing ESXi on AMD hardware! Its cheaper and runs great for a home machine. Yes, I'd rather have Intel and Supermicro, simply because of IPMI and efficiency, but this AMD machine will work fine for me for probably at least several years and only cost about $360 for the CPU, mobo, and 32GB of memory. Not bad at all and works great. Brandon Quote Link to comment
Lerxst Posted September 15, 2012 Share Posted September 15, 2012 Hi Brandon, at last someone who's finally decided on running ESXi on an AMD platform I've been waiting ages for someone to try this as everyone else seems to have gone the Intel Xeon/Sumpermicro route. Post some pics of your setup when you get back Quote Link to comment
StevenD Posted September 15, 2012 Share Posted September 15, 2012 I work for a large corporation with hundreds of ESXi hosts. AMD is our platform of choice. Quote Link to comment
OrangePeel Posted September 15, 2012 Author Share Posted September 15, 2012 I know! I'm not 100% sure I've seen anyone do an AMD (FX series, anyway) ESXi/Unraid setup. Its just A LOT cheaper for me since I already have the correct RAM. The Xeons are much more efficient chips and the Supermicro boards are just awesome (I have Supermicro chassis I use for testing), but I can't justify spending around $300 to $400 more for one. The WAF (wife acceptance factor) just isn't there. I will definitely post some pics up. My HDDs and processor are in the mail. Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform. Brandon Quote Link to comment
StevenD Posted September 15, 2012 Share Posted September 15, 2012 Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform. Brandon We bought some Cisco units to test. We ran them for about a year. We ultimately decided to stay with the HP AMD platform. Quote Link to comment
Johnm Posted September 19, 2012 Share Posted September 19, 2012 Nice to see an AMD build.. Nice amount of ram off the bat.. Yes I prefer intel these days... but i love the guys that say "stuff it" and do things the way they want.. and for less is always a bonus.. Quote Link to comment
Influencer Posted September 19, 2012 Share Posted September 19, 2012 I've been thinking of doing a AMD ESXi build myself. I bounce back and forth between AMD and Intel depending on what I'm building/working on. Intel right now has a great product for a powerhouse that doesn't break the bank with power usage. I also prefer Intel for video encoding and editing with their iGPU's. They are great for computer animation as well. If I am going for a budget home build for family members or friends, I usually lean more towards AMD. Their A-series are great and versatile. At the end of the day, a lot of standards are based off of Intel, so you'll usually get the best implementation from them. All that being said, only reason I'm thinking of a AMD ESXi build is I have the hardware here for that. If I were to purchase parts for a new build I'd probably go with Intel. Quote Link to comment
OrangePeel Posted September 19, 2012 Author Share Posted September 19, 2012 Johnm, I definitely prefer Intel, too, but the ROI on the increased power consumption is probably 15 years... not worth it at all. I'd rather have a Xeon, but this will work fine for my purposes. I'll be encoding video while I'm at work and asleep, anyway, so an extra hour or two (or less) just doesn't matter. At idle, most modern processors idle down to about the same levels, anyway. Influencer, that is the same reason I went with AMD... I already have the RAM in my current computer, and its standard DDR3. Not only do I already have it, its half the price of ECC unbuffered. If it wasn't for the memory thing, I probably would have sprung for a Xeon/Supermicro build, but I just can't justify it. I'm at the airport now headed back home. I'm not sure that I'm going to be able to work on my server tonight or not, but I'm hoping to. I've only been gone since Sunday night, but my wife may not allow me to work on projects tonight... lol I will try to remember to take pics. Brandon Quote Link to comment
OrangePeel Posted September 21, 2012 Author Share Posted September 21, 2012 Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware. I'll return this one to Amazon and pickup an Asrock 970 Extreme4... it has more PCI-e slots, anyway. To be continued... Brandon Quote Link to comment
tranm5 Posted September 21, 2012 Share Posted September 21, 2012 Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware. This is the reason I stick to Intel based platform. There is not enough information about IOMMU usage in consumer motherboard. Quote Link to comment
OrangePeel Posted September 21, 2012 Author Share Posted September 21, 2012 Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware. This is the reason I stick to Intel based platform. There is not enough information about IOMMU usage in consumer motherboard. Yep... but I knew this would be challenging going in, I just need to save the money for now. I also could have saved myself some headache by doing better research, but its all good. I'll get it eventually. Brandon Quote Link to comment
perfessor101 Posted September 21, 2012 Share Posted September 21, 2012 I've been looking into building an ESXi system based on an AMD processor ... so I'm really curious how things are working for you ... I keep on looking into the supermicro motherboards ... but I really try to avoid intel processors These are the AMD motherboards I have been trying to research and that list iommu in BIOS ASRock 990FX EXTREME4 Motherboard ATX AM3+ DDR3 3PCI-E16 2PCI-E1 2PCI SATA3 USB3.0 SLI CrossFireX http://www.ncix.com/products/?sku=65339&vpn=990FX%20EXTREME4&manufacture=ASRock ASUS Crosshair V Formula AM3+ DDR3 ATX AMD 990FX 4PCI-E16 1PCI CrossFireX 6 SATA 6GB/S Motherboard http://www.ncix.com/products/?sku=61991&vpn=CROSSHAIR%20V%20FORMULA&manufacture=ASUS ASUS Sabertooth 990FX AM3+ DDR3 4PCI-E16 1PCI-E1 1PCI SATA3 SLI CrossFireX USB3.0 GBLAN Motherboard http://www.ncix.com/products/?sku=61765&vpn=SABERTOOTH%20990FX&manufacture=ASUS&promoid=1360 Gigabyte 990FXA-UD5 AMD990FX ATX AM3+ DDR3 5PCI-E16 1PCI-E1 1PCI SLI SATA3 USB3.0 Motherboard http://www.ncix.com/products/?sku=61546&vpn=GA-990FXA-UD5&manufacture=Gigabyte&promoid=1360 Gigabyte 990FXA-UD7 AMD990FX ATX AM3+ DDR3 6PCI-E16 1PCI CrossFireX SLI SATA3 USB3.0 Motherboard http://www.ncix.com/products/?sku=61755&vpn=GA-990FXA-UD7&manufacture=Gigabyte&promoid=1360 MSI 990FXA-GD80 ATX AM3+ 990FX DDR3 4PCI-E16 2PCI-E1 1PCI SLI CrossFire SATA3 USB3.0 Motherboard http://www.ncix.com/products/?sku=61886&vpn=990FXA-GD80&manufacture=MSI%2FMicroStar&promoid=1360 I read on one website that someone got the Sabretooth working with ESXi ... but it's hard to tell if they did anymore that boot it up ... and it didn't crash. Video Card/PCI Passthrough seems to work most easily with AMD Graphics cards one person said "In case of AMD i found out that SABERTOOTH 990FX got IOMMU fixed, but the support for the rest of the boards are still questionable..." I'm really interested if you get this working ... I'm debating waiting for Octobers processors releases ... or starting now ... I'm torn between starting with the ASUS Sabertooth 990FX, the Gigabyte 990FXA-UD7 or the MSI 990FXA-GD80 Bobby here's a link to a Sabretooth system running on Zen ... although I'm assuming to get the video cards working (if it's real) he would have had to get passthrough working ... Quote Link to comment
brent112 Posted September 21, 2012 Share Posted September 21, 2012 I know! I'm not 100% sure I've seen anyone do an AMD (FX series, anyway) ESXi/Unraid setup. Its just A LOT cheaper for me since I already have the correct RAM. The Xeons are much more efficient chips and the Supermicro boards are just awesome (I have Supermicro chassis I use for testing), but I can't justify spending around $300 to $400 more for one. The WAF (wife acceptance factor) just isn't there. I will definitely post some pics up. My HDDs and processor are in the mail. Steven, we're moving into ESXi from Hyper-V at work, but we're probably going to go with a Cisco Intel platform. Brandon I have not posted my setup but i have unraid running on ESXi with an AMD FX-4100 chip, it has been running solid for ~5 months. I had major issues upgrading to 5.1 that just came out, ended up having to back out and go back with 5.0U1, it works great. The AMD FX series is a very cost effective platform for 4+ cores. Especially if you need PCI pass through. Try finding an Intel Motherboard + CPU that support Intel VT-d for $120 (that is what i paid for my motherboard and processor). I have an IBM M1015 that i pass through to vmware. Quote Link to comment
brent112 Posted September 22, 2012 Share Posted September 22, 2012 Well, turns out this motherboard, despite having IOMMU setup in the BIOS, cannot pass through hardware. I'll return this one to Amazon and pickup an Asrock 970 Extreme4... it has more PCI-e slots, anyway. To be continued... Brandon That is the same motherboard i have. I picked it up for free at Microcenter when i bought the AMD processor (FX-4100). PCI passthrough works great. Brent Quote Link to comment
OrangePeel Posted September 22, 2012 Author Share Posted September 22, 2012 Nice, Brent. And my point exactly on the AMD vs Intel pricing. Glad to know the ASRock extreme4 does pass through... it won't hurt to have 3 PCI-e x16 sized slots too. Gotta pickup a dual NIC in the x4. Brandon Quote Link to comment
Ford Prefect Posted September 22, 2012 Share Posted September 22, 2012 ...very interesting topic... ...but I am a bit confused on what mobo exactly is confirmed to work with passthrough. Is t the "ASROCK 970 Extreme 4" or the "ASROCK 990FX Extreme 4" ? TIA, ford Quote Link to comment
brent112 Posted September 22, 2012 Share Posted September 22, 2012 Nice, Brent. And my point exactly on the AMD vs Intel pricing. Glad to know the ASRock extreme4 does pass through... it won't hurt to have 3 PCI-e x16 sized slots too. Gotta pickup a dual NIC in the x4. Brandon I was wrong actually, i have a ASRock Extreme3, not 4. I had to re-check my board after you said yours had 3 PCI-e x16 slots. I wish i had the 4 not as mine only has two x16 slots. Quote Link to comment
OrangePeel Posted September 27, 2012 Author Share Posted September 27, 2012 ...very interesting topic... ...but I am a bit confused on what mobo exactly is confirmed to work with passthrough. Is t the "ASROCK 970 Extreme 4" or the "ASROCK 990FX Extreme 4" ? TIA, ford I believe its the 970. I am ordering mine on the 3rd when my eBay bucks are released. Then I will be getting Unraid going! Can't wait to get more of my media virtualized. Brandon Quote Link to comment
Ford Prefect Posted October 2, 2012 Share Posted October 2, 2012 Thanks OrangePeel, please keep us posted on your progress ! Quote Link to comment
OrangePeel Posted October 11, 2012 Author Share Posted October 11, 2012 I'm still on the road, and will be next week, too, but MSI finally did reply to me: http://forum-en.msi.com/index.php?topic=161904.0 Brandon Quote Link to comment
ldasilva Posted October 12, 2012 Share Posted October 12, 2012 this should be nice would open up some to newer cheaper hardware which should easily hit 32gbs ram for vms Quote Link to comment
OrangePeel Posted October 12, 2012 Author Share Posted October 12, 2012 Well, great news and bad news. The great news is the I was easily able to passthrough my M1015 with my new ASRock 970 Extreme4. The bad news is that I'm probably not going to use Unraid. I'm trying out FlexRAID right now and, overall, I like it better. I still have testing to do, but thought I'd mention it. However, the ASRock 970 Extreme4 is awesome. All I did was enable IOMMU, reinstall ESXi 5.1.0, and rebuilt my VMs, which just took a few minutes since I already had the virtual disks. After enabling the M1015 for passthrough, I rebooted the host and was able to add it to my Server2008 VM when it came back up and everything showed up as it should have, even my 3TB Seagate. Beautiful! Now to test FlexRAID and compare it to Unraid and see which I'll be choosing. Brandon Quote Link to comment
Ford Prefect Posted October 13, 2012 Share Posted October 13, 2012 Thanks for the update, much appreciated! Are you able to put a kill-a-Watt in between and measure idle power consumption (w/o drives)? I have a major concern that the AMD based build will eat a lot more than the intel based ones. TIA, Ford Quote Link to comment
OrangePeel Posted October 13, 2012 Author Share Posted October 13, 2012 I don't have a Kill a Watt, but according to Anandtechs review of the 6100, all modern processors pretty much turn themselves almost off at idle. It won't pull any more than an Intel. I was concerned about that, too, and loomed some stuff up. Brandon Quote Link to comment
korith Posted October 14, 2012 Share Posted October 14, 2012 I don't have a Kill a Watt, but according to Anandtechs review of the 6100, all modern processors pretty much turn themselves almost off at idle. It won't pull any more than an Intel. I was concerned about that, too, and loomed some stuff up. Brandon Several years ago there was a program that gave public libraries in most states (usa) kill-a-watt meters that could be signed out just like a book. May want to check it out. Was surprising how much power things were pulling even when they were turned 'off'. Microcenter has a combo deal, 6100 on sale for $100, plus $40-50 off a motherboard. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.