Finally... Unraid on ESXi (AMD Build)

[dave_m]

Another recommendation for the Asrock 970 Extreme4, I've got ESXI 5.1 set up and running with that motherboard and an FX6300 with 32GB of DDR3.  Mainly a test/lab box for now, but I'll probably test out unraid on it eventually.

[indorill]

Hi guys, Im a little late to the party but I had a question I was hoping someone had some insight on.

 

I am leaning towards building with a 970 Extreme4 mobo, a fx6300 CPU, and 16 gigs of ram.  Ideally I will install esxi and then setup some VMs.  Eventually, I would like to use unRaid with a m1015 or something and build an array 1 drive at a time.  For the time being though I will probably just setup a SMB share drive that my attached network devices will use.

 

My question is, for all of you guys using the 970 Extreme4 is, what do you do for display output?    I know you CAN setup a headless server, but I am assuming that for at least the initial install you must have had some sort of monitor hooked up to your machine.    Again, I am assuming, but you probably put some sort of cheap GPU onto your mobo since the Extreme4 doesnt even have a vga port or any onboard graphics.

 

Ideally my setup would run, pfesense on a datastore, an unRaid array through the m1015, and then a linux distro for light office work and maybe some video playback.  Has anyone here successfully passed through a GPU to a guest VM on their ESXi build?  I saw one guy here posted that he had pass through working on a a10-58000k setup with ESXi, and Im not opposed to that build either, but this whole doe the graphics work on a guest VM has me hesitating.

[dave_m]

My question is, for all of you guys using the 970 Extreme4 is, what do you do for display output?    I know you CAN setup a headless server, but I am assuming that for at least the initial install you must have had some sort of monitor hooked up to your machine.    Again, I am assuming, but you probably put some sort of cheap GPU onto your mobo since the Extreme4 doesnt even have a vga port or any onboard graphics.

 

PCI Video card off ebay, I got an ATI Rage but anything should work.  I'm not planning on leaving it in the ESXI box after it's all set up.

[jangjong]

Hi guys, Im a little late to the party but I had a question I was hoping someone had some insight on.

 

I am leaning towards building with a 970 Extreme4 mobo, a fx6300 CPU, and 16 gigs of ram.  Ideally I will install esxi and then setup some VMs.  Eventually, I would like to use unRaid with a m1015 or something and build an array 1 drive at a time.  For the time being though I will probably just setup a SMB share drive that my attached network devices will use.

 

My question is, for all of you guys using the 970 Extreme4 is, what do you do for display output?    I know you CAN setup a headless server, but I am assuming that for at least the initial install you must have had some sort of monitor hooked up to your machine.    Again, I am assuming, but you probably put some sort of cheap GPU onto your mobo since the Extreme4 doesnt even have a vga port or any onboard graphics.

 

Ideally my setup would run, pfesense on a datastore, an unRaid array through the m1015, and then a linux distro for light office work and maybe some video playback.  Has anyone here successfully passed through a GPU to a guest VM on their ESXi build?  I saw one guy here posted that he had pass through working on a a10-58000k setup with ESXi, and Im not opposed to that build either, but this whole doe the graphics work on a guest VM has me hesitating.

 

I'm planning on doing something similar.. (a xbmc vm that uses the video card)

 

Have HD 5450 card right now, so i ordered ATI Rage so that can be the main video output for console and 5450 can be hw passthrough.

 

I should be getting it tomorrow so I will let you know how this works out.

[jangjong]

update..

the problem may not be passing through the video card..

 

i am not able to passthrough USB devices, which means no keyboard or mouse (dedicated input device for the vm)

Looks like it's a bug in 5.1.. because i see that some people were able to do it in 5.0u1, but not able to in 5.1.

maybe you should install 5.0u1 instead to get it to work.

 

there's a thread about this issue here: http://communities.vmware.com/message/2183426#2183426

 

Hopefully.. they will have a fix for this...

 

 

[indorill]

Hi again.  Just to post a followup to my last posts.. I ended up getting the ASUS M5a97 R2.0 mobo.  It has BIOS support for IOMMU and 32-64bit support.  All in all it looked like a safe bet.  Unfortunately it doesn't seem to actually work.  ESXi 5.1 shows that is supports passthrough, but I was unable to reliably configure it.  Rebooting my host intermittently caused passthrough values to be reset, and trying to reassign devices to passthrough was unrelaiable. 

 

Sometimes it would work, sometimes it wouldn't.  I was able to successfully pass through my nics to a VM in ESXI, but upon turning the machine on, it would crash.  I did have more success with 5.0u2, but again there were stability issues when passthrough was enabled, and it didnt always work on reboot.

 

Not being one to give up, I did move on and try to setup my VMs, pfsense and unraid in KVM.  KVM also having the benefit of letting me use a linux desktop on the host.  Again though, and unfortunately, after several days of trying passthrough does not appear to be supported with this board.  Its so close, I can practically taste it, but it just never works.

 

I am RMAing my ASUS board and will try it out with the ASROCK 970 Extreme4.    Ill post more on my adventures when I have something else to contribute.

[jangjong]

okay. here's another info.

 

recently i bought M1015 to replace my MV8. However, in earlier posts, people mention that they weren't able to flash it to IT firmware using 970 extreme4

 

Well, I was able to flash M1015 without any problem using built in EFI shell in 970 extremem4! no need to try to flash it from a different mobo

 

this is what i did pretty much:

 

http://lime-technology.com/forum/index.php?topic=20761.msg186485#msg186485

[Ford Prefect]

Ok, what about ECC memory support ?!?

 

...coming from another requirement, I came across AMD AM3+ platform and discovered that - in theory - ECC UDIMMs should be supported.

Almost all ASUS AM3+ mobos list this feature in the specs/docs and the FX CPUs and Opteron 3300 series support it.

 

Edit: almost all AM2/AM3 CPUs support that and ASUS mobos as well, see: http://www.amd.com/US/PRODUCTS/EMBEDDED/PROCESSORS/Pages/socket-am3.aspx

 

Anyone out there who did a test?

What other brands support it?

I wonder if ASRock does a wonderful job in supporting IOMMU (and vt-d in the intel based chipsets, like with the B75M Pro 3 on S1155),

they might as well have done this for the memory support.

 

...this would open a nice low-cost option for a "real" home server box.

 

BTW: I am on order for a 3350HE and a cheapo ASUS M5A78L-M/USB3

          So I will soon find out for myself. (chosen because of onboard GPU, not interested in IOMMU but ECC and AES Support for another build)

[vl1969]

Here is my planed New Server Build to be ready in 2 month or so(my current unRaid build is in my sig.):

 

Got a server from TAMs

I have always try to use  AMD if I can. much better ROI and usually work better and faster (at least for me.)

 

the server I got came per-built, just plug the HDD and install OS.

I know that I need more memory to build and run all VM I want. but I do want to build out a basic config. to start with and move my current UnRaid to it as I do not have a lot of spare HHDs.

 

server specs and planed config. is below

 

 

CASE: SC846 24 Bay SuperMicro with caddies

MB:  SuperMicro H8DME-2

CPU:  1(one)- AMD Opteron Quad Core 2346HE @ 1.8GHz (planed upgrade to dual AMD Opteron 2419EE HEXA-CORE)

PSU:  2 Ablecom PWS-902-IR redundant Power supplies

RAM:  8GB 4x2GB  667  PS2-5300 ECC chips (planed update to 16 or 32Gb)

SAS CONTROLLERS: 3 SAT2-MV8 Raid cards

IPMI Card: Kira 100

NIC: 2 built-in  100/1000 Gigabit (on the MB) intel pro dual port Gigabit nic total 4 port network access

HDD: (4) Barracuda LP 2TB 5,900 RPM SATA 3Gb/s 3.5" Internal Hard Drive ST320005N4A1AS-RK

 

all HDD for unraid  will be moved from the current build AS IS if possible. (3 2TB for basic setup)

 

an additional 1TB and 2TB  SATA HDD will be added  to be used in other VMs

a 300GB PATA HDD will be used for ESXi  dataStore un less I can find larger drive for it.

the MB have 1 PATA controller might as well use it.

if I can find 2 internal cages to hold 3 HDD each that I can mount inside the chassis

I can add 6 more SATA drives connected to the MB SATA ports.making total of 30 HDDs. :-)

 

 

planned build:

ESXi 5.1

 

1(one) VM -- unRaid 5.0RC13 basic setup with 1 CPU and 1Gb RAM and 500Gb Hard Drive running with no USB Flash

to do new drive preclear only. I want something that I can run preclears on without worrying about breaking.

 

1(one) VM -- unRaid 5.0RC13 (starting with Basic licenses moving to plus in 6-8 month)

1 CPU

2 GB RAM

100MB HD to hold all the files for unRaid. nothing will be run on the setup except file shares + (possible unMenu and/or Simple Features.)

with 1 SAS card hardware pass-through total 8 drive

1(one) VM -- Linux build (Mint or Ubuntu) to run SubNZB + Coach Potato + sick beard + transmission

 

1(one) VM -- Windows 7 or Windows 8 Build (probably win7) to run file conversion/transcoding apps  like  Handbreak , MakeMKV , IVFree converter or such when needed.

 

1(one) VM -- FOG server setup. (imaging and PXE boot)

          Planned usage:

          --- imaging all my home PCs for back up and restore

        Desktop virtualization:

          --- PXE boot for HTPC setup 

          --- PXE boot for main PC setup

          --- PXE boot for Kids PC setup

 

1(one) VM -- pfSence or something like it for custom router/firewall.

 

 

what do you think?

all suggestions and help are  welcome.

 

 

[ZeroK]

Unfortunately I think those tamsolutions servers won't do hardware passthrough to support a proper ESXi install like you want with the motherboard that comes with it, according to this thread.  http://lime-technology.com/forum/index.php?topic=18610.90

and this one http://lime-technology.com/forum/index.php?topic=18610.msg168940#msg168940

 

I could be wrong though.

 

I believe you would have to replace the motherboard with the ASRock 970 Extreme4.

 

[vl1969]

Zerok.  You are right, this setup does not work on esxi. I am testing xenserver right now.

 

Sent from my SGH-T889 using Tapatalk 4 Beta

 

 

[Dephcon]

I'm ordering a FX-6300 and a 990FXA-UD5 to give ESXi a shot.

[perfessor101]

Currently I'm migrating my old unRaid sever over to an Esxi 5.0 update 1 server I have running.

 

It's got a Gigabyte GA-990FXA-UD7 motherboard with 16 GB Kingston Hyperx DDR2-1866 Ram and an AMD FX-8350 Processor.

 

Seems to be working very well ...

 

I have two AOC-SAS2LP-MV8 cards Via PCI-e passthrough attached to a VM running unRaid  5.0-rc16c working.

 

I heard from my local computer retailer that the UD5 was working for people in their various Esxi home server projects.

[Ford Prefect]

Currently I'm migrating my old unRaid sever over to an Esxi 5.0 update 1 server I have running.

 

It's got a Gigabyte GA-990FXA-UD7 motherboard with 16 GB Kingston Hyperx DDR2-1866 Ram and an AMD FX-8350 Processor.

 

Seems to be working very well ...

 

I have two AOC-SAS2LP-MV8 cards Via PCI-e passthrough attached to a VM running unRaid  5.0-rc16c working.

 

I heard from my local computer retailer that the UD5 was working for people in their various Esxi home server projects.

 

...you're sure that this is DDR2 RAM?...should be DDR3, no?

Also, I see lots of hints that this mobo will (unofficially) support ECC memory (unbuffered)....the CPU does for sure....care to try? 

 

Edit: See http://hardforum.com/showpost.php?p=1039470870&postcount=11

[perfessor101]

sorry DDR3 ... didn't check before I posted ...

 

I don't currently have any ECC memory ...

 

I'll check prices later

[chickensoup]

Hey kids, great thread. Just read it from start to finish..

 

About six months ago I moved to Canada and left my unRAID box back in Aus, now I'm thinking it might be worth building another one but I'm on a budget, which is how I ended up here. It's hard to know when to quit I tell you, unRAID is like a drug.. I started looking at getting another external hard drive, then a small NAS, then a larger NAS, then pricing up building a small unRAID server and now I'm thinking of building a mITX unRAID server + ESXi  though I hadn't taken into consideration the cost of a controller card for ESXi. I don't even know if this is possible (AMD mITX unRAID + ESXi), letalone with my budget Obviously AMD is great for saving pennies, was hoping to spend < $500 excluding drives..

 

Has anyone had any luck running ESXi (or unRAID + ESXi) on a mITX board? mITX AMD boards seem to be pretty rare as is.

 

Eye candy: http://www.ncix.com/products/?sku=71144&vpn=BFC-PRO-300-WWXKW-RP&manufacture=BitFenix&promoid=1360

 

[Dephcon]

Lots of people are running ESXi 5.x with Xeon 1230v2/v3 on Z77/Z87 ITX mobos, it's a pretty baller combination for smaller storage requirements.  You might want to look at the Node 304 case to get an extra drive slot and it probably has enough room to jam some SSDs in there as well.  It's also smaller than the Prodigy.

 

Keep in mind that in addition to your unraid disks, you'll need at least one disk (pref. SSD) to run as your VMDK datastore... but your budget will be the deciding factor.  You will need to buy a HBA to pass-through to the VM for the unraid disks, but you can get an IBM M1015 off ebay.ca for about $100CAD, which is a bargain for such a  sweet card.

[Ford Prefect]

..IMHO you won't find a miniITX with the IOMMU capable chipsets...they are AM3+ and these CPUs draw too much power for a board that small..

 

[Dephcon]

Not until get compress they north/southbridge chipsets into one

[chickensoup]

Lots of people are running ESXi 5.x with Xeon 1230v2/v3 on Z77/Z87 ITX mobos, it's a pretty baller combination for smaller storage requirements.  You might want to look at the Node 304 case to get an extra drive slot and it probably has enough room to jam some SSDs in there as well.  It's also smaller than the Prodigy.

 

Keep in mind that in addition to your unraid disks, you'll need at least one disk (pref. SSD) to run as your VMDK datastore... but your budget will be the deciding factor.  You will need to buy a HBA to pass-through to the VM for the unraid disks, but you can get an IBM M1015 off ebay.ca for about $100CAD, which is a bargain for such a  sweet card.

 

I love fractal cases but the drive access in those is a little bit of a pain.. though once it is setup it's not a big deal. I was considering an Array R2 as my mate built his unRAID server in one but they use an SFX PSU and seem to be EOL. As for that CPU, it's way over my budget at the moment.. I really can't afford to spend more than about $100 on the CPU and @ $270+ it's just too expensive. I have a 2.5" 320GB drive that I got for free so was thinking of using that for datastore, otherwise I might have to shell out for a small budget SSD.

[Dephcon]

You might just be better off doing bare metal with an AMD FM2 ITX system.  It'll be pretty cheap and you can re-use it as a HTPC down the road if you invest in a virtualized setup. You wouldn't need to buy an HBA assuming the ITX board has enough SATA ports (seems only the A85x boards have over 4 SATA) .  You could attempt ESXi with the purchase of an HBA, but i'm not sure how well those APUs will work.

 

http://www.newegg.ca/Product/Product.aspx?Item=N82E16813157357

 

Pair that with an A10-6700/A8-6500, 4-8GB ram, case and PSU and you're good to go.

[asdiasx]

Hi, sorry for the newbie question...

 

I'd like to stream via wi-fi video and music that are stored at my unraid server...

 

Instead of using plugins at my actual server, I think a separeted server for streaming could perform better...

Is this true?

 

And... If this is the better direction to go, not considering energy/space/budget savings, but in terms of network performance (stream without stuttering), wich showld be better:

- a new physical machine for stream conected by gigabit ethernet or ;

- A vitualized server running two VM (for unraid and the streamer)?

 

Thanks!

[Ford Prefect]

[...] but in terms of network performance (stream without stuttering), wich showld be better:[...]

 

This is a pure architectural decision.

In terms of performance, saturating a single Gbit NIC , all options can be build to perform equal.

If you are suffering from stuttering now, there is something wrong.

 

Of course a new build could solve that, but independently of being virtual, with plugins or separate machines.

 

...just tell us something about your current unRAID hardware and what you want to achieve or what is going wrong now in detail.

[asdiasx]

Thanks Ford for your interest to help! 

I don't want to digress this topic, so feel free to poit where should be the correct place to deal with my doubts...

 

My actual serve is:

 

MB: ASUS M5A78L-MLX

Processor: Athlon II X2 250

Mem: 4 GB (one slot)

 

Unraid: Plus version 5.0 RC12a

Storage: 6 drivers (2TB parity + 5 other total 8.5TB)

Unmenu + Airvideo +  DMIdecode + HDparm + iStat + Jre + lsof + nfs 3G + PCIutils + reiserfs + rsync + SMART tools

 

Onboard GigE connected to a Apple Airport Extreme

 

At this moment my problem is related to access to my media files... Everything I tried showed up problems, for instance:

 

- Samba file access from Egreat media center (samba): playing FLAC files from server get stuttering...

- Playing FLAC files (high res) from Unraid, accessed via NFS by Voyage MPD server gets stutterin (more intermitent).

- No way to mount BD ISO files from Unraid on my HTPC (I know maybe this requires a GigE instead of wifi).

 

All these were tested via wifi (n/5Ghz) connection...

 

So I'm thinking of trying to process the video/audio at the server and stream to the final client (mainly video)...

 

Look, all these are very new to me and apologise y if I'm having some stupid wishes...

 

Regards,

 

Adriano

 

 

[Ford Prefect]

 

At this moment my problem is related to access to my media files... Everything I tried showed up problems, for instance:

 

- Samba file access from Egreat media center (samba): playing FLAC files from server get stuttering...

- Playing FLAC files (high res) from Unraid, accessed via NFS by Voyage MPD server gets stutterin (more intermitent).

- No way to mount BD ISO files from Unraid on my HTPC (I know maybe this requires a GigE instead of wifi).

 

All these were tested via wifi (n/5Ghz) connection...

 

 

...nothing wrong with the parts in your list, AFAI can see.

These are all Audio  Files...and still you have problems?

 

Did you test the performance with other clients (like a wired PC?)...you need to rule/find out that unRAID is the cause of that.

Edit: I'd first run the memtest on the unRAID box.

 

At first, I'd suspect your wireless connection...Airport AP and cable to unRAID, setup of your wireless clients...maybe the AP is in some compatibility mode and everyone connects at 10Mbit 

I guess with "mounting BD" you mean "playing"...mounting shouldn't be an issue if the connection is at least 54MBit.