Guys,

 

Been lurking for a month around this ESXi thread. Read the whole thing in two days while we were getting pummeled by Sandy. Big thanks to John and the community. Its an awesome thread. Trying to fine tune how best to build a new ESXi environment before I jump in. Thread raises a bunch of good questions for me.

 

Here is a Google Docs link (don't want to clog thread) for what I am thinking for my build but obviously it is very workflow dependent so I would really appreciate any advice or help.

 

https://docs.google.com/spreadsheet/ccc?key=0ApH0dF_Ug3JFdF84MlZSdW5pTWRqclRQTU83ejI3eVE#gid=0

 

I currently have a 10 drive unRAId array (1 parity, 1 cache) which i use to store all my media, my itunes library (mistake), local crash plan backups, and local image backups of our five household computers. Also have a massive hackintosh box to run my torrent downloads and M4V conversions as well as converting my Blu Rays to M4V (we get like one two a week from my wife's company). I also run Crashplan on this box (not on unRAID) to act as the backup box via my unRAID array. Also, my Plex media server runs on this box. Lastly, I also run a win7 VM on this box to do one thing, run AnyDVD.

 

My goal is to expand my unRAID array with two more drives while consolidating my other machines into one box since my small NYC apt cannot handle all these boxes. ESXi seems like the perfect option. Here is what I am thinking, and would love any input.

 

VM One - Add an additional controller card and pass both through to the unRAID VM so I can add three more drives to my unRAID array. I would also have the cache drive and parity drive on the controller card. I am a fan on running unRAID as naked as possible so i prefer not to run crash plan, Plex, or Transmission on this VM.

VM Two - Setup a win7 install to both run AnYDVD as well as manage the ESXi environment w vSphere. No need to have a passthrough drive here i don't think, unless I add to it - See below

VM Three - Crashplan. I really want to have a minimal install via Win7 to just run crash plan. The only question is should i also consolidate this into VM Two? My thought is keep it separate. Again no need for a passthrough drive.

VM Four - On my current OSX machine I have a combination of Automator script, shell scripts, and applescripts that combine with an OSX program called Hazel to automate my torrent downloads, unpacking, conversion to MV4, renaming, tagging and finally move to my unRAID array. Putting aside whether i can get OSX running in ESXi, does it make sense to separate out my Handbrake and MakeMKV conversions into a separate VM just for those? I could also put both of these VM Two. But then I think i may want to get an SSD and RDM it into this VM so its as fast as can be since I am doing a lot of conversions. Regardless, I would still need to run the 4th OSX VM to manage my workflow and run my Plex media server. I may also want to run an SSD drive directly for this VM as well.

 

As you can see, I am a little confused as to the best way to go about this. ESXi provides so much flexibility I am uncertain how best to utilize it. I don't want to have too many VMs but there is obviously a benefit to being able to separate out some of my core processes into distinct VMs. I do think one of those VMs makes sense to have a RDM SSD while the others can all be run off the 3TB drive i am purchasing.

 

I would really welcome any advice on the matter.

 

Looking forward to jumping in and much thanks again for all the guidance.

I don't mind who answers, just looking for a ball park figure really. I'm going to running whs2011 with 1 2tb drive and Unraid with 8 4tb drives. Just looking at whether its going to make my levy bill go through the roof.

 

mine hovers around 125-140 watts.. it can reach 230ish under full load... (disk rebuild while still serving files)

 

that is with 24 spinners and 3 SSD's. 4 HDDs are in a raid 5 spinning 24x7

I had Atlas out of the rack today.

 

I thought I might get a current photo of it's innards for those that like server pr0n.

 

 

I still need to replace the back fans. only one is hooked up right now.

 

I would like to still mod the chassis to hold my Supermicro 4in1 and relocate the SSD's to that.

 

Hey Johnm,

 

As many in this forum I am using Atlas as a reference for my build. By looking at this picture I saw you are using custom cables to power up the backplanes. Can I ask where did you buy those connectors?

 

What do you think of these http://www.jab-tech.com/ConnectorZ-90-Degree-Pass-thru-Molex-Connector-Black-pr-3155.html?

 

I'm also using a modular PSU (Corsair HX 850). Where can I find the connector going into the PSU?

 

Also, and I may be wrong, it looks like you are powering up both plugs in each backplane from a single PSU. Is this so? Why are you doing it?

 

thanks,

 

those look like what i am using.. i believe mine are genuine molex brand.

there are 2 versions. a passthough and an end cap.

 

i ran two because i second guessed both my cable building skill and the wire is a bit thin.. i wanted to make sure there was no bottleneck.. you only need one.

 

Bobpheonix: you have a lsi controller to intel expander in your sig. How many sata ports do you have/what is the maximum with this setup?

I have one cable going to the expander.  So I can then have 20 drives off the expander and 4 off the 2nd port of the M1015 for 24 total drives.  With a second SAS expander connected to the 2nd port on the same M1015 I could get 40 drives connected to the same controller.  By connecting from one SAS expander to another I believe you can get to the M1015 card maxium of 128 drives.  Note I read that somewhere but not sure where and I'm going from memory so that could be wrong.  Another way to connect to the expander would be to use both M1015 ports and 2 on the expander.  Then you would get a max of 16 drives off the expander.  I tried it that way with my first expander but it didn't work for me.  unRAID would appear to work on parity checks but I was getting crashes when copying data to the array.  Not sure if it was a cable problem or a port problem but disconnecting the second connection to the expander solved my problems.  The actual drive count on my 2 unRAID servers is 19 on one server (all on expander) and 15 on the other server (all on expander).

 

Correct..

i have run it in dual connection mode. it is pretty fast that way.. but costs more then 2 m1015's..

And you don't notice any performance sacrifices?

 

Last off topic question I promise.

 

Yes you do..

 

it really depends on your hardware and drives.. i went from about 110 MB/s parity checks to 85MB/s with 20 drives on a single port.. then again.. that could just be the speed of 24 drives in my unraid.

 

The HAL thread reported a big slowdown also. but He has his parity in the 20 drive pile up. mine is on the free channel.

 

keep in mind, day to day use will be unaffected..

@dhy8386

 

the answer is really the limit of your server. CPU and ram. usually ram..

 

the nice part about ESXI (and any hypervisor server) is that you can make a vanilla win7 box. then copy it. test your apps and see how they perform. if you like it assign more resources or delete it and try another test..

 

i tend to have my management guest also my script guest since it is on 24x7 and must be up..

 

my downloading guest is a hybrid.. I had it on its own drive. but now it is on 2 drives. the C: is a virtual drive on an SSD and the target drive is on a raid array.. (a ZFS array that is on a freenas guest. so i have VMs hosted on ESXi that reside on a guest of the same server )

a bit overkill, but it is the best performance for me.

 

 

Johnm, you are the soul of ESXi in this forum.

 

Someone had to say it.

 

Thanks a lot for your time, help and guidance.

 

Sent from my GT-P7500 using Tapatalk 2

 

 

Meh. I'm just a guy that shared his experience so that others can learn from my mistakes.

 

Lol

 

Thanks though

One quick question.

 

I have currently ordered an IBM ServRaid M1015 controller.

 

Is this the correct break-out cable?

 

http://www.monoprice.com/products/product.asp?c_id=102&cp_id=10254&cs_id=1025406&p_id=8188&seq=1&format=2

 

Thanks.

 

--Sideband Samurai

 

 

--- Edit ---

After searching on the forum, I found that the above link from Monoprice is the correct cable.  That one is a 1/2 meter.

 

This link gives you the cable for the 1M cable. 

http://www.monoprice.com/products/product.asp?c_id=102&cp_id=10254&cs_id=1025406&p_id=8188&seq=1&format=2

 

--Sideband Samuirai

Just wanted to say thanks, setup esxi originally following your guide when there was only a few pages in this thread. Every time I research something new to add its always your posts I find that you've already looked at it. So thanks for saving the rest of us so much time lol.

 

Just got my UPS working but its a serial port one so setup differently to yours but using your scripts works perfectly

 

Now just gotta buy all the bits for my DAS.

Okay, so apparently my problem is that when I go from 2 MV8 cards to 3 MV8 cards - my motherboard doesn't report the third MV8 card.  I only see two controllers when I hit ctrl-m.  So I am going to RMA the third card and see if a new one works - unless anyone thinks I am missing something.

 

Thanks

 

Have you disabled INT13 on all of those MV8's?

Right now I have two MV8 cards and I have 16 drives.  Should I really add a third MV8 or should I just get an Intel RES2SV240? 

Right now I have two MV8 cards and I have 16 drives.  Should I really add a third MV8 or should I just get an Intel RES2SV240?

 

I'm not sure that the mv8 support expanders.

 

Also they are a lot slower than the m1015's which is what most people expander the mv8 already seem to get slow down when they are full.

HD Audio Set Up

 

I don't know if anyone else found this problem, I don't remember reading it in the thread.

 

I just installed Windows 7 Ultimate 64 in my ESXi box (v 5.1). I used the same components as Johnm (X9SCM, Xeon 1240, 16GB RAM, Plextor M3 256GB as datastore).

 

When I started the Win 7 guest, it couldn't find an audio driver. Luckily my friend Google set me in the right direction here

 

I hope it helps some else.

 

 

Hi,

 

I followed the guide to the letter and everything went fine, actually I was expecting it to be more difficult. Thanks Johnm for the guide!

 

However, I run into trouble to install VMware tools in unRAID. I posted it here just in case someone wants/can to chip in.

 

 

good to hear..

 

I have honestly never used the plug-in myself. I have not looked, but i believe I just have the TGZ in the extra folder.

if not, then i have a line in my go script that runs it old school.

 

Until there is an "official" standard for plug-ins, I tend to still use the old school methods. I have had a mixed bag of results with some of the earlier plug-ins. I am sure they have been tweaked since then. For now, I am perfectly fine with old school.

 

If you think you found a bug, I would point it out to Zeron. He is pretty on top of things.

I installed the tools today on RC8 via plugin. Copied it over, ran the install command, it appears to be working, rebooted for other reasons, and it's still working. Not used any of the features YET but will as I need to integrate a UPS into my system...

My whole win7 VM freezes up from sabnzbd. I have 2 VMs Unraid and Windows 7 64 running on ESXI. I was unsure of the problem but it appears to be the Par2 tool that runs after a download. What's baffling is that I used the exact same system running the exact same windows 7 before I virtualised it and files were verified repaired reverified and un-rared without even a hint of a performance dip. Before it had all 8 threads and 8gb of ram exclusively, now it has 7 of the 8 logical cores and 6gb of ram. According to the vSphere performance logs it said CPU usage was minimal but running task manager within the windows 7 VM said it was 98-100% utilisation, memory did not appear to be a bottleneck.  The logs also reported a read/write latency on the virtual disk of 70,000-120,000ms which meant the system was totally unresponsive. I'm hoping that someone active on this thread has first hand experience of this or is knowledgeable enough to offer some suggestions.

So far I have: Adjusted the resources assigned to the VM with no effect and followed steps on the known issues part of sabnzbd which said to change the par2 tool to run on 1 core and disabled 64bit operations.

Reduce the amount of cores assigned to the VM - you don't need that many.  I have 2 logical cores and 2Gb of RAM on my Sab/Sick/Couch box and see no issues whatsoever.  It could also be disk IO that is the issue - where is your datastore? 

 

Mine is on a Corsair Performance Pro SSD, which also provides for a Plex VM (Ubuntu), a pfSense VM and the unRaid VM itself.  No problems at all.

 

 

Mines on a standard 7200RPM commercial hard disk, but I am the only user using it. I wanted it to have as many cores as possible as i use it to encode media. Essentially it handles all my media, which is then shared through unRAID.

I am not using SAB. I am using NBPro and PARBuddy.

 

My guest (2008r2) is 4 threads, and 4GB of ram.  I had 8GB RAM For a while, after a recent patch to NBP it uses less ram. I also know it runs with 2 threads just fine.

 

7 seems way to many. Try bumping it down and see what happens.

 

I have run this same guest as RDP and as a virtual hard drive on a single 7200RPM disk. Now I have it as the OS on a VHD on an SSD and the Data folders as a VHD on a Raid5.

All of the above worked fine. the difference was speed.

 

Many others here are using Sab/Sick/Couch in the same way without issues. there has to be an explanation.

Yeah my personal theory is that 7 vCores is smashing the physical CPU and grinding the thing to a halt.  Don't rely on ESXi CPU usage metrics to judge this, I've seen them be way off before.

 

As a note, I also do encoding off this same VM when required, using Handbrake, and find 2 vCores is plenty.  ESXi dynamically allocates CPU usage anyway, providing you don't hard cap it and leave it at the default for a particular VM (which is unlimited).

I Initially had 4 threads and 4gb of ram, however I upped it to the max due to the system freezing. I shall try lowering it to 2 and see what happens. I'm off away for 2 weeks now, thank you both of you for your advice and hopefully it will be a swift fix once I am back home.

http://www.superbiiz.com/detail.php?name=D38GE1333H

 

is this the right ram for this motherboard

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253

http://www.superbiiz.com/detail.php?name=D38GE1333H

 

is this the right ram for this motherboard

 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253

 

Specs look correct. That's a nice price for the ram.