ATLAS My Virtualized unRAID server

[BLKMGK]

Reading reading reading...

 

It looks like each M1015 will support 8 SATA drives using both ports and uses PCIe x8 slots. I need to support 24 drives, I think 16 dedicated to unRAID would be plenty. If I can install a 3rd M1015 into the PCIe x16 slot of this board (appears doable) I'd be supporting all 24 drives I believe. No expander needed correct? When I passthru controllers with ESX does the entire card get passed or can just one port be passed?

 

I do not expect to do crazy things with the remaining drives and I may run some SSD off the internal board connectors. Sounds like I can share the NIC with unRAID and other VMs, I don't see a bandwidth issue there since unRAID mostly streams and supports backups of workstations.

 

Does this plan make sense? Collapse my two servers to one (boo) and no need for an expander?

[cipdor76]

 

 

Is the Supermicro AOC-SAS2LP-MV8 compatible with ESXi 5

 

Unknown. No one has confirmed if it is.

 

AOC-SAS2LP-MV8 passed to unraid is working just fine on my X9SCA-F from almost 1 year now

just be sure to update the FW on controller to IT mode (there is a post in the forum how to do that - for LSI 2008 controllers)

 

 

[RockDawg]

Reading reading reading...

 

It looks like each M1015 will support 8 SATA drives using both ports and uses PCIe x8 slots. I need to support 24 drives, I think 16 dedicated to unRAID would be plenty. If I can install a 3rd M1015 into the PCIe x16 slot of this board (appears doable) I'd be supporting all 24 drives I believe. No expander needed correct? When I passthru controllers with ESX does the entire card get passed or can just one port be passed?

 

I do not expect to do crazy things with the remaining drives and I may run some SSD off the internal board connectors. Sounds like I can share the NIC with unRAID and other VMs, I don't see a bandwidth issue there since unRAID mostly streams and supports backups of workstations.

 

Does this plan make sense? Collapse my two servers to one (boo) and no need for an expander?

 

I recently condensed two unRAID servers into one ESXI machine with three M1015's like you are considering.  I love it!  Yes, if you pass through a controller in ESXi the entire controller is passed through and only to a single OS.  You can also share a single NIC to all your VM's.  I've never run a M1015 in a 16x slot.  Theoretically it should work, but I've read that some boards can be a bit flaky with cards other than graphics cards in a 16x slot.  I have no idea how true that is or how big of a concern it may be, but I did read some claims while researching my own build.

[BLKMGK]

Well, I pulled the trigger on a Tyan board, 3x M1015, 32Gig RAM, a Norco 24, a good PSU, and a SANDY Xeon. I found a guy selling M1015 new server pulls on eBay for a reasonable price but I'll need to flash them. Cables are left but I've got time to run this in parallel as my old hardware will be holiday gifts :-)

 

It kills me to buy Sandy vs Ivy hardware but having read the issues here I don't think I wish to break from the pack! Stay tuned, I'm sure I'll be asking for help as this goes together and appreciate the knowledge here as well as the advice I got from Bob via PM. I'm looking forward to the journey!

[RockDawg]

Good luck.  Be careful though this can become addictive.  After I ordered my parts I ended up browsing craigslist and bought a server rack which lead to a rack mount switch, rack mount power strip and I can't quit looking at rack mount UPS's! 

 

And that's not even mentioning all the things you'll start thinking about having this box do.

[BLKMGK]

I'm already scoping rack mount UPS, if for no other reason than they HAVE to be better than the $100 units I've been getting from Costco that die left and right! I've been trying to hook car batteries up to them with no success - argh!

 

I did see a VERY nice roll around enclosed half rack at a vendor site today that caused much drooling. I hadn't thought about Craig's for rack stuff - uh oh! I have a closet (with cooling) that can fit this box that's out of the way and if not cool enough or too loud I have another more remote spot that would be harder to access. I may do a little construction to best house this stuff out of the way, I'm sick of the hum from existing stuff now and I doubt the Norco will be quieter out of the box! It should be here tomorrow, rest arrives Monday.

 

Technology stuff I do for a living so learning the ESX software better and finding more things to do with it is all a bonus. My existing ESX box got little enough use that it sits idle, this one will be much more heavily utilized!

[dave]

Anyone successfully been able to remotely manage their ESXi server via Android or iOS apps? I tried the Android tonight and it kept giving me HTTP 400 errors.... research online said I need some sort of View Server? 

[derekos]

John - thanks so much for this thread.  It has been hugely helpful.  I couldn't have gotten this working without you.

 

I have a question.  I began with two mv8 cards and just added a third - without any cables, so the card is just plugged in and awaiting cables and drives. 

 

Upon reboot a bunch (7) of my drives weren't found in UnRaid.  Still no luck after much resetting of cables, etc. etc.  So, I removed the new MV8 card.  Boom - all drives show up again.

 

Do I need to do something special in ESXI?  Any ideas?

 

Thanks!

[BetaQuasi]

Have you disabled INT13 on all of those MV8's?

[derekos]

I don't think I have disabled INT 13 on any of them.  I don't recall doing that, in the bios I assume.  I will look at that tonight and post back.

[MaxNL]

 

 

Is the Supermicro AOC-SAS2LP-MV8 compatible with ESXi 5

 

Unknown. No one has confirmed if it is.

 

AOC-SAS2LP-MV8 passed to unraid is working just fine on my X9SCA-F from almost 1 year now

just be sure to update the FW on controller to IT mode (there is a post in the forum how to do that - for LSI 2008 controllers)

 

I got an email from Supermicro support stating that AOC-SAS2LP-MV8 cards are not compatible with ESXi.

 

I will check the FW update approach and I will post back (it will take few days... pretty busy right now)

 

Cheers

Max

[MaxNL]

 

 

Is the Supermicro AOC-SAS2LP-MV8 compatible with ESXi 5

 

Unknown. No one has confirmed if it is.

 

AOC-SAS2LP-MV8 passed to unraid is working just fine on my X9SCA-F from almost 1 year now

just be sure to update the FW on controller to IT mode (there is a post in the forum how to do that - for LSI 2008 controllers)

 

I got an email from Supermicro support stating that AOC-SAS2LP-MV8 cards are not compatible with ESXi.

 

I will check the FW update approach and I will post back (it will take few days... pretty busy right now)

 

Cheers

Max

I forgot the AOC-SAS2LP-MV8 is based on marvell chip. The AOC-USAS2-L8i is based on LSI chip.

[joelones]

I am having troubling trying to get networking in the unRAID guest via the second onboard NIC (shared networking with the first nic works). I'm just worried because it currently has three IPs: IPMI mgmt IP, ESXi mgmt IP and unraid. I followed the instructions as per the OP.

 

In terms of hardware, I am using the Supermicro X8SIL board (xeon cpu). vSphere and IPMI is on the first NIC (which works). And I passed through the second which I want to dedicate to unRAID (v5rc8a). I also have the most recent vmtools installed, ESXi 5.1 and both Intel NICs should be compatible 82574L (whether one could be passed through - I do not know).

 

Unfortunately, eth0 is not coming up in unRAID and the following error is visible in the log:

 

Failed to initialize MSI-X interrupts

 

Please see screens. Google was no help and no one else seems to be having this problem. Tried to disable serial ports in bios and remote management to free up interrupts (if indeed that is the problem), to no avail.

 

 

Guest settings:

http://i.imgur.com/wNbl5.png

http://i.imgur.com/7Rj7m.png

 

Many thanks

 

[Ice_Black]

How to install Unraid on ESXi without controller cards?

 

This tutorial talk about "VMDirectPath Passthough" but my hard drives connected on the motherboard.

[BobPhoenix]

How to install Unraid on ESXi without controller cards?

 

This tutorial talk about "VMDirectPath Passthough" but my hard drives connected on the motherboard.

Use the RDM method in that case.

[joelones]

So I added a Network adapter (E1000) in the configure for unRAID and removed the passthrough NIC and I am able to get network access but surprisingly I still see the error in the log:

 

e1000: Intel® PRO/1000 Network Driver - version 7.3.21-k8-NAPI

e1000: Copyright © 1999-2006 Intel Corporation.

e1000 0000:02:01.0: setting latency timer to 64

e1000e: Intel® PRO/1000 Network Driver - 1.9.5-k

e1000e: Copyright© 1999 - 2012 Intel Corporation.

e1000e 0000:13:00.0: Disabling ASPM L0s L1

e1000e 0000:13:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode

e1000e 0000:13:00.0: (unregistered net_device): Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.

e1000e 0000:13:00.0: (unregistered net_device): Failed to initialize MSI interrupts.  Falling back to legacy interrupts.

e1000e: probe of 0000:13:00.0 failed with error -2

e1000 0000:02:01.0: eth0: (PCI:66MHz:32-bit) 00:50:56:1b:34:c3

e1000 0000:02:01.0: eth0: Intel® PRO/1000 Network Connection

e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

 

Only thing I could think is that one of the PCIe slots where I have the M1015 & MV8 are causing some sort of hardware conflict or is some bug in the e1000e module. Sucky thing is that I have 3 pcie slots of which two are used and the first one is completely obstructed by the shroud of my Supermicro 933T-R760 case.

 

Another issue: when I attempted to add the network adapter I have only two options: "Flexilble" or "E1000", which is different than what I see for my windows vms where I see "VMXNET3". Could it be that Zeron's vmtools is not fully compatible with ESXi 5.1 ? In vSphere Client I can see VMware Tools are running (3rd-party/Independent). Maybe that's why I do not see the VMXNET3 option.

 

Any thoughts?

[Ice_Black]

Ah I see. I have 3 x 2TB and a 250GB Drives.

 

My Plan are:

 

UnRaid: 2TB parity drive and 2 x 1.5TB drives.  250GB Drive for cache.

 

Windows 7: Rest of the spaces left from two drives and setup Raid 1.

 

Would that work or what do you recommend?

[brian89gp]

Another issue: when I attempted to add the network adapter I have only two options: "Flexilble" or "E1000", which is different than what I see for my windows vms where I see "VMXNET3". Could it be that Zeron's vmtools is not fully compatible with ESXi 5.1 ? In vSphere Client I can see VMware Tools are running (3rd-party/Independent). Maybe that's why I do not see the VMXNET3 option.

 

Any thoughts?

 

Which options you get for the NIC hardware is dependent on the OS you select for the guest.  Which one did you choose?

[joelones]

That could explain it. I just noticed that the os for unraid is "Other (32-bit)", and not "32bit FreeBSD". Perhaps that could explain my second problem.

 

Edit: indeed it does. Thanks for the tip, you think this could have some relation to the "Failed to initialize MSI-X interrupts." problem. When passing through the second nic, i do not need to add a NIC, correct?

[brian89gp]

That could explain it. I just noticed that the os for unraid is "Other (32-bit)", and not "32bit FreeBSD". Perhaps that could explain my second problem.

 

Edit: indeed it does. Thanks for the tip, you think this could have some relation to the "Failed to initialize MSI-X interrupts." problem. When passing through the second nic, i do not need to add a NIC, correct?

 

It might.  If there is an option, it must do something right?

 

That is correct.  Unless you have a specific need for a dedicated NIC it is usually not necessary though.

[BobPhoenix]

Ah I see. I have 3 x 2TB and a 250GB Drives.

 

My Plan are:

 

UnRaid: 2TB parity drive and 2 x 1.5TB drives.  250GB Drive for cache.

 

Windows 7: Rest of the spaces left from two drives and setup Raid 1.

 

Would that work or what do you recommend?

The software raid on a motherboard is not compatible with ESXi as far as I know.  So you won't be able to setup your Raid 1 for your Windows VM without a controller card.  Also you have to dedicate a drive to ESXi for a datastore drive anyway.  So I would put one of your 2TB drives as the datastore drive and create virtual HDD(s) on it for your Windows 7 VM and any other VMs you might want to setup.  Then you can use the snapshot capability of ESXi and backup software to keep backups of your virtual HDD(s).  Then add the extra drive you were going to use in your Raid 1 as another data drive for unRAID.

[joelones]

I'm just wondering what your avg parity check speed is with the esxi setup.

 

I know it varies largely with hardware. By randomly hitting the refresh button mine seems to go between 10-60mb/s, though it is not constant as it was with a bare metal setup. In fact I see the speed mount and then dip constantly between the aforementioned speeds, which is odd to me. This was not case before as the parity speed used to be much more constant at around 50-60mb/s

 

I have all my disks on a M1015 (parity is here) and MV8 with a xeon x3430 processor.

[BLKMGK]

Okay, stupid question!

 

Where is "tip #1"?!

 

Apparently this is a tip that explains how to store ISO on the server for booting purposes etc. and I assume they get uploaded via winSCP or some such. I have connected to my server via SCP and I've navigated to my storage device but I'm not sure if this is where I ought to be placing an ISO. The first post referrs to doing this - multiple times referencing tip #1 - but there's no tip #1. Argh! Am I nutz?

 

I'm going to try the Plop method that boots an ISO that then boomerangs to a USB stick. So far searches and text searches on pages aren't finding this tip. A straightforward way to boot and get the USB firing would be most welcome.

 

thanks!

[RockDawg]

Tip #1 is here:

 

http://lime-technology.com/forum/index.php?topic=14695.msg138466#msg138466

 

It is a way to transfer ISO's or other files to an ESXi dataset without using WinSCP.

[BLKMGK]

Tip #1 is here:

 

http://lime-technology.com/forum/index.php?topic=14695.msg138466#msg138466

 

It is a way to transfer ISO's or other files to an ESXi dataset without using WinSCP.

 

Thank you! <sigh> Even text searches were failing me but I guess I was looking for it higher in the posting. Working on my own build and trying to get past some lumps and bumps. ESX 5.1 is slightly different and for some reason is refusing to pass my USB at all so far. I have some sneaking suspicions as to what it's doing but once I've got that solved I'll need the ISO to get further...

 

I gotta' say that Johnm's instructions are pretty darned detailed on this process and a big help! Appreciate you pointing out what I was missing there <ahem>