ESXi can't pass-through


Pinch

Recommended Posts

Hey guys

 

This is an amazingly difficult google and it doesn't seem like any of the existing ESXi threads cover this...

 

This isn't unRAID related as such, but I figured I'd start here: I'm trying to set up unraid on an ESXi build, but am having some trouble using pass-through.

 

I have an ASRock Extreme6 z87 motherboard with an Intel Core i5 4670 connected. Both support VT-d etc.

 

If I pass anything through, unRAID won't recognize it. If I pass it to a Windows VM instead, Windows sees it, but can't use it. It doesn't matter what I try to passthrough, but take fx the ASM1062 SATA3 controller: at first it's recognized as a "SATA controller", but apparently missing the drivers. Installing the drivers enables Windows to see it as "ASM106x controller" but still with a yellow exclamation mark and an error code 10 (can't start the device).

 

I've tried passing through the GPU, SATA controller and NIC, all show the same issue. Is this just a clear sign that my hardware doesn't support pass-through?

Link to comment

...I don't have any experience with that board, but:

 

- you need support in BIOS for vt-d as well..make sute yiu enable that feature.

- FWIW, all I can see is that you tried to passtrough on-board components.

  AFAIK it is normal behaviour with intel chipsets that this does not work properly.

  Try an add-on card, like a SAS controller or NIC that is seated in a PCIe slot.

Link to comment

Thanks for your response.

 

Everything VT-d related has been enabled in the BIOS.

 

Although most people do seem to use addon cards, I haven't come across anything that made it clear that onboard devices won't work, but my issues seem to indicate you're right.

 

I'll try with an addon card.

Link to comment

Just so that you know.  I am on ESXi 5.0 and I am passing through a USB controller that is built into the MB.  I also passed through the NICs on my 4 nic MB without problem.  Other than things built into the CPU (built in CPU graphics for instance) I've not had any problems passing through MB components.  I would still suspect a bios setting that is affecting this.  I would reset to defaults again in the bios and then just turn on the VT-d settings changing nothing else.  If that works then turn on any other bios settings one at a time that you might be using like WOL or ACPI settings until it stops working.  Also look for a bios update and post on the MB forums about this and your ESXi problems.  You might also check with VMware forums to see if anyone else has had success with your MB and ESXi.

Link to comment

Hey BobPhoenix

 

Nice to hear someone's had success passing through onboard devices!

 

I've triple checked the BIOS and everything regarding virtualization has been enabled, but I'll try starting from the bottom and reset to defaults.

 

The MB should be able to automatically fetch updates through the BIOS, which claims there are none.

Link to comment

Hey BobPhoenix

 

Nice to hear someone's had success passing through onboard devices!

 

I've triple checked the BIOS and everything regarding virtualization has been enabled, but I'll try starting from the bottom and reset to defaults.

 

The MB should be able to automatically fetch updates through the BIOS, which claims there are none.

Definitely check the VMware forums and the MB manufacturer forums for your MB and ESXi.  It is possible that they have a bad bios and haven't fixed it yet or broke it and you need to roll back.  I had that with a bios upgrade to support Ivy Bridge CPUs and an Aver Media Duet tuner card.  My Intel MB bios got an update and the Duet worked but my ESXi server MBs do not.  I downgraded my Tyan MBs to an earlier BIOS that still supported VT-d but also did NOT support Ivy Bridge CPUs to get the Duet's to work.  I would bet that it is another bios setting causing your issue as long as both the MB and CPU support pass through.  But a check on other forums might provide insight on your MB from other users and what they may have needed to do to get pass through working.  You might also try to downgrade your ESXi version if you are beyond 5.0.  I've been using 5.0 because of problems with later versions and pass through of USB and PCI based on posts on the VMware forums.
Link to comment

So I tried both resetting the BIOS and at the same time installed ESXi 5.1u1 instead of 5.5 - much to my own surprise pass-through worked!

 

Tried reinstalling 5.5, not changing any BIOS setting, and it was broken again, so it appears 5.5 is the issue...

 

I'd like to know why 5.5 fails, but at the moment 5.1u1 works, so I'm happy :)

Link to comment

So I tried both resetting the BIOS and at the same time installed ESXi 5.1u1 instead of 5.5 - much to my own surprise pass-through worked!

 

Tried reinstalling 5.5, not changing any BIOS setting, and it was broken again, so it appears 5.5 is the issue...

 

I'd like to know why 5.5 fails, but at the moment 5.1u1 works, so I'm happy :)

Since ESXi kernel is essentially a Linux kernel they probably upgraded the kernel to a latter version in 5.5 and broke things like your MB and pass through that were previously working.  My guess anyway.
Link to comment

While on the subject of ESXi and unRAID, what's your experience with performance? I am currently not able get more than 50-60MB/s on parity checks - doesn't matter if I only have virtual disks mapped or if only disks on the pass-throughed controller are mapped.

 

Another weird thing is that plugins (.plg) located in /boot/config/plugins won't install automatically. If I installplg them manually they work, also after a reboot...

 

I'm close to being ready to move my current array from my old server, but am still a bit hesitant because of the performance issue and the plugins not installing (it's such a basic feature, makes me think I've set up something incorrect).

 

I'm using the pre-built VMDK from http://lime-technology.com/forum/index.php?topic=26639.0.

Link to comment

You are not the only one having problems. If you look within the Virtualaztion Forums many people are having the same issues with ESXi 5.5 breaking pass through.

 

Between that and their new licensing... Many home users are switching away from it or dropping back down to ESXi 5.1.

 

 

You might want to consider moving to XenServer or Linux with either Xen or KVM.

 

Link to comment

I have no need for the added features in 5.5, so as long as I'm not the only one having issues, I'll have faith it'll be fixed at some point...

 

If only I knew why plugins aren't being automatically installed, I would be ready to migrate to the new server. It's not really an issue since they can be installed manually, I just hate not knowing why it's not working!

Link to comment

Thanks :)

 

Keeping this thread on subject, I just moved an ASRock SATA3 (Marvell 88SE9128 / 88SE9120) from the old tower to check if ESXi would recognize it. It does and disks are read fine.

 

But when passing it through to either unRAID or a Win7 VM, neither can see the controller - Win7 doesn't even show anything unrecognized in the system management.

 

I am planning on replacing this controller either way (it only has two ports), but is it odd that ESXi can use it, but apparently not pass it through? Will a different - confirmed working - controller most likely still work?

Link to comment

Thanks :)

 

Keeping this thread on subject, I just moved an ASRock SATA3 (Marvell 88SE9128 / 88SE9120) from the old tower to check if ESXi would recognize it. It does and disks are read fine.

 

But when passing it through to either unRAID or a Win7 VM, neither can see the controller - Win7 doesn't even show anything unrecognized in the system management.

 

I am planning on replacing this controller either way (it only has two ports), but is it odd that ESXi can use it, but apparently not pass it through? Will a different - confirmed working - controller most likely still work?

I believe my Highpoint RocketRaid 622 controller is a Marvell 88SE9128 and that passes through to my Windows 7 x64 VMs just fine.  Using a port multiplier external drive cage for the drives connected to the RR622.  I could be wrong on the Marvell controller however will have to check.  My favorite controller for unRAID is the IBM M1015 because I could pick them up on ebay for 60-120 each.  If you want essentially the same thing but NEW then you could get a LSI-9211-8i which is what the M1015 becomes after flashing to IT firmware.

 

NOTE I am using ESXi 5.0.  Later versions had problems with pass through and they may have broken 88SE9128 pass through capabilities as well.  So you might try 5.0 if you are on later versions of ESXi.

Link to comment

Thought I'd give 5.0 a quick try, just to see if it would pass-through the Marvell controller; the initial setup wouldn't recognize any of the three connected controllers, so I think I'm gonna stick with 5.1u1 ;)

Correct 5.0 lists lots of things I pass through as "UNKNOWN UNKNOWN" but it passes through just fine.  My HVR-2250 and the HighPoint RocketRaid 622 cards both show up that way but pass through to the Windows VM just fine.  The only thing I pass through to unRAID is my M1015 and it is recognized as a LSI based controller.  Heck even the USB 2.0 and 3.0 cards I pass through to a Windows VM show up as unrecognized.  When I get home tonight I will post a graphic of my 3 ESXi servers pass through list so you can see what I have.

 

Long story short don't let the fact that ESXi doesn't recognize it determine your possibilities of pass through.  The only thing I've found so far that HASN'T passed through correctly was a PCI USB 3.0 card.  The PCIe version of that card works fine but the PCI one does not.

Link to comment

I meant the setup utility wouldn't even recognize any controller, so there was no disk available to install to (had one disk hooked up to each controller).

 

5.5 supports only the Lynx Point controller (don't know about the Marvell controller)

5.1u1 supports both the Lynx Point, ASM1062 and the 88SE9120, but can't pass-through the Marvell

5.0 apparently supports none

Link to comment

ESXi server 0:

SuperMicro X9SCM-F MB.  01:00.0 M1015, 02:00.0 Asm1061, 03:00.0 HVR-2250 & 04:00.0 USB 3.0

 

ESXi server1:

Tyan S5512GM2NR MB. 01:00.0 M1015, 02:00.0 & 02:00.1 RR622 (Marvell 99SE9128), 03:00.0 HVR-2250, 08:00.0 USB 3.0, 09:00.0 AVerMedia Duet, 0c:01.0 nVidia DualTV

 

ESXi server2:

Tyan S5512GM2NR MB. 01:00.0 M1015, 02:00.0 M1015, 03:00.0 & 03:00.1 RR622 (Marvell 99SE9128), 08:00.0 HVR-2250, 09:00.0 AVerMedia Duet, 0c:01.0 HVR-1600

 

So for me the ASM 1061 controller works and the 99SE9128 controller work.  So it may be due to your models since I see you said it was ASM 1062 & Marvel 88SE9120.  Note I also had a PCI-X controller based on Marvel 88SX6081 on pass through in ESXi server2.

 

The highlighting was just to show what the unknown controllers look like.

ESXi_server_0.png.17206845c3324f3e3c535c6c7b029599.png

ESXi_server_1.png.819050f2848bf6cadb713c7e6a116a96.png

ESXi_server_2.png.c38b10e1b25cea5ed351edae87859669.png

Link to comment

Hey Guys,

 

Im running esxi on an "asrock z77 pro4-m" and got the same problem i used these two vib files to prepare the esxi installation.

 

http://www.tinkertry.com/install-esxi-5-5-with-realtek-8111-or-8168-nic/

http://www.v-front.de/2013/11/how-to-make-your-unsupported-sata-ahci.html

 

On 5.5 everything in the passthrough section got listed perfectly!

And it was no problem to passthrough my m1015 to my unraid vm except the on board controlers.

 

It was my first setup until i found this thread now im gona try the 5.1u1 version.

 

Edit:12.12.13

 

Now esxi 5.1u1 is running. I inculuded the sata drivers from above to the installation iso. Everything in the passthrough section got listed corectly.

I was able to passthrough the ASM 1062 and the m1015(lsi2008) to the unraid vm without problems.

 

Thx 4 this ;-)

Link to comment

Hmm, seems like I'm either really unlucky that the ASRock card won't pass-through or there's something wrong with my setup.

 

It doesn't make sense 5.0 doesn't recognize any of the controllers...

 

I took the chance and ordered an M1015... As far as cross flashing goes, I get a lot of hits where people's been flashing it as a LSI-9240-8i, but you mentioned LSI-9211-8i. Any specific reason you're using that controller?

 

Do you happen to have a link of some sort regarding the cross flashing? :)

Link to comment

Hmm, seems like I'm either really unlucky that the ASRock card won't pass-through or there's something wrong with my setup.

 

It doesn't make sense 5.0 doesn't recognize any of the controllers...

 

I took the chance and ordered an M1015... As far as cross flashing goes, I get a lot of hits where people's been flashing it as a LSI-9240-8i, but you mentioned LSI-9211-8i. Any specific reason you're using that controller?

 

Do you happen to have a link of some sort regarding the cross flashing? :)

The firmware I used in on these forums.  I believe it will be the same firmware on the 9240 and 9211 (but I could be wrong) but in any case once flashed with the firmware in a thread on these forums it looks like the 92XX series LSI cards.
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.