UnRAID on VMWare ESXi with Raw Device Mapping

[BRiT]

It depends on what OS you use. Typically you would use VMWare Tools to access the other VMs remotely or you could use RDP (Remote Desktop) if you're running a WinOS system as the particular guest os.

[graywolf]

Her machine is Win XP.

 

Possible I wasn't clear on my question. Or I'm not understanding your answer.

 

I will have the ESXi server in room A and it will have a Win XP VM on it.

 

Is there something (other than a networked computer desktop/laptop) in room B to access the Win XP VM?

I'm thinking way back to the old dumb terminal type days.

 

I have my office space, and my wife has her office space (different rooms). So just trying to figure out if I can eliminate her physical computer somehow and have her run off a VM.

 

 

[Johnm]

No, she would still need some sort of client to connect. a PC or a thin client.

 

The Pc can be any windows pc that can run XP. so even a mini atom nettop, would work.

 

In the end I think you are stuck with her having a PC. but you can always keep the wife factor high by getting her some sort of SFF pc.

 

I know my GF would love a 2010 or 2011 mac mini with windows 7 on it (she dislikes osx but loves the look of the mac)

 

in a completely unlated, yet related topic..

there is a monitor (even dual screen) that has a thin client built in. but you need  VMware view and PC over IP (PoIP) for that.

PoIP is awsome stuff, fully hardare accelerated. it takes VM's to a whole new level.... but VERY expensive.

not for home use.

[EasyME]

By going with VMWare ESXI, you make the machine/pc you installing it on into a server and therefore you can't gain access directly to any guest os'es installed on it.

As a server you will have to connect to it via another computer on the same network (using a product called VMWare Vsphere client) to administer it (installing guest oses) /  or RDP to the guest already installed is guest is a WIndows OS or VNC for Non-Windows.

 

So in your case you would be looking at really 3 computers (1 for "esxi" the server , 1 for your office, 1 for your wife).

 

In the advance versions of VMWare (More costly models/products) they have where you can purchase dumb terminals like a DevonIT that will connect directly, but honestly a cheap PC does more a DUMB terminal and just because the word is "Dumb terminal" doesn't mean "A Cheap Terminal".

 

Disadvantages of a dumb terminal vrs a PC

1 - videos doesn't stream/ play well / so hi-def youtube is a no-no.

2 - sound is not transmitted well over IP (if it all)

3 - Action games just suck (very slow frame rates)

4 - Not Cheap.

 

 

ESXI for me isn't a replacement of my computer in the office but as an aid to it.

It serves as storage / stream component that is on 24hrs vrs my PC.

I have some additional guest os'es on ESXI like (web-server/ print-server / scanner / fax / Music / Video / Cloud Drive) these I don't shutdown.

 

so the long and short, you not eliminating a PC, but would be gaining a server instead.

 

Hope this helps.

 

[brian89gp]

How much money do you want to spend?  You will have a low WAF with RDP so that would leave PCoIP  

 

For PCoIP AMD has a vmdirectpath capable video card with PCoIP output for around $500-600. Then for the user side a standalone PCoIP client is around $350 (EVGA has a new one that is "cute") and Samsung has two monitors with the client built into the back for about $100 more.

 

I run a 400 user VMware View (software driven PCoIP) install at work and use one of the Samsung NC240 zero clients. Cool stuff if you are willing to pay for it. USB is capped at 1.1 so keep that in mind.

 

As far as quality, the NC240 monitor is a 23" wide screen at 1920x?? resolution. I have watched a 1080p video on it with no problems. This was on the software PCoIP (VM View) so the hardware driven one would be even better. PCoIP was built for this type of thing.

 

www.Teradici.com

VMware just licenses PCoIP tech and it existed a a direct 1-1 mapped client-workstation long before VMware started using it.

[graywolf]

well, guess that answers it. We keep the PC (client) machines.

 

But ESXi will still be a good addition for me since I'm wanting to have playgrounds for work related training/certification. And it'll give me a playground for unRaid 5 and leave all my media files on unRaid 4.7 for the time being.

 

 

[cipdor76]

I thought ESXi allowed one to pass through a physical device without the use of VT-d/IOMMU. Is that incorrect?

 

Also, semi-related to this, does anyone know of any Intel motherboards for 1155 socket / i7-2600 CPU that do support VT-d? I only see three from Intel but they don't support integrated video. I see some that claim to support it but in reality do not. [ http://siphon9.net/loune/2011/01/list-of-sandy-bridge-lga1155-h67p67-motherboards-that-support-vt-d/ ]

 

I am using a Supermicro X9SCA-F-O (Socket 1155, C204 chipset, integrated video + IPMI/KVM-over-IP) motherboard with a Xeon E3-1270 and 16GB of RAM. Supermicro AOC-USAS2-L8i (with IT firmware: 10.00.02.00 / BIOS: 7.19.00.00 - thanks to “madburg”) pass through directly to unRAID  (latest 5.0-beta12) via VT-d. USB Passthrough (not by VT-d). USB boot using modified “plpbt.iso”.

 

Works very nice, full temps and spindown, 100MB/s on Parity-Check.

 

VMware vSphere Hypervisor boot from an old WD Raptor 74GB. Data store on a Seagate SAS 16K.6 conencted to an Dell 6i/R SAS controller.

 

Second USB controller from the mobo pass through via VT-d to an Win7 x64 VM. Work great, real USB 2.0 speed.

 

First Intel NIC from the mobo used for vSwitch.

Second NIC from  the mobo pass through via VT-d, used as WAN in pfSense.

 

In total I have 4 VM:

- unRAID - unMenu, VMWare Tools

- Win7 Enterprise x64 – Airvideo server, Servetome server, FTP server  + some home automation

- pfSense 2.0

- ZoneMinder  1.24.4 on CentOS 5.6 for 8 IP cameras

 

[Kilack]

I thought ESXi allowed one to pass through a physical device without the use of VT-d/IOMMU. Is that incorrect?

 

Also, semi-related to this, does anyone know of any Intel motherboards for 1155 socket / i7-2600 CPU that do support VT-d? I only see three from Intel but they don't support integrated video. I see some that claim to support it but in reality do not. [ http://siphon9.net/loune/2011/01/list-of-sandy-bridge-lga1155-h67p67-motherboards-that-support-vt-d/ ]

 

I am using a Supermicro X9SCA-F-O (Socket 1155, C204 chipset, integrated video + IPMI/KVM-over-IP) motherboard with a Xeon E3-1270 and 16GB of RAM. Supermicro AOC-USAS2-L8i (with IT firmware: 10.00.02.00 / BIOS: 7.19.00.00 - thanks to “madburg”) pass through directly to unRAID  (latest 5.0-beta12) via VT-d. USB Passthrough (not by VT-d). USB boot using modified “plpbt.iso”.

 

Works very nice, full temps and spindown, 100MB/s on Parity-Check.

 

VMware vSphere Hypervisor boot from an old WD Raptor 74GB. Data store on a Seagate SAS 16K.6 conencted to an Dell 6i/R SAS controller.

 

Second USB controller from the mobo pass through via VT-d to an Win7 x64 VM. Work great, real USB 2.0 speed.

 

First Intel NIC from the mobo used for vSwitch.

Second NIC from  the mobo pass through via VT-d, used as WAN in pfSense.

 

In total I have 4 VM:

- unRAID - unMenu, VMWare Tools

- Win7 Enterprise x64 – Airvideo server, Servetome server, FTP server  + some home automation

- pfSense 2.0

- ZoneMinder  1.24.4 on CentOS 5.6 for 8 IP cameras

 

 

I have a very similar board to you minus the pci slots.  Currently its just running unraid but it seems a waste.  

With these bare metal virtualisations, do you need drivers at the host OS level?

eg I have a tv card that I would love to use that only has windows drivers so would have a host with windows.. would that still be possible ? Is there direct hardware access to the pci/pci-e slots through the bare metal virtualiser?

 

If so it sounds great.. no idea why I haven't done it earlier

 

Update: Ok so after some reading it seems virtualisation isn't quite there yet as far as video goes and pci cards etc.  It won't work with pci/pci-e video cards etc.  Hopefully one day.

So I guess I will continue to run two machines.

 

[znelbok]

simple question

 

How do I boot a VM into unraid.  I tried to get it to boot from the USB, but their bios does not support USB booting.  What have you guys been doing to get it to boot.  I have an idea, but it is so long winded that I believe their is a simpler way.

 

Mick

[cj0r]

Everything is in this thread, even the OP described a method for accomplishing it. However Johnm was nice enough to write up a definitive guide here: http://lime-technology.com/forum/index.php?topic=14695.0 .

[cipdor76]

simple question

 

How do I boot a VM into unraid.  I tried to get it to boot from the USB, but their bios does not support USB booting.  What have you guys been doing to get it to boot.  I have an idea, but it is so long winded that I believe their is a simpler way.

 

Mick

 

i use Plop - Boot Manager iso, you can find somewhere on this thread a modified iso image that boot directly from USB without wellcome-boot screen.

My unRAID vm have 1Gb of RAM and 1 CPU alocated + 100Mb of virtual disk (not important as long as you boot from USB) ... and it works like charm.

[cipdor76]

I thought ESXi allowed one to pass through a physical device without the use of VT-d/IOMMU. Is that incorrect?

 

Also, semi-related to this, does anyone know of any Intel motherboards for 1155 socket / i7-2600 CPU that do support VT-d? I only see three from Intel but they don't support integrated video. I see some that claim to support it but in reality do not. [ http://siphon9.net/loune/2011/01/list-of-sandy-bridge-lga1155-h67p67-motherboards-that-support-vt-d/ ]

 

I am using a Supermicro X9SCA-F-O (Socket 1155, C204 chipset, integrated video + IPMI/KVM-over-IP) motherboard with a Xeon E3-1270 and 16GB of RAM. Supermicro AOC-USAS2-L8i (with IT firmware: 10.00.02.00 / BIOS: 7.19.00.00 - thanks to “madburg”) pass through directly to unRAID  (latest 5.0-beta12) via VT-d. USB Passthrough (not by VT-d). USB boot using modified “plpbt.iso”.

 

Works very nice, full temps and spindown, 100MB/s on Parity-Check.

 

VMware vSphere Hypervisor boot from an old WD Raptor 74GB. Data store on a Seagate SAS 16K.6 conencted to an Dell 6i/R SAS controller.

 

Second USB controller from the mobo pass through via VT-d to an Win7 x64 VM. Work great, real USB 2.0 speed.

 

First Intel NIC from the mobo used for vSwitch.

Second NIC from  the mobo pass through via VT-d, used as WAN in pfSense.

 

In total I have 4 VM:

- unRAID - unMenu, VMWare Tools

- Win7 Enterprise x64 – Airvideo server, Servetome server, FTP server  + some home automation

- pfSense 2.0

- ZoneMinder  1.24.4 on CentOS 5.6 for 8 IP cameras

 

 

I have a very similar board to you minus the pci slots.  Currently its just running unraid but it seems a waste.  

With these bare metal virtualisations, do you need drivers at the host OS level?

eg I have a tv card that I would love to use that only has windows drivers so would have a host with windows.. would that still be possible ? Is there direct hardware access to the pci/pci-e slots through the bare metal virtualiser?

 

If so it sounds great.. no idea why I haven't done it earlier

 

Update: Ok so after some reading it seems virtualisation isn't quite there yet as far as video goes and pci cards etc.  It won't work with pci/pci-e video cards etc.  Hopefully one day.

So I guess I will continue to run two machines.

 

 

yes, it is realy a waste to use this mobo and a Xeon CPU only for unRAID.

if you pass through via VT-d a PCIe card - it will appear like a hardware (not virtualized) PCIe card to your windows vm - and yes of course you will need the windows driver for that card.

as regarding your TV card you can give it a try and let us know.

 

if it works, it will be very easy then to stream the TV chanels into your network - which is a very cool thing

for example now i'm using the Airvideo on windows vm to stream my Dreambox chanels to my Iphone/Ipad.

[zoltran]

Hi Folks

Bryan R described a way to boot Unraid of a virtual drive.

This works and is very fast for startup.

 

But I'm a bit confused on how you would update this.

For example ...adding plugins or even updating unraid itself

 

I tried adding in one of the plugins .. it worked fine until next restart.

And then i was back to basic/no plugins installed.

 

tia

Wes

[jortan]

I tried adding in one of the plugins .. it worked fine until next restart.

And then i was back to basic/no plugins installed.

 

Whatever command you ran to install the plugin, you need to add to your "go script" so it will be run each time unRAID starts up.  From the command line, do this:

 

echo "COMMAND HERE" >> /boot/config/go

 

Where COMMAND HERE is the command to run your plugin (keep it in quotes)

[madburg]

Zoltran, read the following post and see if it clears things up for you.

 

http://lime-technology.com/forum/index.php?topic=16270.0

 

[zoltran]

Thanks guys

I'll read up on it. 

 

This would apply to Zeron's VMTools as well ? 

 

 

[madburg]

Yeap

[znelbok]

Well, with a Supermicro X9SCM-F and a Xeon E3-1230 here, I tried to passthrough AOC-SASLP-MV8 to an unRAID VM, but like others without any luck, so ESXi will need to wait two BR10i cards I bought on Ebay. This fact corroborates those statements that not every PCIe peripheral can be successfully passed through VM's.

 

 

 

Well, I successfully made AOC-SASLP-MV8 work with ESXi 4.1 and VMDirectPath.

 

With "Remote Tech Support" enabled, use WinSCP to connect to ESXi, and add there two lines to the /etc/vmware/passthru.map file:

 

# Marvell Technologies, Inc. MV64460/64461/64462 System Controller, Revision B

11ab  6485  d3d0     false

 

Now open your VM's .vmx file and change this:

pciPassthru0.present = "TRUE"

pciPassthru0.deviceId = "6485"

pciPassthru0.vendorId = "11ab"

pciPassthru0.systemId = "4dfc27f9-93be-d5c1-9198-00259027d9d8"

pciPassthru0.id = "01:00.0"

 

to this:

 

pciPassthru0.present = "TRUE"

pciPassthru0.msiEnabled = "FALSE"

pciPassthru0.deviceId = "6485"

pciPassthru0.vendorId = "11ab"

pciPassthru0.systemId = "4dfc27f9-93be-d5c1-9198-00259027d9d8"

pciPassthru0.id = "01:00.0"

 

The catch is force the use of IOAPIC mode with the "pciPassthru0.msiEnabled = 'FALSE'" statement.

 

Reboot the hypervisor and start your unRAID VM!

 

Good luck.

 

Hi Guys

 

I unfortuantely have two SAS2LP cards that I want to use with ESXi5.

 

Can someone let me know what changes nee to be made to the above to get them to work in passthrough with Unraid.

 

Thanks

 

Mick

[pegounet]

hi,

I need some more help regarding real/unreal issues i have with my unraid vm config on ESXi

 

Here is my HW conf:

asus P8BWS (C206 chipset) mobo with xeon e3-1245 proc

adaptec 51245 raid card

a bunch of 1To and 2 To sata harddrives in backplanes

...

My SW conf is following: ESXi 5.0 with plop key for startup and Unraid key in trial conf

harddrives are direct-attached through VT-d passthrough

My unraid is version 5 beta14

 

here is the issues:

- no drive temp: is there a way to have it "captured" from a SMART command ?

- from time to time, either parity or one of the 2 other drives appear in red. I can change the drive and re-use it, so it doesn't seem to be a "real" problem with the drive

- disks make some noise from time to time (like it were broken but i doubt that most of them are)

- don't know if i can benefit from disk spin down ?

 

Can you tell me if these issues are coming from with unraid V5beta14 trial version or if they come from a configuration issue on my side or from a bad hardware choice (adaptec card - C206 chipset) and furthermore how to solve them?

You'll find enclosed printscreens of what i have with my system log.

 

Note that i can use Unraid for my data storage without any pb for a few weeks, but i need to be sure that what i'm seeing in Unraid configuration interface is ok or not?

Thanks in advance for your replies

Pierre

unraid-issues.zip

[hellbringer]

Thanks for the great guide, used it to install ESXi 5 on a Gigabyte GA-EP43-DS3 with 2 Dell SAS 6/iR controllers (flashed to LSI firmware in TI mode)

Works fantastic, spindowns, temps, etc and a parity check just takes about 12 hours (total array: 13.25TB, 12 data drives, 2 TB parity & 300GB cache drive) copying files from the server at about 40-50MByte/s.

The only problem is the temperature of the HD's that run of the onboard ICH10 controller cant be monitored from unRAID.

[pegounet]

To complete to my previous post: when i configure spinup/spindown delay (to 15 mn or 1hour...) in unraid interface, unraid tries to continuoulsy spinup/spindown my disks, every second... (this is the "disk noise" i was talking about) In log it appears to give that:

 

 

Jan 8 14:13:43 Tower kernel: sd 0:1:5:0: [sdc] 3900682240 512-byte logical blocks: (1.99 TB/1.81 TiB)

Jan 8 14:13:45 Tower kernel: mdcmd (2211): spindown 0

Jan 8 14:13:47 Tower kernel: mdcmd (2212): spindown 1

Jan 8 14:13:48 Tower kernel: mdcmd (2213): spindown 2

Jan 8 14:13:50 Tower kernel: sd 0:1:10:0: [sde] Spinning up disk...

Jan 8 14:13:52 Tower kernel: sd 0:1:8:0: [sdd] Spinning up disk...

Jan 8 14:13:56 Tower kernel: sd 0:1:5:0: [sdc] Spinning up disk....ready

Jan 8 14:13:56 Tower kernel: sd 0:1:8:0: [sdd] 1952428032 512-byte logical blocks: (999 GB/930 GiB)

Jan 8 14:13:56 Tower kernel: .ready

Jan 8 14:13:56 Tower kernel: sd 0:1:10:0: [sde] 3900682240 512-byte logical blocks: (1.99 TB/1.81 TiB)

Jan 8 14:13:59 Tower kernel: .ready

Jan 8 14:13:59 Tower kernel: sd 0:1:5:0: [sdc] 3900682240 512-byte logical blocks: (1.99 TB/1.81 TiB)

Jan 8 14:14:00 Tower kernel: mdcmd (2214): spindown 0

Jan 8 14:14:03 Tower kernel: mdcmd (2215): spindown 1

Jan 8 14:14:03 Tower kernel: mdcmd (2216): spindown 2

Jan 8 14:14:06 Tower kernel: sd 0:1:10:0: [sde] Spinning up disk...

Jan 8 14:14:08 Tower kernel: sd 0:1:8:0: [sdd] Spinning up disk...

Jan 8 14:14:11 Tower kernel: sd 0:1:5:0: [sdc] Spinning up disk....ready

Jan 8 14:14:11 Tower kernel: sd 0:1:8:0: [sdd] 1952428032 512-byte logical blocks: (999 GB/930 GiB)

Jan 8 14:14:12 Tower kernel: .ready

Jan 8 14:14:12 Tower kernel: sd 0:1:10:0: [sde] 3900682240 512-byte logical blocks: (1.99 TB/1.81 TiB)

Jan 8 14:14:14 Tower kernel: .ready

Jan 8 14:14:14 Tower kernel: sd 0:1:5:0: [sdc] 3900682240 512-byte logical blocks: (1.99 TB/1.81 TiB)

Jan 8 14:14:16 Tower kernel: mdcmd (2217): spindown 0

Jan 8 14:14:19 Tower kernel: mdcmd (2218): spindown 1

Jan 8 14:14:19 Tower kernel: mdcmd (2219): spindown 2

Jan 8 14:14:22 Tower kernel: sd 0:1:10:0: [sde] Spinning up disk...

Jan 8 14:14:24 Tower kernel: sd 0:1:8:0: [sdd] Spinning up disk...

 

 

When i give unraid a smart command "smartctl  -a  -d  ata  /dev/sda" from unraid virtual console i have following message:

"smartctl:device read identity failed (not an ata/atapi device). A mandatory SMART command failed: exiting. "

 

Do i have to change my raid card and/or my mobo card to allow supported spindown and/or disk temp ?

[pegounet]

Don't know if i'm in the right area to post... is there any support thread for virtualisation in this forum?

Nevertheless i continue on my topic, hoping some help. Find enclosed the result of smartctl -a -A /dev/sdc command

Seems that it's hopeless to have correct behaviour for my unraid array? Is this an unraid version (software issue) or hardware? (i'm in V5b14)

 

Pierre

 

[Johnm]

Don't know if i'm in the right area to post... is there any support thread for virtualisation in this forum?

Nevertheless i continue on my topic, hoping some help. Find enclosed the result of smartctl -a -A /dev/sdc command

Seems that it's hopeless to have correct behaviour for my unraid array? Is this an unraid version (software issue) or hardware? (i'm in V5b14)

 

Pierre

 

the easiest way to answer your own question is to  bypass the ESXi and boot straight to your unraid flash and see if the same behavior still exists.

 

there is a known bug in this build with LSI cards. if you spin them down and try to spin them back up, drives do start to redball as you mentioned in a previous post. have you tried rolling back to an older beta? perhaps your adaptec card is also suffering from this.

[pegounet]

thanks for your advise johnm of testing without VM layer. will do that

fyi i had these issues when in v5b12

 

Do you think my adaptec card and asus mobo would be able to provide me with Temp/spindown? Should i need to configure in Vsphere client my disk controller in a specific way (LSI or paravirtualized?), cause i've let it configured by default at setup?

[Johnm]

I forgot to ask. Are you using RDM or passthrough via VT-d?  That will make a difference as far as the temp readings.

 

Unfortunately I am not familiar with your raid card so I can't give you any solid answers based on the card.