I have been reading through various forums on here and all over the web on Xen and wanted to get a feel for how other users are setting up Xen with unRAID.

 

My goal to to have one computer with unRAID, some linux server (for CB/SB ect.), pfSense, and Windows.

 

So far I have been able to successfully get a test server setup with unRAID as a Dom0 and an Ubuntu server as a DomU which is working really well. I am having a very difficult time attempting to get a pfSense DomU started. I have been attempting to follow guides using freeBSD as it should be a similar install for pfSense; however, the guides are using LVM (logical volume) which I have read is preferred over a partitioned image file (.img). As I understand it, unRAID (as it stands) does not support LVM's. With that being said below are my laundry list of questions 

 

1. With my goals of getting all these systems setup, would it be easier and/or better to make Linux my Dom0 and have unRAID as a Domu?

2. Are there any limitations / concerns for using unRAID as a DomU?

3. Is it better to run VM's from LVM's instead of IMG partitions?

 

Thank you for reading and any other suggestions are welcome.

 

1. With my goals of getting all these systems setup, would it be easier and/or better to make Linux my Dom0 and have unRAID as a Domu?

2. Are there any limitations / concerns for using unRAID as a DomU?

3. Is it better to run VM's from LVM's instead of IMG partitions?

[\QUOTE]

 

 

I think you mixing up the setup here.

what guides?  the guides for pfSense? than it refers to using LVM inside the VM and have nothing to do with unRaid.

unRaid does not support LVM because it uses different logic to create and maintain the array.

 

now for your questions:

Q1. if it will be easier it will not be by much.

      the only real difficulty with using unraid as Dom0 is creating and managing the    VMs as unRaid have no real managing interface. all is CLI and text editors.

 

that also leads as to

Q2. in order to properly run unRaid as DomU and get the full benefits.

you MUST have hardware that supports VT-D /IOMMU 100%

meaning BOTH of your CPU and MotherBoard MUST implement and fully support it.

VT-D is an Intel virtualization technology

and IOMMU is for AMD.

the many issues arise because some manufacturing have the support of the tech listed but not fully implemented thus you really need to do your research very very carefully

you need this support in order to pass-through the disc controller(s) into VM to

give unRaid full direct access to the controller and disks.

also you need to pass through your USB port where the unraid flash is to run with a license(free setup can run with virtualized USB port).

 

 

Q3.  I am not sure what you mean with LVM vs IMG

I think they mean where the VM image is stored/run from and in your setup it dose not really matter. specifically IF you will run unRaid as dom0.

 

 

 

 

 

 

I was attempting to follow this guide:

 

http://wiki.sysconfig.org.uk/display/howto/Xen+FreeBSD+8.2+DomU+(PV)+--+Step+by+Step+Howto

 

What has me tripped up is the meaning of

'phy:/dev/vg0/freebsd,hdb,w',

and

lvcreate -n freebsd -L5G vg0

 

Then I was looking at other users config files from here http://lime-technology.com/forum/index.php?topic=31795.msg300089#msg300089 and it seemed like these users were running unRAID as a DomU which made me think that is how I should setup mine system.

Please keep in mind, a pfsense under a Dom0 setup - when unRaid is taken offline you WILL loose internet.

 

I have a http://store.netgate.com/ pfsense appliance with wireless in it as well.  Running from a 4G 2 partition compaq flash card - have had this unit for aprox 7-8 years now as is still running just like i got it on day one.

 

I like being able to clone the working partition to the 2nd partition, boot into 2nd partition - apply any upgrades to test etc - if problems I can just boot back into the primary and my system is back up.

 

This unit also has 3 eth ports, so I can dual WAN, and the 3rd go into my network switch here at the house.  Some programs are setup to use a specific wan port to split up traffic between the two isp.

 

Myk

 

Then I was looking at other users config files from here http://lime-technology.com/forum/index.php?topic=31795.msg300089#msg300089 and it seemed like these users were running unRAID as a DomU which made me think that is how I should setup mine system.

 

I don't see where you got that people were running unraid as DomU

but I did not read the whole thread though.

 

not that is is not possible mind you, but everyone situation is different.

and as I said in my last post you do need a compatible hardware to do so.

also the freeBSD  guide is too old, it's still based on Xen 3.4 .

lots of things changed since than.

 

I would say, just create a generic VM.

get the proper image(ISO) here https://www.pfsense.org/download/mirror.php?section=downloads

 

add the ISO as a CD drive to VM

and simply install it as it were a real machine. that what I did for my virtualBox test run.

 

just make sure you create proper bridge interfaces and attache those to the VM.

 

you need 2 interface(NIC) WAN and LAN

and if you connecting the WAN to the ISP modem device the MAC address MUST be something real not the Xen generated one.

 

 

Does my test machine have to have (VT-x) support in order for me to get pfsense installed? I understand that I will not be able to actually use pfsene without VT-x and VT-d but I was hoping to at least get it configured before I started messing around with my main server.

actually you can use it without VT-x or VT-d.

you need VT-d to pass-through the nics if you want.

but you can simply create bridges to the nics you want to use and attach the bridge to VM.

 

OK, that good to know. I am reading the the forum on http://lime-technology.com/forum/index.php?topic=31716.0 and it looks like pfsense will need to be run as a HVM Guest?

probobly,

 

while you are checking the pfSense check out Sophos UTM as well.

I am planning the hardware build for that one.

Well I tried to install pfsense from the iso but I got

Parsing config from /mnt/disk1/apps/VM/pfsense/pfsense.cfg
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware
libxl: error: libxl_create.c:416:libxl__domain_make: domain creation fail
libxl: error: libxl_create.c:644:initiate_domain_create: cannot make domain: -3
libxl: error: libxl.c:1383:libxl__destroy_domid: non-existant domain -1
libxl: error: libxl.c:1347:domain_destroy_callback: unable to destroy guest with domid 4294967295
libxl: error: libxl_create.c:1209:domcreate_destruction_cb: unable to destroy domain 4294967295 following failed creation

 

So I decided to try and install Windows 7 (as that is where I got the cfg file from) to see if I could get that started and got the same error message. I posted the error in the window 7 vm page as well so hopefully someone knows why I am getting that error message.

are you passing anything through?

it looks like an error I got when I try to pass through my controller

on an noncompliance hardware.

I do not think so but here is my .cfg

 

name="pfsense"
kernel = 'hvmloader'
builder="hvm"
vcpus = '1'
memory=512
disk = ['file:/mnt/disk1/apps/VM/pfsense/pfsense.img,hda,w' , 'file:/mnt/disk1/apps/VM/pfsense/pfSense-LiveCD-2.1.2-RELEASE-amd64-20140410-0541.ISO,hdc:cdrom,r']
vif = [ 'mac=00:16:3e:01:01:02,bridge=xenbr0' ]
boot="dc"
acpi = '1'
apic = '1'
viridian = '1'
xen_platform_pci='1'
sdl = '0'
vnc = '1'
vnclisten = '0.0.0.0'
vncpasswd = ''
usb = '1'
usbdevice = 'tablet'  
on_poweroff="destroy"
on_reboot="restart"
on_crash="destroy"

Try removing the "kernel = 'hvmloader'" line that probably won't fix the problem but WILL get rid of the WARNING.

 

Here is my WHS2011 cfg for comparison:

name="WHS2011"
builder="hvm"
vcpus=2
memory=3072


disk = ['file:/mnt/cache/domains/WHS2011/whs2011.img,hda,w']


boot="dc"


acpi = '1'
apic = '1'
viridian = '1'
xen_platform_pci='1'
sdl = '0'
vnc = '1'
vnclisten = '0.0.0.0'
vncpasswd = ''
usb = '1'
usbdevice = 'tablet'  
localtime = 1


on_poweroff="destroy"
on_reboot="restart"
on_crash="destroy"


pci=['0f:00.0','04:00.0','05:00.0']

It is booting fine.

 

Wish the code tags wouldn't add extra space for a blank line

removing  "kernel = 'hvmloader'" does remove the warning but does not fix the issue.

 

Parsing config from pfsense.cfg
libxl: error: libxl_create.c:416:libxl__domain_make: domain creation fail
libxl: error: libxl_create.c:644:initiate_domain_create: cannot make domain: -3
libxl: error: libxl.c:1383:libxl__destroy_domid: non-existant domain -1
libxl: error: libxl.c:1347:domain_destroy_callback: unable to destroy guest with domid 4294967295
libxl: error: libxl_create.c:1209:domcreate_destruction_cb: unable to destroy domain 4294967295 following failed creation

Well it is defiantly an issue with my test server. I just tried the same settings on my main server and installed pfSense just fine.

you need 2 interface(NIC) WAN and LAN

and if you connecting the WAN to the ISP modem device the MAC address MUST be something real not the Xen generated one.

 

 

When you say 2 interfaces, do you mean I will need 2 Nics in total ie (1 onboard and 1 PCI) or will I need 3 (2 for pfsense and 1 for unraid / VM's)? My goal would be to have a modem connect to the unraid box -> pfsense takes that as WAN -> pfsense redirects all traffic through VPN (I already have this setup on a separate pfsense box) -> unRAID VM's are forced to use pfsense settings.

 

So, I am trying to determine if I will need 3 NIC's ie (PCI#1 for pfsense WAN, PCi#2 for pfsense Lan, Onboard #1 for unRAID/ VM's) or can I get away with 2 NIC's ie. (PCI#1 for pfsense WAN, Onbaord #1 for pfsense LAN/VM's)?

I have read that it is possible to set this up with only 2 NICs but more often the suggestions are to have 3.

 

1 for pfSense WAN

1 for pfSense  LAN

1 for host LAN

 

so you would have  2 NICs dedicated to pfSense VM

1  NIC connected to WAN(your ISP)

1 NIC to out put LAN into main switch

 

and 3rd. would be your server interface connected to switch as usual.

 

if your MB have one on-board just get an Intel dual port card.

if your hardware supports VT-d/IOMMU simply pass-through the whole card into VM

if not, the what seams to be the best option is to create the bridge for each interface and add the bridges to the VM. people suggest that it makes easier to swipe the hardware cards if needed, as you only need to change the bridge configuration to point to the new card, and restart the VM.