[GUIDE] Virtualizing unRAID in XenServer 6.2 - Updated to 5.0.4

[BobPhoenix]

 

 

I'm debating on going the Virtualizing unRAID in KVM on openSUSE route, as it seems more user-friendly in the end, will run nicely on my USB flash, and should recognize the two SSDs I would like to RAID 1 for datastores using the onboard RAID.

 

you might be better off using Linux software RAID rather than MB raid.

also with opensuse you can try RAID-1 using BTRFS volume instead of MD RAID

 

What's the advantage of using Linux software RAID over the MB raid controller?

 

Have to admit, unTER had the same idea as me - two 120GB SSD's, in RAID 1 for a datastore.  I had it listed as a pro, that the X9SCM had a second SATA controller, which was capable of RAID1 - maybe I'm not seeing a pitfall here 

The MB software raid (MB raid is not hardware raid like an Areca card for example) would be proprietary to the MB controller/bios.  Linux software raid works with any controller.  So you would be able to transfer your disks from AMD MB to Intel MB with a Linux software raid but maybe not with the MB raid.

[TheDragon]

 

 

I'm debating on going the Virtualizing unRAID in KVM on openSUSE route, as it seems more user-friendly in the end, will run nicely on my USB flash, and should recognize the two SSDs I would like to RAID 1 for datastores using the onboard RAID.

 

you might be better off using Linux software RAID rather than MB raid.

also with opensuse you can try RAID-1 using BTRFS volume instead of MD RAID

 

What's the advantage of using Linux software RAID over the MB raid controller?

 

Have to admit, unTER had the same idea as me - two 120GB SSD's, in RAID 1 for a datastore.  I had it listed as a pro, that the X9SCM had a second SATA controller, which was capable of RAID1 - maybe I'm not seeing a pitfall here 

The MB software raid (MB raid is not hardware raid like an Areca card for example) would be proprietary to the MB controller/bios.  Linux software raid works with any controller.  So you would be able to transfer your disks from AMD MB to Intel MB with a Linux software raid but maybe not with the MB raid.

 

That makes sense, thanks for pointing that out BobPhoenix.  I know in my planned setup, I have already filled all the PCIe slots - in my mind anyway!!  So hardware RAID for the datastore would be out of the question.  Do you know if there would be any significant performance difference between Linux software raid vs MB raid?

[vl1969]

 

 

I'm debating on going the Virtualizing unRAID in KVM on openSUSE route, as it seems more user-friendly in the end, will run nicely on my USB flash, and should recognize the two SSDs I would like to RAID 1 for datastores using the onboard RAID.

 

you might be better off using Linux software RAID rather than MB raid.

also with opensuse you can try RAID-1 using BTRFS volume instead of MD RAID

 

What's the advantage of using Linux software RAID over the MB raid controller?

 

Have to admit, unTER had the same idea as me - two 120GB SSD's, in RAID 1 for a datastore.  I had it listed as a pro, that the X9SCM had a second SATA controller, which was capable of RAID1 - maybe I'm not seeing a pitfall here 

The MB software raid (MB raid is not hardware raid like an Areca card for example) would be proprietary to the MB controller/bios.  Linux software raid works with any controller.  So you would be able to transfer your disks from AMD MB to Intel MB with a Linux software raid but maybe not with the MB raid.

 

That makes sense, thanks for pointing that out BobPhoenix.  I know in my planned setup, I have already filled all the PCIe slots - in my mind anyway!!  So hardware RAID for the datastore would be out of the question.  Do you know if there would be any significant performance difference between Linux software raid vs MB raid?

 

I think Linux raid  might be faster sometimes, but not sure 100%

[BobPhoenix]

 

 

I'm debating on going the Virtualizing unRAID in KVM on openSUSE route, as it seems more user-friendly in the end, will run nicely on my USB flash, and should recognize the two SSDs I would like to RAID 1 for datastores using the onboard RAID.

 

you might be better off using Linux software RAID rather than MB raid.

also with opensuse you can try RAID-1 using BTRFS volume instead of MD RAID

 

What's the advantage of using Linux software RAID over the MB raid controller?

 

Have to admit, unTER had the same idea as me - two 120GB SSD's, in RAID 1 for a datastore.  I had it listed as a pro, that the X9SCM had a second SATA controller, which was capable of RAID1 - maybe I'm not seeing a pitfall here 

The MB software raid (MB raid is not hardware raid like an Areca card for example) would be proprietary to the MB controller/bios.  Linux software raid works with any controller.  So you would be able to transfer your disks from AMD MB to Intel MB with a Linux software raid but maybe not with the MB raid.

 

That makes sense, thanks for pointing that out BobPhoenix.  I know in my planned setup, I have already filled all the PCIe slots - in my mind anyway!!  So hardware RAID for the datastore would be out of the question.  Do you know if there would be any significant performance difference between Linux software raid vs MB raid?

Personally I wouldn't bother with MB raid then it is portable if your MB dies.  Don't think there would be any performance differences but if anything the Linux raid should be the same or faster since they are both just software based raid.

[TheDragon]

 

 

I'm debating on going the Virtualizing unRAID in KVM on openSUSE route, as it seems more user-friendly in the end, will run nicely on my USB flash, and should recognize the two SSDs I would like to RAID 1 for datastores using the onboard RAID.

 

you might be better off using Linux software RAID rather than MB raid.

also with opensuse you can try RAID-1 using BTRFS volume instead of MD RAID

 

What's the advantage of using Linux software RAID over the MB raid controller?

 

Have to admit, unTER had the same idea as me - two 120GB SSD's, in RAID 1 for a datastore.  I had it listed as a pro, that the X9SCM had a second SATA controller, which was capable of RAID1 - maybe I'm not seeing a pitfall here 

The MB software raid (MB raid is not hardware raid like an Areca card for example) would be proprietary to the MB controller/bios.  Linux software raid works with any controller.  So you would be able to transfer your disks from AMD MB to Intel MB with a Linux software raid but maybe not with the MB raid.

 

That makes sense, thanks for pointing that out BobPhoenix.  I know in my planned setup, I have already filled all the PCIe slots - in my mind anyway!!  So hardware RAID for the datastore would be out of the question.  Do you know if there would be any significant performance difference between Linux software raid vs MB raid?

Personally I wouldn't bother with MB raid then it is portable if your MB dies.  Don't think there would be any performance differences but if anything the Linux raid should be the same or faster since they are both just software based raid.

 

Think I'll be doing as you suggested! Thanks for sharing your wisdom, not the first time you've helped me out ;-)

 

Sent from my Nexus 7 using Tapatalk 4

[johnodon]

For those who missed it...

 

XS 6.2 SP1 released:  http://support.citrix.com/article/CTX139788

 

What's New in This Service Pack

This Service Pack delivers the following new functionality:

 

The 3D Graphics Pack supporting NVIDIA GRID GPUs

Support for Windows 8.1 and Windows Server 2012 R2

Improvements to the Site Recovery wizard for large deployments

This Service Pack also resolves a number of issues described in CTX139789 – XenServer XS62ESP1 Release Notes.

 

In addition, this Service Pack includes the following previously released hotfixes.

 

CTX138186 – Hotfix XS62E001 - For XenServer 6.2.0

CTX138349 – Hotfix XS62E002 - For XenServer 6.2.0

CTX138833 – Hotfix XS62E004 - For XenServer 6.2.0

CTX139299 – Hotfix XS62E005 - For XenServer 6.2.0

CTX139308 – Hotfix XS62E009 - For XenServer 6.2.0

CTX139643 – Hotfix XS62E010 - For XenServer 6.2.0

CTX139776 – Hotfix XS62E011 - For XenServer 6.2.0

CTX139749 – Hotfix XS62E012 - For XenServer 6.2.0

[coppit]

Many thanks for all the great info! Some comments and questions...

 

Xen is a fully mature robust Type 1 Hypervisor that runs on Linux and has always been open sourced.

 

For a while XCP was the open source version, and lagged Xen Server in features and fixes. But as of last year Xen Server is also open source (and XCP end-of-life'd). Yay!

 

get a motherboard like mine where you aren't forced to passthrough all your onboard sata ports

 

How can I determine which motherboards support selective passthrough?

 

Unless you have a motherboard that supports USB passthrough... Your unRAID VM will not be able to read the Plus / Pro unRAID key.

 

Ditto. How can I know?

 

Basically I'm looking for a good Xeon motherboard that meets the above requirements. Anyone have a recommendation?

 

why even bother with a cache drive

 

You'd still want it for performance, right?

 

Currently I have a separate Windows PC that is my gaming rig and also my PVR (with 2 network tuners). I'd like to build a new server, have UNRAID running in one VM, and move the TV recording work to a Windows VM. On top of that I'd like to add security cam recordings. I don't think I need a new VM for that, but I was thinking of saving the security video to a separate disk, since I'm worried about I/O contention. (Up to two MPEG2 streams from my HDHomeRun, plus 3 lower-quality H.264 security cam streams, at once.) Since I don't care to parity-protect this video, and since I want full speed, I was thinking of this setup:

 

• Xen: VMs stored on 128GB SSD. I assume we want won't want to compromise performance with a spinning disk
  
• Windows VM: SageTV (PVR software) and Sighthound (security software) both running, saving to 2 different unprotected, passthrough spinning hard disks. 1 dedicated NIC, passthrough to the VM.
  
• UNRAID VM: USB drive, parity and 9 data disks, 128GB SSD for cache drive. All disks via passthrough. NIC shared with Linux VM
  
• Linux VM: CentOS, airvideo server, and other stuff that's currently running in the unraid OS.
  

 

Does this setup make sense? I really like unmenu, but I think it would be better to leave the unraid VM pristine and use a full CentOS distribution for everything else. One thing I worry about though is that I don't get to leverage all the hard work others have put into more traditional setups. i.e. I'm going to be building source and futzing around more than if I could simply install some unmenu package and then configure it though the UI.

[kunkler99]

I want to first say thank you for all the help and information you have provided us all. Secondly I have a question about my eth0 connection. It seems that I can not get it to connect at 1000 at all on any of my machines. I have two, one connects at 100 and the other at 10. I have tried to run ethtool -s can it does not return anything at all. I have a feeling this is a driver issue. Is there a xen driver like the one needed for windows (gplpv )? If so how can i install the correct drivers.

 

Thanks! Scott

[ChrisPlak]

This is a great guide.  THANK YOU. 

 

Couple of items of note (maybe I missed them in another post)

 

- I needed to pass through my USB as well as my sata.  When I passed through the USB unraid was then able to get the uuid from the usb stick and thus validate my license.  Note that no other VM's are able to access the USB ports that you assign to unraid.  This worked out well as I gave unraid my UPS as well.  (XenServer should be passing through the USB GUID and you should not have to take ownership of the USB ports.  I'd consider this a bug.)

 

- In the step where you need to pass in the PCI devices (xe vm-param-set other-config:pci=0/0000:XX:XX.X,0/0000:XX:XX.X,0/0000:XX:XX.X uuid=YOUR UUID FOR YOUR VM HERE) you MUST include all PCI devices that you want to pass through on one single call.  Initially, I added only my USB pci devices and saw that my license was correctly validated.  I then added my sata pci devices and I lost my USB pass through.  I had to go through the entire process specifying the entire list on that one call.

 

- If you happen to run into problems where you are not seeing all devices in the vm-param-set options, change the 0/0000 where you increase the first 0/ for each entry.  Basically the "0/" portion of "0/0000 " tells it where in the list to put that pci device.  So long as they are on different ports it won't matter but for items on the same it may.  So change it to "0/0000", "1/0000", "2/0000" and so on. 

 

- This will only work with a VT-d or IOMMU processor.  Search Intel's ARK site for cpu's that support it.  "K" processors from Intel generally do not support vt-d, there are a few exceptions.

 

- Unraid performance is identical as when running without XenServer.

 

 

[dirtysanchez]

- This will only work with a VT-d or IOMMU processor.  Search Intel's ARK site for cpu's that support it.  "K" processors from Intel do not support vt-d.

 

In general K-series SKU's do not support VT-d, but there are a few that do, such as the i7-4930K and the i7-4820K.

[gtroyp]

If you wanted to upgrade/test unRaid 6.x-b as a domU in Xenserver, would I need to do anything more that upgrade the usb drive copy of bzroot etc.?

[joshpond]

Okay, I'm now testing xenserver and xencenter at the moment. I see at the start of the guide there is a local storage SR. They recommend against this mainly for xenmotion and redundancy if the xenserver host machine fails.

I'm assuming for our purposes and only running one machine local storage is fine?

The other question then is how do I create a new SR with the space left over from the xenserver install (sda3)? Most things I can search talk about adding a new drive, not use an existing drive.

Thanks Josh

 

Sent from my GT-N7000 using Tapatalk

 

 

[joshpond]

Okay,

 

I figured out the problem. When I installed there was no Local Storage being created. Thanks to this post:

http://discussions.citrix.com/topic/315298-no-local-storage-repository-after-clean-install/

As I was previously using the HDD in a Raid array, there was some stuff on it that was stopping XenServer from writing to it.

Josh

[elBradford]

- This will only work with a VT-d or IOMMU processor.  Search Intel's ARK site for cpu's that support it.  "K" processors from Intel generally do not support vt-d, there are a few exceptions.

 

I hope not. According to xen.org's documentation

 

On systems without an IOMMU, devices can be passed through to trusted PV guests, but doing so removes the security or stability advantages (though not the performance advantages).

 

The kernel provided in this thread has PV support, so this should work without VT-d.

 

SATA pass-through on non-VT-d hardware was the reason I am trying out Xen. I used ESXi and thought it was fine, except it required VT-d for any hardware pass-through (I bought a k processor that doesn't support VT-d).

 

After I assign my PCI device (the on-board SATA controller) to my unRAID guest, I try to start it and get the error:

 

Feb 13, 2014 11:40:12 PM Error: Starting VM 'STORAGE - unRAID' - Internal error: xenopsd internal error: Xenctrl.Error("38: Function not implemented")

 

And it doesn't boot. What does this mean? I can't find any other details about the error, and it only appears when I follow Grumpy's step:

 

xe vm-param-set other-config:pci=0/0000:XX:XX.X uuid=YOUR UUID FOR YOUR VM HERE

 

Grumpy, have any ideas? Your kernel is PV-enabled, so this shouldn't be a VT-d issue. Thank you!

[elBradford]

In addition to my last post, I wonder if you have included the pciback module in the kernel? That is apparently required for PV passthrough to work without VT-d.

[elBradford]

After reading through this a second time, it looks like this is all under the assumption of VT-d, which is a shame because it isn't necessary as far as I can tell. I may come back someday and try XenServer and unRAID again, but I'll be going back to ESXi for now, I've wasted way too much time trying to get this to work.

[vl1969]

yes this is all assumes that your hardware supports VT-d

but ESXi also require VT-d support for  pass-through to virtualize unRaid

so what's the diff?

 

as far as I understand if you can do pass-through on your hardware in ESXi

XenServer should work as well.

[elBradford]

yes this is all assumes that your hardware supports VT-d

but ESXi also require VT-d support for  pass-through to virtualize unRaid

so what's the diff?

 

as far as I understand if you can do pass-through on your hardware in ESXi

XenServer should work as well.

 

ESXi supports RDM (Raw Device Mapping) for hard drives without VT-d, up to 2TB for VM 8, and >2TB for VM 10. Since the only way to manage VM10 machines running on ESXi is to use vSphere Web Client (which is overkill for a single server like mine), I am still using VM 8, which, again only supports 2TB drives in RDM and smaller. HOWEVER, ESXi 5.5 supports VMDK of >2TB, so my 2TB drives are RDM and my 4TB drive is using VMDK. Kind of a frankenserver but I can't spin my wheels much longer with this, and I don't have to buy a VT-d processor in order for it to work.

 

This could all work with unRaid as far as I can tell, without the need for VT-d.  From my reading you don't need VT-d for PCI Passthrough with XenServer as long as the guest is paravirtualized, which unRAID is. You just need to compile the pciback module with the unRAID kernel. Hopefully Grumpy will do that for the future. In fact, PV guests not needing VT-d was the MAIN reason I scrapped my ESXi server and tried to go with XenServer. For now I'll use my Server Essentials 2012 R2 license with storage spaces for my NAS. Overkill, yes, but I got it working rather quickly. Someday I will be able to give unRAID a shot and I look forward to it.

[vl1969]

yes this is all assumes that your hardware supports VT-d

but ESXi also require VT-d support for  pass-through to virtualize unRaid

so what's the diff?

 

as far as I understand if you can do pass-through on your hardware in ESXi

XenServer should work as well.

 

ESXi supports RDM (Raw Device Mapping) for hard drives without VT-d, up to 2TB for VM 8, and >2TB for VM 10. Since the only way to manage VM10 machines running on ESXi is to use vSphere Web Client (which is overkill for a single server like mine), I am still using VM 8, which, again only supports 2TB drives in RDM and smaller. HOWEVER, ESXi 5.5 supports VMDK of >2TB, so my 2TB drives are RDM and my 4TB drive is using VMDK. Kind of a frankenserver but I can't spin my wheels much longer with this, and I don't have to buy a VT-d processor in order for it to work.

 

This could all work with unRaid as far as I can tell, without the need for VT-d.  From my reading you don't need VT-d for PCI Passthrough with XenServer as long as the guest is paravirtualized, which unRAID is. You just need to compile the pciback module with the unRAID kernel. Hopefully Grumpy will do that for the future. In fact, PV guests not needing VT-d was the MAIN reason I scrapped my ESXi server and tried to go with XenServer. For now I'll use my Server Essentials 2012 R2 license with storage spaces for my NAS. Overkill, yes, but I got it working rather quickly. Someday I will be able to give unRAID a shot and I look forward to it.

 

Well you can also wait for unraid 6.

Since it is build to be a vm host

You will not need to do pass through on it

 

Sent from my SGH-T889 using Tapatalk

 

 

[ezbox]

I followed your guide and migrated a server from ESXI to XenServer 6.2. 

 

My questions are:

 

Is there a way to install Xenserver tools in Unraid?

How difficult is to create our own Unraid template file to import into Xenserver, example make a 5.05 version?

 

Thanks!

 

EzBox

 

[el-tel]

It appears that I am struggling to passthrough everything I need in order to get my server up and running.

 

I managed to passthrough a USB controller and an AOC-SASLP-MV8 controller with no problems, although I did have to change the command below from:

xe vm-param-set other-config:pci=0/0000:00:1d.0,0/0000:01:00.0 uuid=9a998389-50ce-9395-14cd-16d9fe7085ea

 

to include a 1 before the 2nd forward-slash:

xe vm-param-set other-config:pci=0/0000:00:1d.0,1/0000:01:00.0 uuid=9a998389-50ce-9395-14cd-16d9fe7085ea

 

However, I can't seem to passthrough a second SATA controller (AOC-SAS2LP-MV8). It just doesn't get recognised when starting up the unRAID VM. I have tried a number of variations of the vm-param-set command by reordering the devices in the list:

 

xe vm-param-set other-config:pci=0/0000:01:00.0,1/0000:05:00.0,2/0000:00:1d.0 uuid=9a998389-50ce-9395-14cd-16d9fe7085ea

 

I even tried removing the USB controller passthrough altogether and it still fails to recognise the 2nd controller. Interestingly it is always the 2nd controller in the list that fails.

 

Server OS: XenServer 6.2.0

unRAID: v5.0.4

Motherboard: Asrock Z87M Pro4

 

I hoped that somebody here may be able to help or have some ideas on what it might be. Or failing that perhaps I should go with the unRAID v6 beta. Or even try Xen running on Debian.

 

Many thanks in advance.

[apgood]

Try passing through the pcie slot instead of just the card. I had a similar problem where I could pass through a sata card so it tried passing through the pcie slot and that worked.

 

Sent from my Nexus 4 using Tapatalk

 

 

[el-tel]

Many thanks for your reply apgood.

 

I shall try that, but can you assist in the commands I need to use to do that?

[apgood]

It's been many months since I did it but from memory I just used the lspci command which gives a listing of all the devices. You'll see that there is a device ID for each of the pcie slots. What I did was work out which pcie slot had the data card in it and then just put that in the passthrough string as per grumpy's guide instead of the card's device ID. Note: I only had to do this for the marvel data card not my M1050. I basically just stumbled across it while pulling my hair out and trying different things. Can't really check as not running same hardware anymore and running unraid 6.0beta3 baremetal now.

 

Sent from my Nexus 4 using Tapatalk

 

 

[vsixtyfour]

Hey Guys,

 

Thanks to Grumpybutfun for the amazing guide, I have had some great success virtualising my Unraid install - I just have one stumbling block.

 

I have passed through the Flash Drive successfully and the 12 port Areca Card is identified during the bootup process but it appears that there are no drivers loaded in the vm for it.

 

The Card is an Areca ARC-1230, I have located the linux drivers on the website here: http://www.areca.com.tw/support/s_linux/linux.htm

 

It appears that I'll need to use the first option and compile from source:

Linux

 

1.30.0X.16-131015

 

2013/10/15 Linux driver source code. Kernel buildin driver supports ARC-188x/12x3 series 6Gb/s SAS RAID controller since kernel version 2.6.36. This source has included ARC-12x4 series support.

 

 

3 files are includes in the zip:

 

-rw-rw-rw- 1 root root  1557 2014-03-03 11:44 Makefile

-rw-rw-rw- 1 root root 158794 2014-03-03 11:44 arcmsr.c

-rw-rw-rw- 1 root root  51493 2014-03-03 11:44 arcmsr.h

 

I have managed to copy this over to the unRAID VM and unpack it however I don't believe there are any compilers installed as when I run make it fails:

 

-bash: make: command not found

 

Is there anyone who might be able to step me through the process or point me to a guide to be able to compile this? I would really appreciate it as it has taken quite some time to get this up and running lol

 

Kind regards

 

v