Virtualizing unRAID with Xen on Arch Linux, XBMC & Windows with VGA Passthrough

[grumpybutfun]

Either PM me or create a thread on Arch Linux forums where 1,000+ people like me will help.

 

This doesn't belong in the unRAID Forum and we are messing up this thread.

[smakovits]

true, spiraled way off topic...standby

 

 

[mattkhan]

So, should I download one of the pre-compiled vhd files?  I mean bzimage and bzroot are obvious since they are the normal unRAID files, but all the reading says I need to compile and such or else download an already compiled one

 

I will send you a link for the bzroot and bzimage. Ironic needs it too.

 

Just sent you a PM.

if I want to run unraid in xen but not in xenserver (and on debian rather than arch) then what's the best way to go?

 

the vhd files in the xenserver thread?

the bzroot/bzimage from wherever?

compile my own based on some .config tweaks?

 

Cheers

Matt

[ironicbadger]

I've written a post on LVM and snapshots based on the information provided by grumpy...

 

Take a look.

 

http://blog.ktz.me/?p=285

[joelones]

Having problems. I installed arch on a single ssd using lvm for the root and efi. Installed xen 4.3.1, and booted the kernel, installed xenbr0. Although, for some reason I was not present with a console while booting the xen kernel. Luckily I had installed ssh prior, so I was able to login.

 

I then proceeded with the following guide to configure passthrough of the m1015: http://blog.ktz.me/?p=166 . Not sure what I did, but now I'm unable to boot either the arch or xen kernel. It complains about "arch error: device uuid not found" and drops me to a root shell.

 

I attempted to rerun the mkinitcpio commands in the arch-chroot via the usb, (yes i added lvm to the hooks) and see same, it cannot find my root on boot. Not sure what I messed up, was working fine though.

 

btw, /boot is on its separate partition formatted as vfat.

 

Seems to be some issues with lvm: https://bbs.archlinux.org/viewtopic.php?id=158012&p=5

 

Thoughts?

[vl1969]

Having problems. I installed arch on a single ssd using lvm for the root and efi. Installed xen 4.3.1, and booted the kernel, installed xenbr0. Although, for some reason I was not present with a console while booting the xen kernel. Luckily I had installed ssh prior, so I was able to login.

 

I then proceeded with the following guide to configure passthrough of the m1015: http://blog.ktz.me/?p=166 . Not sure what I did, but now I'm unable to boot either the arch or xen kernel. It complains about "arch error: device uuid not found" and drops me to a root shell.

 

I attempted to rerun the mkinitcpio commands in the arch-chroot via the usb, (yes i added lvm to the hooks) and see same, it cannot find my root on boot. Not sure what I messed up, was working fine though.

 

btw, /boot is on its separate partition formatted as vfat.

 

Seems to be some issues with lvm: https://bbs.archlinux.org/viewtopic.php?id=158012&p=5

 

Thoughts?

 

is your /boot partition on different drive?

if  "yes" is the drive that holds your /boot connected to different controller.

 

what it sounds like is that you passed through the controller where your /root drive connected to.

[joelones]

Thanks for the reply vl1969, I think I found the issue, I had

use_lvmetad = 0

in the lvm.conf file, to resolve an issue I had during installation. I am able to boot the xen kernel again. Now on to configuring passthrough.

 

Although, I still cannot get a console login prompt with the Xen kernel though. I see as the last line "[ OK ] Listening on LVM2 metadata daemon socket." and keystroke don't bring up the prompt.

 

Another thing, what is the proper disk line for unraid and the vhd? This one I found is an example winxp.vhd. Similar?

 

disk = [ 'tap:aio:vhd:/root/images/winxp.vhd,hda,w' ]

 

or if someone would be kind enough to send me a pm with the compiled in drivers for the bzimage/root that woud be nice, i'd like to see if passing in the m1015 works or not. last time i tried xen, I had passthrough issues with the m1015 card. thx

[grumpybutfun]

Thanks for the reply vl1969, I think I found the issue, I had

use_lvmetad = 0

in the lvm.conf file, to resolve an issue I had during installation. I am able to boot the xen kernel again. Now on to configuring passthrough.

 

I would change it back after you have updated grub.

 

Although, I still cannot get a console login prompt with the Xen kernel though. I see as the last line "[ OK ] Listening on LVM2 metadata daemon socket." and keystroke don't bring up the prompt.

 

Run the following command show:

 

systemctl

 

Do you see system-getty.slice enabled and running?

 

Are there other services that are not loading / giving errors? If so, what are those?

 

Another thing, what is the proper disk line for unraid and the vhd? This one I found is an example winxp.vhd. Similar?

 

Don't use the VHD I posted for XenServer. I will send you a PM with the link for a bzroot and bzimage file since you are running it on Linux.

 

kernel = "/root/VMs/unRAID/bzimage" <---- Change the path for your System
ramdisk = "/root/VMs/unRAID/bzroot" <---- Change the path for your System
extra = "console=hvc0"

 

You do not have a drive so just use the above in your unRAID.cfg file.

 

[joelones]

 

Run the following command show:

 

systemctl

 

Do you see system-getty.slice enabled and running?

 

Are there other services that are not loading / giving errors? If so, what are those?

 

thanks grumpybutfun, still getting accustomed to arch, how would I track down these services? I attached all the services with the command you mentioned.

 

systemctl | grep system-getty.slice
system-getty.slice                                                                       loaded active active    system-getty.sli

 

Another thing, not sure if related, I went ahead with trying to configure a Windows 8 vm (config file: http://pastebin.com/Z1aaZxuU), and noticed the following while starting:

 

Daemon running with PID 1268

xenconsole: Could not read tty from store: No such file or directory

libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: console child exited with error status 2

 

The xenconsole service seems running:

 

ps aux | grep xenconsole

root      298  0.0  0.1  84520  1012 ?        SLsl 14:03  0:00 /usr/bin/xenconsoled --pid-file /run/xenconsoled.pid --log=all --log-dir=/var/log/xen/console

bob      1276  0.0  0.1  10660  1060 pts/0    S+  14:23  0:00 grep xenconsole

 

EDIT: Looks like I still cannot pass the M1015 to unRAID with having unRAID kernel panic. Thought I would give it another try, oh well. Here's the boot log http://pastebin.com/Fb02CgFi

 

Just wondering if anyone has gotten the M1015 passthrough to unRAID working on AMD?

systemctl.txt

[grumpybutfun]

Are there other services that are not loading / giving errors? If so, what are those?

 

Post your /etc/default/grub in pastebin and post here.

 

thanks grumpybutfun, still getting accustomed to arch, how would I track down these services? I attached all the services with the command you mentioned.

 

Assuming you are more familiar with Ubuntu... Once you "learn" Arch you will know most Linux Distros (CentOS, Red Hat, openSUSE, etc.)

 

Another thing, not sure if related, I went ahead with trying to configure a Windows 8 vm (config file: http://pastebin.com/Z1aaZxuU), and noticed the following while starting:

 

Daemon running with PID 1268
xenconsole: Could not read tty from store: No such file or directory
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: console child exited with error status 2

 

I didn't see anything wrong in your Windows cfg but the fact you are not getting a login screen and that tty error above is why it probably bombs. Did you try starting it without the -c at the end?

 

EDIT: Looks like I still cannot pass the M1015 to unRAID with having unRAID kernel panic. Thought I would give it another try, oh well. Here's the boot log http://pastebin.com/Fb02CgFi

 

Just wondering if anyone has gotten the M1015 passthrough to unRAID working on AMD?

 

I don't have a M1015 card but I know people have passed them through.

 

You have a VT-D CPU and Motherboard and your motherboard can do IOMMU, right?

[joelones]

Here's my grub: http://pastebin.com/cLBe2V28

 

I didn't see anything wrong in your Windows cfg but the fact you are not getting a login screen and that tty error above is why it probably bombs. Did you try starting it without the -c at the end?

 

Starting it without the -c seems to not display the error. I guess the -c is not necessary as I would be installing through VNC I take it? I'm assuming the VM is accessible with IP:5900 for the first VM instance? I tried it with ChickenOfTheVNC on mac and vncviewer on Windows to no success.

 

 

I don't have a M1015 card but I know people have passed them through.

 

You have a VT-D CPU and Motherboard and your motherboard can do IOMMU, right?

 

Yes, the AMD does support it through AMD-V and IMMOU is enabled in the bios. I ruled out hardware as I was able to pass the M1015 in ESXi on the same hardware. This KP is kinda been racking my head for some time. Searched high and low, found sporatic posts about something to do with OptionRom on the card. May have to live with the fact that the card simply does not work with my setup.

[grumpybutfun]

Here's my grub: http://pastebin.com/cLBe2V28

 

Remove quiet from GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and update your grub again.

 

Then reboot and you will see more stuff during bootup. See if anything is failing. If do you not see anything.

 

dmesg | more

 

Look for any errors in there.

 

Before all that, try switching Virtual Terminals. CTL+F1 through F7 and see if you get a logon screen then.

 

Last thing...

 

1. What video card / GPU do you have in your computer?

 

2. Post your /etc/mkinitcpio.conf in pastebin and post the link here.

 

I didn't see anything wrong in your Windows cfg but the fact you are not getting a login screen and that tty error above is why it probably bombs. Did you try starting it without the -c at the end?

 

Starting it without the -c seems to not display the error.

 

The -c tells it to start the VM with a console. Since windows doesn't have a console, you do not use it.

 

After you start Windows do the following command:

 

xl list

 

Do you see the Windows VM running?

 

[joelones]

I removed quiet from GRUB_CMDLINE_LINUX_DEFAULT and did not notice any errors in dmesg. Here's a pastebin: http://pastebin.com/9iwadffD

 

Cycling CTL+F1 through F7 did not present me with any console.

 

I have two GPUs (BIOS boots the PCI card first)

[*]

A cheap PCI ATI Rage XL (8MB) for a dumb console

[*]

A Radeon HD6450 PCI Express (for passthrough to a windows vm)

 

Here's my /etc/mkinitcpio.conf http://pastebin.com/BiBFjd8B

 

After I start Windows I do see it running with xl list. I am to connect with TightVNCViewer by setting the encoding to Raw or Hextile.

 

Actually I just noticed, after grub and just prior to booting the Xen kernel I see some text, ever so briefly, that quickly gets overwritten by the Xen kernel output. Something to the effect of, "no console will be available to os". I guess that's why I see no console. Not sure how that happens though.

[ironicbadger]

I removed quiet from GRUB_CMDLINE_LINUX_DEFAULT and did not notice any errors in dmesg. Here's a pastebin: http://pastebin.com/9iwadffD

 

Cycling CTL+F1 through F7 did not present me with any console.

 

I have two GPUs (BIOS boots the PCI card first)

[*]

A cheap PCI ATI Rage XL (8MB) for a dumb console

[*]

A Radeon HD6450 PCI Express (for passthrough to a windows vm)

 

Here's my /etc/mkinitcpio.conf http://pastebin.com/BiBFjd8B

 

After I start Windows I do see it running with xl list. I am to connect with TightVNCViewer by setting the encoding to Raw or Hextile.

 

Actually I just noticed, after grub and just prior to booting the Xen kernel I see some text, ever so briefly, that quickly gets overwritten by the Xen kernel output. Something to the effect of, "no console will be available to os". I guess that's why I see no console. Not sure how that happens though.

 

What does you device manager look like over VNC once you are booted and have you installed the AMD drivers? Screenshot pls.

 

I only had success with windows 8 and 8.1. 7 was a miserable failure!

 

Also I have to change the VNC encode settings from my Mac to connect to the windows VM. I think its to do with qemu modes. Weird, but it works so who cares.

 

Sent from my Nexus 5 using Tapatalk

 

 

[joelones]

What does you device manager look like over VNC once you are booted and have you installed the AMD drivers? Screenshot pls.

 

I wish I could provide you with one. I'm getting stuck at roughly the same point during installation. Not sure what I did wrong here. I create two logical volumes in a separate Volume Group (vg0); one for the Windows VM of 30Gb (not formatted) and a second one of 10GB for ISOs, formatted ext4. I then mounted the ISO lv via /dev/mapper to /mnt/iso.

 

'phy:/dev/vg0/win,hda,w',
'file:/mnt/iso/Win81AIO-20in1-x64-en-US-Nov2013.iso,hdc:cdrom,r'

 

I get, "windows installation encountered an unexpected error verify that the installation sources..."blah, blah, blah

http://imgur.com/D7Sv5T5

 

I do apologize, I realize this is not a Windows thread.

 

[grumpybutfun]

I wish I could provide you with one. I'm getting stuck at roughly the same point during installation. Not sure what I did wrong here. I create two logical volumes in a separate Volume Group (vg0); one for the Windows VM of 30Gb (not formatted) and a second one of 10GB for ISOs, formatted ext4. I then mounted the ISO lv via /dev/mapper to /mnt/iso.

 

'phy:/dev/vg0/win,hda,w',
'file:/mnt/iso/Win81AIO-20in1-x64-en-US-Nov2013.iso,hdc:cdrom,r'

 

I get, "windows installation encountered an unexpected error verify that the installation sources..."blah, blah, blah

http://imgur.com/D7Sv5T5

 

I do apologize, I realize this is not a Windows thread.

 

Instead of:

 

'phy:/dev/vg0/win,hda,w',

 

Try this instead:

 

'phy:/dev/mapper/vg0-win,hda,w',

 

Post the following in pastebin and share here

 

lspci -k

 

Post what you have in the following two lines from /etc/grub.d/09_xen

 

XEN_HYPERVISOR_CMDLINE=
XEN_LINUX_CMDLINE=

 

[guiper]

Just one tip, don't use the port 5900 from the Chicken of the VNC, use simply display 0 if it's 5900, 1 if it's 5901 etc. No port is needed there. Had some troubles exactly with that.

[ironicbadger]

Also, it could be a bad copy of the iso.

 

Try a fresh one.

 

Sent from my Nexus 5 using Tapatalk

 

[smakovits]

testARCH ~ # lspci -k
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
        Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Kernel driver in use: agpgart-intel
        Kernel modules: intel_agp
00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated                                                                                                                                                              Graphics Controller (rev 03)
        Subsystem: Gigabyte Technology Co., Ltd Device d000
        Kernel driver in use: i915
        Kernel modules: i915
00:02.1 Display controller: Intel Corporation 4 Series Chipset Integrated Graphi                                                                                                                                                             cs Controller (rev 03)
        Subsystem: Gigabyte Technology Co., Ltd Device d000
00:1b.0 Audio device: Intel Corporation NM10/ICH7 Family High Definition Audio C                                                                                                                                                             ontroller (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD (Realtek ALC887)
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00:1d.0 USB controller: Intel Corporation NM10/ICH7 Family USB UHCI Controller #                                                                                                                                                             1 (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci_hcd
00:1d.1 USB controller: Intel Corporation NM10/ICH7 Family USB UHCI Controller #                                                                                                                                                             2 (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci_hcd
00:1d.2 USB controller: Intel Corporation NM10/ICH7 Family USB UHCI Controller #                                                                                                                                                             3 (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci_hcd
00:1d.3 USB controller: Intel Corporation NM10/ICH7 Family USB UHCI Controller #                                                                                                                                                             4 (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci_hcd
00:1d.7 USB controller: Intel Corporation NM10/ICH7 Family USB2 EHCI Controller                                                                                                                                                              (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-D525TUD
        Kernel driver in use: ehci-pci
        Kernel modules: ehci_pci
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bri                                                                                                                                                             dge (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd Device 5001
        Kernel driver in use: lpc_ich
        Kernel modules: intel_rng, lpc_ich, leds_ss4200
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (re                                                                                                                                                             v 01)
        Subsystem: Gigabyte Technology Co., Ltd Device b001
        Kernel driver in use: ata_piix
        Kernel modules: ata_piix, pata_acpi, ata_generic
00:1f.2 IDE interface: Intel Corporation NM10/ICH7 Family SATA Controller [iDE m                                                                                                                                                             ode] (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd Device b002
        Kernel driver in use: ata_piix
        Kernel modules: ata_piix, pata_acpi, ata_generic
00:1f.3 SMBus: Intel Corporation NM10/ICH7 Family SMBus Controller (rev 01)
        Subsystem: Gigabyte Technology Co., Ltd GA-8I945PG-RH/GA-D525TUD Mainboa                                                                                                                                                             rd
        Kernel driver in use: i801_smbus
        Kernel modules: i2c_i801
01:05.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8110SC/8169SC G                                                                                                                                                             igabit Ethernet (rev 10)
        Subsystem: Gigabyte Technology Co., Ltd GA-MA69G-S3H Motherboard
        Kernel driver in use: r8169
        Kernel modules: r8169

[joelones]

Post the following in pastebin and share here

 

lspci -k

 

Post what you have in the following two lines from /etc/grub.d/09_xen

 

XEN_HYPERVISOR_CMDLINE=
XEN_LINUX_CMDLINE=

 

Here is the output of

lspci -k

http://pastebin.com/MABxU1J5

 

As for the console issues, maybe there's something in xl dmesg that I can't see, here is the output

xl dmesg

http://pastebin.com/mr2CbSNX

 

XEN_HYPERVISOR_CMDLINE="xsave=1 cpufreq=xen:performance dom0_mem=1024M dom0_max_vcpus=2 dom0_vcpus_pin iommu=1"
XEN_LINUX_CMDLINE="console=tty0 xen-pciback.hide=(02:00.0)(05:05.0)"

 

Also, it could be a copy of the iso.

 

Try a fresh one.

 

I copied it over to /mnt/iso a couple of times. But the iso is fine as I used it to install other machines. Will try grumpybutfun's suggestion to try /dev/mapper and see what happens. Thanks though.

 

EDIT: I just realized why I was losing console support, passing in the wrong video adapter - sorry about that. And /dev/mapper/vg0-win works and doesn't hang the install.

[joelones]

Concerning the Windows 8.1 vm in Xen and my HD6450 passthrough, would like to share my experience.

 

Installing the AMD Catalyst Control Center will cause the vm to hang, you need to simply install the drivers in order for it not to crash. For some reason while connecting the display to the VGA connector on the card, windows wasn't able to determine my optimal resolution (1680x1050), however, connecting it with DVI was better. But with both, I was getting screen tears/black horizontal lines zooming up/down the display, not sure about this. It appeared as thought the AMD driver installed correctly, so cannot say...

 

Oh yeah, I did install the ati passthrough xen patch prior to compilation.

 

I did not passthrough usb keyboard/mouse as I was still using a VNC session to control them while passing through the video to a second display.

 

EDIT: Passed-through the second pci associated with my video card (guess hdmi audiou) and a keyboard/mouse seems like tearing is gone.

 

EDIT: Spoke too soon, BSOD, VIDEO_TDR_FAILURE atikmpag.sys, Xen ssh unreachable

[smakovits]

Going back to my testing and being back on topic with unraid (my win7 VNC session still doesnt work because it never listens), do I need to have the unRAID USB in order for unraid to start/work/allow me to connect via http?

 

I ask because from the looks of it, my unraid vm works from the perspective that it starts and boots to login.  From here, I get the IP and attempt to connect like normal with my browser but it fails.  I am simply trying to determine if this is expected behavior or if there is an issue with the way my traffic is being passed and hence my vnc issue...however, since I can ping the system, I suspect it is simply an issue with not having the USB stick and therefore expected.

 

 

[ironicbadger]

Going back to my testing and being back on topic with unraid (my win7 VNC session still doesnt work because it never listens), do I need to have the unRAID USB in order for unraid to start/work/allow me to connect via http?

 

I ask because from the looks of it, my unraid vm works from the perspective that it starts and boots to login.  From here, I get the IP and attempt to connect like normal with my browser but it fails.  I am simply trying to determine if this is expected behavior or if there is an issue with the way my traffic is being passed and hence my vnc issue...however, since I can ping the system, I suspect it is simply an issue with not having the USB stick and therefore expected.

 

yes, the webGUI won't work without the USB being detected correctly.

[smakovits]

thats good to know.  Any thoughts on why vnc with xen doesnt seem to work?  I think it is my last real hurdle in liking Xen and Arch...

 

For whatever reason vnc is not running or my windows vm is broken and never gets to the point of starting it, even thought it appears under xl list, so I assume it is running

[grumpybutfun]

EDIT: Passed-through the second pci associated with my video card (guess hdmi audiou) and a keyboard/mouse seems like tearing is gone.

 

EDIT: Spoke too soon, BSOD, VIDEO_TDR_FAILURE atikmpag.sys, Xen ssh unreachable

 

I noticed you are trying to passthrough the AMD Rage (I don't even know if that card works for passthrough). I thought this was the "console" / Arch video card and the AMD 6450 was the card you meant to passthrough (I know 100% that works. I have 2 of them).