[GUIDE] Virtualizing unRAID in Xen / KVM on Arch Linux

[smakovits]

Smakovits:

What is the OS of the VM that's giving you the message? Is it able to boot, or does it hang? Was it built using Virt-Manager or from a config file (by hand, so to speak)?

 

OS is Arch.  Built using a script from grumpy.  I am not sure if it is booted or not as those are the last 2 messages I see at the console.  Some stuff online says it might be booted still and they give a crazy method to check, but I much rather just get rid of the message and have the console work.

[doorunrun]

I haven't seen the script. Sorry to be pestering you about it..... You can use "xl list" to list the VM's running. Are you launching this from command line (terminal)? If so then adding a "-c" should have it output the VM to the console. If things go sour just jump over to another terminal session Ctl-Alt-F2 and kill the VM from there, as in "xl destroy domU1."

 

I built one Arch VM by hand and to get it to run I type: "xl create -c /etc/xen/domU1.cfg" In my case the VM's start up (init script) was a little too elaborate and caused problems. This page helped a lot: https://wiki.archlinux.org/index.php/Talk:Xen and points out some of the flaws of the documentation 

 

Particularly this:

Grub2 entry is broken and PyGrub won't boot

A more simpler Grub2 entry can be created in DomU's /boot/grub/grub.cfg:

menuentry 'Arch Linux' --class gnu-linux --class gnu --class os {

    insmod gzio

    insmod ext2

    insmod part_msdos

    set root='(hd0)'

    linux  /boot/vmlinuz-linux root=UUID=04968465-e3b9-4ad3-9526-b8bab72779d1 ro console=hvc0 splash quiet

    initrd  /boot/initramfs-linux.img

}

 

Does that help?

 

Edit: one more thing, the initial VM was built with 256 mb of RAM...I thing I bumped it up to 512 in the config file after several stalled boot ups.

[grumpybutfun]

Daemon running with PID 972

[    0.254021] i8042: No controller found

[    0.254021] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)

 

 

Any thoughts?

 

Those are warnings and you can ignore them.

 

1. You don't have a keyboard hooked up in the VM.

 

You don't need it and if you want the warning to go away you have to pass through a USB Controller with a USB keyboard hooked up. Why you would want to do that, I dunno since the warning doesn't hurt anything.

 

2. In Xen the VMs do not have access to the Hardware Clock. That is designed by purpose so VMs dont access / can't change the clock on the server.

 

Ignore both warnings is my advice or compile a kernel where it doesn't look for a keyboard or the hardware clock.

 

[doorunrun]

Good morning/day GrumpyButFun,

 

After more tinkering around with Arch/Xen/LibVirt a little thought is beginning to sink in.....is Xen/LibVirt a good fit? It seems most use (and support) of LibVirt is connected with KVM based builds.

 

I've grown very fond of Arch's basic structure, but I was impressed with KVM on openSUSE (as covered in your guide) and the relative ease of getting a VM going with LibVirt.

 

So, what's your view of KVM on Arch? As a newcomer to all of this I haven't given up on the idea of a rebuild from the ground up and see if stuff hangs together better. My current build of Arch is OK to a point, but occasionally when booting Arch's Xen kernel, my screen is blank and can't be remotely connected to. To me this points to some basic low level issues that may only get worse as you go up in the "application stack" even when it does boot correctly.

 

What I hope to explore today is the idea of building a VM through config file using your KVM-unRAID image file that was part of the KVM/openSUSE guide. I think I read Xen supports VHD's, right? Thanks!!

[grumpybutfun]

After more tinkering around with Arch/Xen/LibVirt a little thought is beginning to sink in.....is Xen/LibVirt a good fit? It seems most use (and support) of LibVirt is connected with KVM based builds.

 

Arch doesn't have Xen support in libvirt they have it solely focused on KVM. As you know, you have to custom compile with xen support enabled. Therefore, You have to do a lot of configuration changes to make libvirt and Xen play well together on Arch.

 

There are other distros that do not require you to jump through the hoops to make libvirt and xen play well together.

 

I've grown very fond of Arch's basic structure, but I was impressed with KVM on openSUSE (as covered in your guide) and the relative ease of getting a VM going with LibVirt.

 

I am fond of Arch... It's my distro of choice. The freedom / Keep it Simple (After you get it running) approach / Arch User Repository / package manager / Systemd is why.

 

So, what's your view of KVM on Arch?

 

I use KVM in Arch and it's "installed" by default. All you need to do is install QEMU and you are ready to go.

 

You can either use libvirt or qemu to configure / start / stop your VMs.

 

Start with the Arch KVM Wiki and then QEMU one.

 

I would start with using libvirt and virt-manager and after you get comfortable with those take a look at the GUIs for QEMU (listed in the Arch Wiki)

 

As a newcomer to all of this I haven't given up on the idea of a rebuild from the ground up and see if stuff hangs together better. My current build of Arch is OK to a point, but occasionally when booting Arch's Xen kernel, my screen is blank and can't be remotely connected to. To me this points to some basic low level issues that may only get worse as you go up in the "application stack" even when it does boot correctly.

 

I suspect that is your Network Card / Driver that is causing the problems. Is it a realtek?

 

Add your NIC driver to the ramfs and make it a static ip.

 

To add it to the ramfs...

 

Get the name of the Linux Driver for your NIC.

 

lspci -k

 

Add it to the modules line in mkinitcpio.conf

 

nano /etc/mkinitcpio.conf

 

example: MODULES="xen-pciback radeon ext4 vfat r8169"

 

Update ramfs

 

mkinitcpio -p linux

 

Reboot your server.

 

To make your ip static go take a look at /etc/netctl/examples/ethernet-static and you will see what you need to do.

 

Edit your NIC profile (xenbr0 in your case) and make the changes.

 

Then reenable the netctl profile

 

netctl reenable <profile name> (in your case xenbr0)

 

What I hope to explore today is the idea of building a VM through config file using your KVM-unRAID image file that was part of the KVM/openSUSE guide. I think I read Xen supports VHD's, right? Thanks!!

 

Either uninstall Xen "pacman -R xen" or boot into the normal Linux Kernel through your grub menu and follow the KVM and QEMU guides on Arch.

 

You can switch between KVM and Xen simply through grub. KVM is my default now but I am testing Xen 4.4 too and boot between the two.

[doorunrun]

I suspect that is your Network Card / Driver that is causing the problems. Is it a realtek?

I'm using the NIC on the mobo.....

Ethernet controller: Intel Corporation 82567LM-3 Gigabit Network Connection (rev 02)

 

Add your NIC driver to the ramfs and make it a static ip.

To add it to the ramfs...

Get the name of the Linux Driver for your NIC.

lspci -k

Add it to the modules line in mkinitcpio.conf.....................

 

Thanks for the expert and detailed instructions!! I'll migrate in that direction. I appreciate the your help and guidance in this project!

[smakovits]

Particularly this:

Grub2 entry is broken and PyGrub won't boot

A more simpler Grub2 entry can be created in DomU's /boot/grub/grub.cfg:

menuentry 'Arch Linux' --class gnu-linux --class gnu --class os {

    insmod gzio

    insmod ext2

    insmod part_msdos

    set root='(hd0)'

    linux  /boot/vmlinuz-linux root=UUID=04968465-e3b9-4ad3-9526-b8bab72779d1 ro console=hvc0 splash quiet

    initrd  /boot/initramfs-linux.img

}

 

/quote]

 

This here is the most likely issue.  I, for fun, after some messing and still seeing things hang tried the fallback option in grub and that does boot properly.  I tried replacing my grub with the one suggested, replaced my UUID, but it still hangs the same way.  However, if right at the initial boot, once grub displays, I hit Ctrl+] then the machine loads.

 

However, that too is not without issue.  When loading I see a box pop up that says the root device is not configured to be mounted read-write, but in my fstab, it does show as rw not ro like some forum posts.

 

https://bbs.archlinux.org/viewtopic.php?id=167153

 

 

[grumpybutfun]

This here is the most likely issue.  I, for fun, after some messing and still seeing things hang tried the fallback option in grub and that does boot properly.  I tried replacing my grub with the one suggested, replaced my UUID, but it still hangs the same way.  However, if right at the initial boot, once grub displays, I hit Ctrl+] then the machine loads.

 

I dunno know what to tell you.

 

Install Arch on a LV using EXT4. Following the Arch Beginners Guide and add the Xen Drivers per the Arch Xen wiki. Install Grub per the Arch Beginners Guide and it should work. If not, it's something you are doing wrong.

 

Also, this isn't the place for this so take these type of questions / support issues to the Xen or Arch Forums.

[smakovits]

This here is the most likely issue.  I, for fun, after some messing and still seeing things hang tried the fallback option in grub and that does boot properly.  I tried replacing my grub with the one suggested, replaced my UUID, but it still hangs the same way.  However, if right at the initial boot, once grub displays, I hit Ctrl+] then the machine loads.

 

I dunno know what to tell you.

 

Install Arch on a LV using EXT4. Following the Arch Beginners Guide and add the Xen Drivers per the Arch Xen wiki. Install Grub per the Arch Beginners Guide and it should work. If not, it's something you are doing wrong.

 

Also, this isn't the place for this so take these type of questions / support issues to the Xen or Arch Forums.

 

Word.

 

I just don't like the Arch forum as much.  At least here someone talks to me.  Over there every useful comment is followed by 5 people saying to RTFM (wiki in this case).  Either they are super stuck up or they don't like being helpful.  I mean really, sometimes people just need a push in the right direction aside from being told to read an entire wiki.  Be it a command or a log or just simple dialog.  Much like you do here.  Often you know and post awesome instructions, other times you dont and you say it, like now...I just don't enjoy going there...

 

Regardless.  All this KVM talk has me intrigued.  If it really is better and easier and still on Arch, I would be for trying it instead.  Especially if the end result is the same.  Till this very moment I thought KVM was opensuse only, but maybe not...

 

So now the tough question, based on your use and experience, do you actually recommend KVM over Xen?  And based on knowing it is your current default, would you say it is better?

[grumpybutfun]

Regardless.  All this KVM talk has me intrigued.  If it really is better and easier and still on Arch, I would be for trying it instead.  Especially if the end result is the same.  Till this very moment I thought KVM was opensuse only, but maybe not...

 

KVM is built into the Linux Kernel so pretty much every Linux Distro has it enabled.

 

As far as KVM in Arch is concerned... RTFM. Read the Arch KVM Wiki and then the QEMU and use libvirt / virt-manager to manage it.

 

So now the tough question, based on your use and experience, do you actually recommend KVM over Xen?  And based on knowing it is your current default, would you say it is better?

 

That is a loaded question. Simple answer....

 

KVM / Xen both use QEMU as a machine emulator and virtualizer and each have their own PV drivers. Speed wise you will not notice a difference between the two. KVM is the Virtual Server of choice in most Linux Distros because it's native in the Linux Kernel and doesn't require separate packages like Xen does (there are benefits to a separate Xen Kernel in an enterprise environment)

[doorunrun]

Particularly this:

Grub2 entry is broken and PyGrub won't boot

A more simpler Grub2 entry can be created in DomU's /boot/grub/grub.cfg:

menuentry 'Arch Linux' --class gnu-linux --class gnu --class os {

    insmod gzio

    insmod ext2

    insmod part_msdos

    set root='(hd0)'

  linux  /boot/vmlinuz-linux root=UUID=04968465-e3b9-4ad3-9526-b8bab72779d1 [glow=red,2,300]ro[/glow] console=hvc0 splash quiet

    initrd  /boot/initramfs-linux.img

}

 

Oh, sorry, I had the exact issue, got it fixed and forgot about it! You see how documentation can go astray...the error is in the Grub line right after the UUID. "RO stands for Read Only. I think it has to be W.  (Edit-- Probably more likely RW) 

 

[grumpybutfun]

Oh, sorry, I had the exact issue, got it fixed and forgot about it! You see how documentation can go astray...the error is in the Grub line right after the UUID. "RO stands for Read Only. I think it has to be W. 

 

Thanks for coming back and posting. I hate that I am a lone ranger and the people who have this successfully working don't "pay it forward" and help me / other people out.

 

I don't have Tom's approval to do so.... But if did, this is the reason why I wouldn't post a guide on how to add unRAID into Arch / CentOS / openSUSE / etc.

 

1. People would hound me to death.

 

2. It's like a job that I don't get paid for.

 

3. People do not want to make 1 & 2 worth it by "paying it forward" and helping me / others out.

 

4. It seems that a lot of the unRAID community is "Gimmie, Gimmie... I'm gone". I can see why plugins are in the shape they are in. No incentive for those guys to take a beating, solve problems, etc. when people don't stick around and help out their fellow man.

[smakovits]

As a noob,  I must ask,  what does the ram disk do and what affect does it have for booting because I think that's where the issue falls

[grumpybutfun]

As a noob,  I must ask,  what does the ram disk do and what affect does it have for booting because I think that's where the issue falls

 

You have to put all the drivers it needs to be able to access the disk so it can load all of the drivers.

 

The "xen-blkfront xen-fbfront xen-netfront xen-kbdfront" modules must be added to Mkinitcpio and if you are running on some odd SATA / RAID controller you would add that too.

 

If you installed your Arch VM using LVM... You will need to add LVM2 to the hooks file in Mkinitcpio also.

[smakovits]

does the -fallback.img ramdisk do something different or better?  I ask because it boots fine...

[grumpybutfun]

does the -fallback.img ramdisk do something different or better?  I ask because it boots fine...

 

Apparently grub is messed up on your system. If the fallback works, the normal way should too.

[johnodon]

I'll try and help where I can but I have installed KVM and Xen (successfully and unsuccessfully) on so many different distros it is all a blur to me.    I don't want to give out bad advice.

 

"Don't call me Stephen!"

              - Stiffler (American Pie)

 

(inside joke) 

[grumpybutfun]

I'll try and help where I can but I have installed KVM and Xen (successfully and unsuccessfully) on so many different distros it is all a blur to me.    I don't want to give out bad advice.

 

LOL!

 

I give out bad advice all the time...  You saw the other threads where I was called all kinds of names, ridiculed and mocked. Join in the fun.

 

Speaking of which...

 

You uncovered something HUGE the other day, installed it and have it working on one of your development machines. Are you too chicken to share or are you are hogging it all for yourself?

 

Either tell your wife that she married a girly man or stop being selfish.

[johnodon]

I'll try and help where I can but I have installed KVM and Xen (successfully and unsuccessfully) on so many different distros it is all a blur to me.    I don't want to give out bad advice.

Speaking of which...

 

You uncovered something HUGE the other day, installed it and have it working on one of your development machines. Are you too chicken to share or are you are hogging it all for yourself?

 

She knows what she married. 

 

I haven't had time to really play with that lately.  Actually, I need to burn it to the ground and start over.  I butchered the partitions and couldn't add an SR.

 

Maybe tonight....

 

J

[grumpybutfun]

I haven't had time to really play with that lately.  Actually, I need to burn it to the ground and start over.  I butchered the partitions and couldn't add an SR.

 

Maybe tonight....

 

I did some checking around. I might be able to do it in CentOS too. I have to finish some other things first, post my screenshots and then I was going to attempt it.

[johnodon]

I haven't had time to really play with that lately.  Actually, I need to burn it to the ground and start over.  I butchered the partitions and couldn't add an SR.

 

Maybe tonight....

 

I did some checking around. I might be able to do it in CentOS too. I have to finish some other things first, post my screenshots and then I was going to attempt it.

 

The little bit that I did play with it looked positive.  I was able to map an NFS ISO SR like you can normally.

[grumpybutfun]

The little bit that I did play with it looked positive.  I was able to map an NFS ISO SR like you can normally.

 

It should work and it works in Debian too. CentOS should be any day now.

 

Now that they have it working in 4.1.5 they will start will roll it out to later versions of Xen.

[smakovits]

Here is something of interest.  I started to think about what was happening.  Vm boots,  hangs,  ctrl+]  made it boot and seem Ok.  So it was like I was in a double Console,  so I tried starting the VM without - c.

 

I waited a few seconds and executed the console command and jumped right into my booted vm.  Now,  obviously this is not the way it should work,  but it proves at least my RAM disk is Ok,  maybe?  I assume from here maybe it is something to do with a command in grub and maybe it too is calling the console?

 

Not sure but I call it progress and 90% successful at this point.

[grumpybutfun]

Here is something of interest.  I started to think about what was happening.  Vm boots,  hangs,  ctrl+]  made it boot and seem Ok.  So it was like I was in a double Console,  so I tried starting the VM without - c.

 

I waited a few seconds and executed the console command and jumped right into my booted vm.  Now,  obviously this is not the way it should work,  but it proves at least my RAM disk is Ok,  maybe?  I assume from here maybe it is something to do with a command in grub and maybe it too is calling the console?

 

Not sure but I call it progress and 90% successful at this point.

 

What is happening is the grub menu is popping up, you can't see it and the timer is being interrupted.

 

To make it so you can see the grub menu...

 

1. Edit and ADD (not replace) the following to the GRUB_CMDLINE_DEFAULT="" line to start the console on /dev/ttyS0

 

GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,38400n8"

 

2. Add a serial console section

 

# Serial console
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=38400 --unit=0 --word=8 --parity=no --stop=1"

 

3. Rebuild the grub.cfg file

 

grub-mkconfig -o /boot/grub/grub.cfg

 

Now when you boot into the VM you should see the grub menu.

 

Of course you could make it default without a timer straight into the VM too. Google is your friend.

[smakovits]

Hell yeah,  might do that,  screw grub,  I don’t need it.  I will let you know if I get stuck...

 

Google is my friend,  but it is love hate,  especially when I find nothing relevant to what I want.

 

Sent from my Nexus 7 using Tapatalk