[GUIDE] Virtualizing unRAID in KVM on openSUSE 13.1 (64Bit) <--- Completed 12/19

[bnevets27]

I'm stuck here,

trying to passthrough my SAT2-MV8 cards.

 

Specs:

Motherboard: Supermicro X7DWN+

CPU: Intel Xeon L5420 - 2.50GHz Quad-Core

SATA Expansion Card(s): 2 x SAT2-MV8 Raid Controller Cards

 

EDIT:

Working after running:

echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

Reference: https://access.redhat.com/site/articles/66747

 

Looks like I successfully have unRAID VM running! 

 

Also a FYI there is a bug during the install openSUSE install:

This topic can be closed, folks at the bugzilla have found the problem: according to bug #773058 - non-existent floppy drive enabled in BIOS causes hang while searching for partitions.

I don't really know, why openSUSE hangs there, while whole bunch of other linux distributions just don't care. 

http://forums.opensuse.org/english/get-technical-help-here/install-boot-login/485066-installation-gets-stuck-while-scanning-hdd-partitions.html#post2543969

[johnodon]

EDIT:

Working after running:

echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

Reference: https://access.redhat.com/site/articles/66747

 

I had the same exact issue and this worked for me also.  Thank you for the information!

 

Only problem now is that I cannot reach the unraid webgui.  It is getting an IP, but when I ifconfig, I see lots of RX dropped packets.  I'll try another LAN port (or put a card in) and another cable but the opensuse box itself is not having any issues.  Maybe a problem with br0?

 

John

[doorunrun]

Working after running:

echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

Reference: https://access.redhat.com/site/articles/66747

I too was having similar troubles and the patch worked for me. The URL was helpful in testing my older hardware and it provided the method for persistence across reboots.

 

In my case starting my unRAID VM causes the host to seemingly grind to a halt, the host GUI that is. After a minute or so the host system returns to normal. This bears some examination, but for now I'm very pleased to report PCI and USB passthrough on this hardware.

 

[hernandito]

Boy, I have read all the posts on the unRAID 64 thread... and my head is spinning a bit. I am a typical unRAID user who likes to push the envelope... as I posted here:

 

http://lime-technology.com/forum/index.php?topic=30815.msg281336#msg281336

 

I read about the hardware requirements and the motherboards listed on the links on the first post are basically old-out of stock. Can someone please recommend a cpu and mobo that can run Vt-d....  something that is available at Newegg and will allow Plex Media Server to run....

 

Grumpy.... thanks for all this.

  :)

 

[joelones]

Was wondering if anybody has some thoughts, I have a ridiculously old PCI card, an ati rage xl that I'd like to use as a primary video card, unfortunately once I install suse I'm getting 640x480 res. I know that I should use the mach 64 driver, but can't force xorg to use it.

[rvandaalen]

Had some time and gave it another go after I saw the intel_iommu=on  posted and the echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts

 

Clean new install, followed the steps to the letter. Still I get an error.

 

When I boot my hdd from a pci card with intel_iommu=on added my system won't boot:

https://bugzilla.redhat.com/show_bug.cgi?id=605888  intel_iommu=on => No root device found

 

So I switch my hdd to internal sata (which I don't want) but then the system boots. So atleast I can tell the intel_iommu=on option does something.

 

I have a z77 extreme4 board and intel 3470T proc  Vt-d passthrough works fine on xenserver! but can't get it to work on suse which is a shame.

Any ideas to try?

 

 

Error starting domain: internal error: process exited while connecting to monitor: char device redirected to /dev/pts/4 (label charserial0)

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: PCI region 0 at address 0xf7f17000 has size 0x400, which is not a multiple of 4K.  You might experience some performance hit due to that.

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: No IOMMU found.  Unable to assign device "hostdev0"

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization failed.

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: Device 'kvm-pci-assign' could not be initialized

 

 

Traceback (most recent call last):

  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper

    callback(asyncjob, *args, **kwargs)

  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 117, in tmpcb

    callback(*args, **kwargs)

  File "/usr/share/virt-manager/virtManager/domain.py", line 1168, in startup

    self._backend.create()

  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 697, in create

    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)

libvirtError: internal error: process exited while connecting to monitor: char device redirected to /dev/pts/4 (label charserial0)

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: PCI region 0 at address 0xf7f17000 has size 0x400, which is not a multiple of 4K.  You might experience some performance hit due to that.

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: No IOMMU found.  Unable to assign device "hostdev0"

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization failed.

qemu-system-x86_64: -device pci-assign,configfd=26,host=00:1d.0,id=hostdev0,bus=pci.0,addr=0x7: Device 'kvm-pci-assign' could not be initialized

 

 

[joelones]

I seem to be experiencing a call trace, although the unraid vhd seens to power up find and all. Looks like a warning.

------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/perf_event_amd.c:772 amd_pmu_init+0x185/0x232()
Hardware name: Bochs
Odd, counter constraints enabled but no core perfctrs detected!
Modules linked in:
Pid: 1, comm: swapper/0 Not tainted 3.9.11p-unRAID #2
Call Trace:
[<c104d3d9>] warn_slowpath_common+0x77/0x8e
[<c15b5a50>] ? amd_pmu_init+0x185/0x232
[<c15b5a50>] ? amd_pmu_init+0x185/0x232
[<c104d46c>] warn_slowpath_fmt+0x2e/0x30
[<c15b5a50>] amd_pmu_init+0x185/0x232
[<c15b54b7>] init_hw_perf_events+0x34/0x448
[<c1047d00>] ? set_memory_x+0x53/0x55
[<c15b0f0b>] ? set_real_mode_permissions+0x56/0x5d
[<c100116f>] do_one_initcall+0x87/0x13d
[<c15b9e19>] ? native_smp_prepare_cpus+0x3a5/0x3df
[<c15b5483>] ? check_bugs+0xc8/0xc8
[<c15af44e>] kernel_init_freeable+0x57/0x188
[<c14285c5>] kernel_init+0xb/0xbe
[<c1437277>] ret_from_kernel_thread+0x1b/0x28
[<c14285ba>] ? rest_init+0x5a/0x5a
---[ end trace e9344054878c8eb0 ]---
Broken PMU hardware detected, using software events only.
Failed to access perfctr msr (MSR c0010001 is fffffffffffffff

 

On a positive note, with KVM I'm able to passthrough the M1015 to unRAID whereas Xen would not.

 

Btw, using AMD FX8320 cpu.

 

Also, what's the best way to add logical volumes to a VMs in KVM. I see you can create LVs within "Virtual Machine Manager" or should you do it on the command line to be sure to use thin provisioning etc...?

[grumpybutfun]

Also, what's the best way to add logical volumes to a VMs in KVM. I see you can create LVs within "Virtual Machine Manager" or should you do it on the command line to be sure to use thin provisioning etc...?

 

You can do it either way.

 

I am usually a command line kind of guy but if I am using the GUI, I just configure it to use LVM and create the LVs in the GUI.

 

BTW - You can ignore that earlier warning you posted. It's fixed in later versions of the Linux Kernel.

 

[joelones]

Thanks, great guide btw.

 

Only thing, I wish I was able to get my el cheapo PCI card (ATI Rage XL) to work and free up a PCI Express slot. Your guide installs a GUI and I had problems trying to navigate with a 640x480 resolution when I tried with the Rage card. I probably missed something, like a driver or something. Not sure an 8MB Video Card can do more that 1024 though.

 

In any case I opted to install a PCI Express card and reinstalled with that. Maybe I'll backup and try it again.

 

I used the following partition scheme on a 120Gb SSD.

 

512MB Ext2 /boot

1G Swap

10GB Btrfs /

100GB LVM vg1

 

Wondering whether Clonezilla can read/backup Btrfs/LVM partitions?

[grumpybutfun]

Only thing, I wish I was able to get my el cheapo PCI card (ATI Rage XL) to work and free up a PCI Express slot. Your guide installs a GUI and I had problems trying to navigate with a 640x480 resolution when I tried with the Rage card. I probably missed something, like a driver or something. Not sure an 8MB Video Card can do more that 1024 though.

 

Did you look YaST to and adjust your monitor settings and change the screen resolution?

 

 

I used the following partition scheme on a 120Gb SSD.

 

512MB Ext2 /boot

1G Swap

10GB Btrfs /

100GB LVM vg1

 

That is a good partition scheme.

 

I typically use EXT4 for boot is the only difference.

 

Swap I doubt you will ever use unless you put your Server to sleep and tell it to it to disk instead of memory. I haven't used a swap partition in YEARS in an Enterprise Environment or Home.

 

Wondering whether Clonezilla can read/backup Btrfs/LVM partitions?

 

Should work like a champ. I have a PXE Server and I back up every File System / Partition scheme under the sun in Clonezilla no problems. It's only complicated if you go the UEFI route.

 

[joelones]

Did you look YaST to and adjust your monitor settings and change the screen resolution?

 

I was not able to find anything pertaining to screen resolution with YaST Control Center, maybe I didn't look hard enough. I believe I looked in "Configure Desktop -> Display and Monitor" in which case it did not present me with a Display Configuration - it was pretty much empty.

 

Does your guide "Run Linux GUI Applications On Your Windows / Mac via SSH + X11 Forwarding" still apply for the opensuse installation?

 

http://lime-technology.com/forum/index.php?topic=30980.msg279085#msg279085

 

Also if I need to resize the Btrfs partition, must I do something special seeing how I have a LVM partition right after it consuming the rest of the drive? Guess I need think about it.

[grumpybutfun]

I was not able to find anything pertaining to screen resolution with YaST, maybe I didn't look hard enough. I believe I looked in "Configure Desktop -> Display and Monitor" in which case it did not present me with a Display Configuration - it was pretty much empty.

 

Sounds like what I would have done.

 

You could try generating an xorg.conf file which might help.

 

As root, you will need to kill X and run Xorg -configure and copy /root/xorg.conf.new to /etc/X11/xorg.conf.

 

Does your guide "Run Linux GUI Applications On Your Windows / Mac via SSH + X11 Forwarding" still apply for the opensuse installation?

 

Yes but a few changes have to be made.

 

uncomment and edit the following in /etc/ssh/sshd_config

 

AddressFamily inet # <--- It defaults to 
#ChallengeResponseAuthentication yes #<-- Comment it out if it isn't already.

 

I am not in front of a computer but I believe that is the only changes needed.

 

ssh -vx root@IPADDRESS

 

Post what it shoots out and I will be able to assist.

[jbartlett]

It seems that the MSI Big Bang II motherboard (the one with the pretty gatlin gun heat sink that likes to short out the mosfet) doesn't support the PCI passthrough. I managed to get a return on the motherboard because the mosfet shorted out against the gatlin gun heat sink.

 

Currently trying to see if ASUS X79 DELUXE LGA 2011 supports IOMMU.

[joelones]

ssh -vx root@IPADDRESS

 

Post what it shoots out and I will be able to assist.

 

Thanks, it seems to work, I'm able to forward X to windows 7. I had some issues with Clonezilla and the Btrfs partition, I ended up trying the Testing Release and it did the trick.

 

Was also wondering the following?

[*]How do you automate VM startup / shutdown upon boot and power off? I guess we have to script something in a startup file?

[*]Also if I need to resize the Btrfs partition, must I do something special seeing how I have a LVM partition right after it consuming the rest of the drive? (Guess I need to think about that one).

Sorry for the bombarding of questions.

 

[grumpybutfun]

Was also wondering the following?

How do you automate VM startup / shutdown upon boot and power off? I guess we have to script something in a startup file?

 

If we are talking KVM and using Virt-Manager... Look at the VM details, somewhere in there it gives you the option to tell it to autostart.

 

Also if I need to resize the Btrfs partition, must I do something special seeing how I have a LVM partition right after it consuming the rest of the drive? (Guess I need to think about that one).

 

BTRFS is amazing once you play around with it. One of the many features... You can shrink BTRFS on the fly on a running system.

 

Since you want to shrink the partition too and use the space elsewhere...

 

Download the gparted ISO and install it onto to a USB Flash Drive. Use it to shrink your root btrfs, free up the space on the partition, shrink the partition and use the free space elsewhere.

 

It's all via a GUI... Simply point, click and use slidebars.

 

Beats typing all the commands via command line.

 

[vl1969]

this question is for Grumpy :-0

 

a side from using different unriad immage

is there any thing special in virtualizing unRaid on opensuse 13.1 using Xen?

 

I tried all I could with KVM and it will not boot the VM once I do PCI passthough.

 

with just the usb passthough  I get a nice runing  unraid VM but no drives. to use.

I followd all the sugestions here, added all the kernel settings etc. nothing works.

want to try Xen setup one more time. with the parameters.

 

thanks.

[grumpybutfun]

is there any thing special in virtualizing unRaid on opensuse 13.1 using Xen?

 

Nope. Go ahead and try. They have plenty of documentation on their site on how to do it.

 

with just the usb passthough  I get a nice runing  unraid VM but no drives. to use.

I followd all the sugestions here, added all the kernel settings etc. nothing works.

want to try Xen setup one more time. with the parameters.

 

The issue I believe is the SAT2-MV8 PCI-X Raid cards.

 

Have you tried the motherboard SATA ports? I believe some others have had success with them.

[vl1969]

I will try mb sata. But it will not solve anything. I only have 6 ports.

 

Sent from my SGH-T889 using Tapatalk

 

 

[bnevets27]

with just the usb passthough  I get a nice runing  unraid VM but no drives. to use.

I followd all the sugestions here, added all the kernel settings etc. nothing works.

want to try Xen setup one more time. with the parameters.

 

The issue I believe is the SAT2-MV8 PCI-X Raid cards.

 

Have you tried the motherboard SATA ports? I believe some others have had success with them.

 

I have successfully passed through the SAT2-MV8 card. There is also evidence of another forum member being successful in one of my threads.

 

Sent from my SM-N900W8 using Tapatalk

 

[Othi]

Is it possible to pass through the SATA disks directly? I got them to show up as /dev/vdX with these libvirt settings:

   <disk type='block' device='lun'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/sda'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>

 

Sadly, emhttp doesn't seem to like it. Any way to make them show up as /dev/sdX? This thread from 2011 talks about patching emhttp (urgh) to see /dev/vdX drives: http://lime-technology.com/forum/index.php?topic=13446.0 - I don't know if this would be enough for them to work as expected.

 

Since the vhd you provided includes the virtio drivers, is it possible for you to make this change?

[BLKMGK]

Wow, only just now spotting this guide and wanted to say Thanks! Well done and I might just give this a shot to play with it. I had always thought Xen > KVM but it seems they are closer to interchangeable. Wit this guide and some additional hardware i just might give it a shot to play with - well written and clear enough I feel like I might have a chance at it ;-)

[syntaxx]

cool i want to try this to be able to passthrough my vc to a windows vm and use it in one of my led to act as a personal computer with gaming as well. currently, i am using esxi 5.1 my board has vt-d capabilities. do you think i wont get problems?

[syntaxx]

Hi, I have tried this however there comes a time that I wont be able to connect to my unraid box via ip address. On virt-manager console there are no network anymore and will end up in reboot. Can someone help?

[siddharta42]

First of all thanks to Grumpybutfun for the excellent guide. I'm getting my feet wet with virtualization and this is a great starting point. I have some limited experience with Fedora and some with Ubuntu but that's about it and I'm eager to dig in 

 

I'm sort of stuck at a silly point though, creating the UnRAID VM *edit: solved, see below*

 

"Select Linux for the OS, select Version, select Other OS and Select Generic 2.6.25 or later kernel with virtio"

 

If I select Linux for OS in the first drop down box, then "Show all OS options" from the drop down box in the Virtual Machine Manager it becomes unresponsive to the point where the system fully hangs (have tried this numerous times).

 

messages doesn't seem to have anything related to this and I don't know which other log may be useful here. Could be just a bug in the Virtual Machine Manager I suppose. I'm using XFCE rather than KDE but have otherwise followed the instructions to the letter.

 

Two minor questions apart from that, more for my understanding than anything else:

 

1) why the need the give sudo rights to your user without a password? Convenience or would it break something if you don't do that?

 

2) why have authentication to "none" instead of "sasl" in libvirtd.conf when using TCP?

 

I guess my next step would be to look into how to set up the UnRAID VM using command line, as I'm loathe to reinstall again for what may be a bug in a GUI.

 

Anyhow great project.

 

*edit* when selecting "Windows" as OS type there is also a "Show all OS options" in the Version drop down box - no problems with that one though. It's only when selecting "Linux" as OS type that the New VM wizard hangs (often taking the system with it but not always).

 

*edit 2* it seems to be an issue with the AMD 7950 or with AMD driver. At the last attempt I saw that the display went black a few times and the whole list appeared, only to freeze again at mouseover. So I removed the AMD 7950 and am now using the iGPU of the SB i5.

 

I'm now /unstuck and will venture onwards. Thanks!

[dirrtyjoe]

My CPU is not VT-d compatible but it is VT-x. Will I be able to setup a pfSense VM and properly use the NICs to setup my network? Will that require the passthrough?