January 20, 201610 yr Goodday all, my name is Jim and I am new to the Lime Technology. I am excited to start with this great product. Plz if this is already solved of this is the wrong thread plz tell so and point me out I have a problem starting VM’s that are created through the Web gui. Let me first list down my setup: Hardware: Mobo Asus Rampage III Extreme (both Mem 12GB (3 x 4GB) CPU i7-970 hexacore GFX AMD HD 6970 x 3 HDD1 OCZ Revo 120GB PCI-E HDD2 WD Green 1TB BOOT USB 16GB unRaid config: No parity Single disk raid 1 share bridged br0 Software: unRaid v6 Trial license (want to try it first ofc) VM: Windows 8.1 Installed through VNC After we created a virtual machine and we want to start the machine we get the following error. vfio: failed to set iommu for container: Operation not permitted vfio: failed to setup container for group # (depends on which VGA we passtrough but always same error) vfio: failed to get group # Device initialization failed Device 'vfio-pci' could not be initialized If we choose an other GFX card the message is the same execpt for the different device number (GFX). I think it's has something to do with the GFX card. I am able to install and start a virtual machine with VNC mode but I am not ablo to start it with direct i/o. Did anybody else ran into this problem? Is it my hardware configuration? What is the minimum setup for two GFX cards? Thank you in advanced, I realy would like to start using this awesome software! Greetings, Jim
January 20, 201610 yr Author Thank you CHBMB, I will let you know today if it's a succes. I think the extra article of Alex Williamson will help a lot.
January 20, 201610 yr Author I tried the ACS on and off... It didn't make a different. The bios change from SeaBIOS to OFMV didn't work either. Adjusting machine type from i440fx to Q35 also to no avail. And the manual ROM files also didn't work. In the syslog I get the message "BIOS IS BROKEN"... below a excerpt of the error: Jan 20 19:28:52 Tower kernel: WARNING: CPU: 6 PID: 1 at drivers/iommu/intel-iommu.c:4744 init_dmars+0x258/0x454() Jan 20 19:28:52 Tower kernel: Your BIOS is broken; DMA routed to ISOCH DMAR unit but no TLB space. Jan 20 19:28:52 Tower kernel: BIOS vendor: American Megatrends Inc.; Ver: 1502 ; Product Version: System Version Jan 20 19:28:52 Tower kernel: Modules linked in: Jan 20 19:28:52 Tower kernel: CPU: 6 PID: 1 Comm: swapper/0 Tainted: G I 4.1.15-unRAID #1 Jan 20 19:28:52 Tower kernel: Hardware name: System manufacturer System Product Name/Rampage III Extreme, BIOS 1502 10/03/2011 Jan 20 19:28:52 Tower kernel: 0000000000000009 ffff880351137d38 ffffffff815f1ad0 0000000000001d2c Jan 20 19:28:52 Tower kernel: ffff880351137d88 ffff880351137d78 ffffffff8104775b ffff88035093c400 Jan 20 19:28:52 Tower kernel: ffffffff818d6767 ffff880350ece000 0000000000000000 ffff88034d636180 Jan 20 19:28:52 Tower kernel: Call Trace: Jan 20 19:28:52 Tower kernel: [<ffffffff815f1ad0>] dump_stack+0x4c/0x6e Jan 20 19:28:52 Tower kernel: [<ffffffff8104775b>] warn_slowpath_common+0x97/0xb1 Jan 20 19:28:52 Tower kernel: [<ffffffff818d6767>] ? init_dmars+0x258/0x454 Jan 20 19:28:52 Tower kernel: [<ffffffff810477b6>] warn_slowpath_fmt+0x41/0x43 Jan 20 19:28:52 Tower kernel: [<ffffffff818d6767>] init_dmars+0x258/0x454 Jan 20 19:28:52 Tower kernel: [<ffffffff818d6bf8>] intel_iommu_init+0x295/0x585 Jan 20 19:28:52 Tower kernel: [<ffffffff818ac2a8>] pci_iommu_init+0x13/0x3e Jan 20 19:28:52 Tower kernel: [<ffffffff818ac295>] ? memblock_find_dma_reserve+0x146/0x146 Jan 20 19:28:52 Tower kernel: [<ffffffff81002145>] do_one_initcall+0xe9/0x178 Jan 20 19:28:52 Tower kernel: [<ffffffff818a405d>] kernel_init_freeable+0x17c/0x204 Jan 20 19:28:52 Tower kernel: [<ffffffff818a3807>] ? initcall_blacklist+0xa5/0xa5 Jan 20 19:28:52 Tower kernel: [<ffffffff815e4da5>] ? rest_init+0x89/0x89 Jan 20 19:28:52 Tower kernel: [<ffffffff815e4dae>] kernel_init+0x9/0xd5 Jan 20 19:28:52 Tower kernel: [<ffffffff815f75d2>] ret_from_fork+0x42/0x70 Jan 20 19:28:52 Tower kernel: [<ffffffff815e4da5>] ? rest_init+0x89/0x89 Jan 20 19:28:52 Tower kernel: ---[ end trace 2470d3252461daf9 ]--- Jan 20 19:28:52 Tower kernel: IOMMU: hardware identity mapping for device 0000:00:1b.0 Jan 20 19:28:52 Tower kernel: IOMMU: Setting RMRR: Jan 20 19:28:52 Tower kernel: IOMMU: Setting identity map for device 0000:00:1a.0 [0x9f7dc000 - 0x9f7dbfff] Jan 20 19:28:52 Tower kernel: ------------[ cut here ]------------ I hope to solve this puzzle.... Greetings , Jim
January 20, 201610 yr In the syslog I get the message "BIOS IS BROKEN"... below a excerpt of the error: I've seen a post with this in before as well, but can't remember where, and am pretty sure jonp ended up commenting. Not terribly helpful I know, but do some searching and hopefully it'll turn up..
January 20, 201610 yr Author There is a post about a guy with the same motherboard, but it didn't showup with a solution. Thread: https://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107
January 20, 201610 yr There is a post about a guy with the same motherboard, but it didn't showup with a solution. Thread: https://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107 That was the one.... Try asking jonp if there's been any developments on that front..
January 20, 201610 yr I've seen a post with this in before as well, but can't remember where, and am pretty sure jonp ended up commenting. Not terribly helpful I know, but do some searching and hopefully it'll turn up.. I'm bored, here http://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107
January 20, 201610 yr I've seen a post with this in before as well, but can't remember where, and am pretty sure jonp ended up commenting. Not terribly helpful I know, but do some searching and hopefully it'll turn up.. I'm bored, here http://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107 Jim beat you to it...
January 20, 201610 yr I've seen a post with this in before as well, but can't remember where, and am pretty sure jonp ended up commenting. Not terribly helpful I know, but do some searching and hopefully it'll turn up.. I'm bored, here http://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107 Jim beat you to it... I just refreshed and seen that.. Stupid work!
January 20, 201610 yr I wonder, have you tried with just two graphics cards, and perhaps with the second graphics card in the slot farthest from the first? Just a hunch.
January 20, 201610 yr I wonder, have you tried with just two graphics cards, and perhaps with the second graphics card in the slot farthest from the first? Just a hunch. As far as I'm aware, passthrough of two gfx cards isn't supported, although may be in the future? I'm of the distinct opinion that despite LinusTechTips and all the other "Wow, this is amazing and easy" videos out there, that virtualisation on Unraid can be problematic and nothing has yet convinced me that a large source of the problems is that some hardware just doesn't play nice.....
January 20, 201610 yr His motherboard doesn't have onboard video, so one graphics card would be for UnRaid, whilst the second would be for his virtual machine. In his hardware summary he lists his graphics card x 3 which suggests three graphics cards and an attempt to passthrough two, which as you say, is not supported, hence my suggestion of removing one graphics card and trying with only two.
January 20, 201610 yr Author Hey we made some progres. atm we are using 2 6970 and a lame old x1500 (X1500 for unraid and 1 for a vm win We got it to work with following sysconfig: default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append vfio_iommu_type1.allow_unsafe_interrupts=1 iommu=pt initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest We are going to test later if the unsafe interupts is needed or not.
January 20, 201610 yr His motherboard doesn't have onboard video, so one graphics card would be for UnRaid, whilst the second would be for his virtual machine. In his hardware summary he lists his graphics card x 3 which suggests three graphics cards and an attempt to passthrough two, which as you say, is not supported, hence my suggestion of removing one graphics card and trying with only two. Ah, gotcha...
January 20, 201610 yr Author We dont know exactly why but after some testing. This config works: default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append iommu=pt initrd=/bzroot append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest
January 22, 201610 yr Author Well we got it working and it works pretty wel. What I notice is that when you do a GFX benchmark on two VM's the score is lower then doing the same on one VM. I thought the scores would probably end near to eachother because I thought the systems (GFX's) are working alone. So how come my GFX score is about 50% when doing a single VM GFX benchmark? I understand that there is always some lost in performance as you share the lanes on the motherboard and 1 CPU. Is this maybe a PCI-X lane issue? Or is it normal behaviour? Thank you in advanced. The config files an screenshots will follow this weekend for other users to use.
February 4, 201610 yr Author We got it working steady now. I posted my solution in an other threat. http://lime-technology.com/forum/index.php?topic=45530.msg440853#msg440853 Thank you al for your support. I'm still looking for the best way to select my processors and HT cores, cpu scheduling esxi wise.
Archived
This topic is now archived and is closed to further replies.