stottle

Well, partial success. The controller shows up in the VM (I have to use remote desktop to check, BTW), but in device manager it is showing up as not having a driver. Windows can't automatically find drivers. I wasn't sure what to put for address, so I have <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=00:14.0,bus=root.1'/> Also, it seems like the vfio-bind in my go script isn't "taking" for some reason - I have to manually enter the command once unRAID is running. Any suggestions for these two issues?

So I found this stackoverflow question which seems to be trying to tell which pci device is linked to a specific usb port. So I plugged in a 2nd flash drive (named "Test" instead of "UNRAID") into a port I was hoping to pass thru. blkid | grep -i test tells me this is /dev/sdi1 blkid | grep -i unraid tells me unRAID is /dev/sda1 lspci | grep -i usb 00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04) 00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04) 00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04) Shows three usb controllers Now figure out which is which ls -al /sys/block/sda lrwxrwxrwx 1 root root 0 Oct 12 12:34 /sys/block/sda -> ../devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1/1-1.1:1.0/host0/target0:0:0/0:0:0:0/block/sda/ ls -al /sys/block/sdi lrwxrwxrwx 1 root root 0 Oct 5 03:50 /sys/block/sdi -> ../devices/pci0000:00/0000:00:14.0/usb4/4-2/4-2:1.0/host10/target10:0:0/10:0:0:0/block/sdi/ Looking at the red text (and comparing to lspci output) I can see that unRAID is on a different controller than my "test" usb. So I can pass thru the usb controller 0000:00:14.0 - the xHCI controller. I'll post back if it works. I wanted to post this while it was still fresh.

Did you change the name from vfio-bind to vfio-pci? Or is this a typo? Also, per a previous answer from you, if you want the settings to work after a reboot, this line would need to go into your "go" script, correct?

Thanks for all of the info! Attached is a pic of the system devices. root@Tower2:/mnt/btrfs/VM/Win7# lsusb -t /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M [b]<--Not unRAID[/b] /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M |__ Port 1: Dev 5, If 0, Class=Hub, Driver=hub/3p, 12M |__ Port 2: Dev 6, If 0, Class=Human Interface Device, Driver=usbhid, 12M |__ Port 3: Dev 7, If 0, Class=Human Interface Device, Driver=usbhid, 12M /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 480M [b]<--Pretty sure unRAID[/b]

I'm making progress getting GPU passthru working, but now I need usb as well to get input devices to the VM (mouse/keyboard). I bought a hot-swap tray with included usb3 (http://www.microcenter.com/product/392923/25_-_35_Hot_Swap_Rack_w-_Key_Lock_and_SuperSpeed_USB_30_Hub). I'm thinking that rather than forward a specific usb input device (since the VM won't start if the device isn't present), I would rather forward the usb3 hub itself. I assume this would then passthru any device attached to the hub? However, the hub attaches to the usb3 internal connector on my motherboard. So it isn't showing up in lspci. Seems like I have two options. 1) Is there a way to tell from the lspci output which controller the hub is attached to? I could then passthru that whole controller, which would include the hub. 2) Pass the hub by itself. The problem here is that it is showing up in lsusb as a generic "Linux Foundation 3.0 root hub", which can have duplicate vendor/product id's with any other usb hub. I'm not sure how to specific the product/vendor plus the bus/device. I'd appreciate any recommendations on how to proceed. Thanks!

Yeah, I seem to have better luck with virsh create, in part because it gives errors in the terminal window. Little bit of a story here, but maybe it helps someone. Per your previous response on vfio-bind in the go script, I put the vfio-bind command at the start of my script. However, it doesn't seem to "take", and I don't see any error in the syslog. But if I run ls /dev/vfio , I don't see my video group (1) listed there. Without the vfio-bind, my qemu commands would give an error in virsh create. If I manually run the same vfio-bind command from a terminal window, I get the group listed in /dev/vfio, and then virsh create would succeed. I was able to get GPU passthru. However, this led to the next problem, as the usb I was trying to passthru (a BT keyboard) seems to be failing - it won't run on a non-VM. So I had no input device to enter my password to log in to windows. When I unplugged the usb, it locked up the entire unRAID server. So now it is running a parity check. Oops. So now I need to get usb passthru working. But I will start another thread for that. BTW, another advantage of virsh create - your raw xml isn't modified. Virsh create can add the address to devices and controllers that are specific to your hardware. If you post the modified xml here on the forum, I *think* it assumes you want to hardcode the address in, even if it isn't valid on someone else's system. So it seems like the input to virsh create is a "cleaner" xml to share, isn't it? Not sure there is a good solution, but wanted to mention the concern.

I'm struggling with GPU passthrough using the new plugin. It seems like the old qemu:commandline tags are ignored, and I'm not having any luck with the hostdev tags. Does anyone have GPU passthrough working for windows (I'm trying for Win7)? If so, I'd appreciate hints or the relevant pieces of your xml file. Part of my problem is that if I use dmacias' vnc settings, that allows vnc, but seems to break the GPU drivers. If I don't include the vnc settings, I can't vnc or RDP, and there is no video from the GPU DVI output. So I have to force shutdown the VM.

I know the recent Betas have had vfio-bind included with the unRAID image. Do these plugins handle the vfio-bind element of passthrough setup? Or is it necessary to add a vfio-bind call to (I assume) the go script? Thanks!

I've got the beta10 version of libvirt plugin installed, and the 2014.09.18 WebVirtMgr installed with Beta10a. When I restart my unRAID server, both virtMan and VM Manager show up in the Settings with a status of "Stopped". I thought both were supposed to start during boot, as I don't see any settings for autostart. Am I correct in thinking something is wrong because these plugins aren't starting? If so, any suggestions on troubleshooting? I do not so any errors in the log. The only thing "special" I can think of is I am mounting /mnt/btrfs (an ssd drive) in my go script, and pointing WebVirtMgr there instead of to my cache drive.

FYI, the forum front page still lists Beta 9 as the latest. I didn't look at the announcements and see beta 10 until I saw jonp's post about testing beta10 in the kvm section.

I'm near a Microcenter. They had a good deal on the Xeon E3-1245, so I picked that one. I haven't touched the VM in a bit, but I think the audio was working over HDMI.

It isn't clear to me what the current status is for the following in terms of passthru (Win7 VM) 1) Is it necessary to manually reset the passed GPU when the vm is restarted? If so, how? 2) What is the recommended way to start a VM when unRAID starts? Thanks in advance

Trying to copy from one server (reiserfs) to new server (xfs). I'm following instructions for starting rsync as a daemon from user WeeboTech. The rsync is working, but my source files are root/root for uid/gid, but the destination is getting nobody/users and seems to be failing, presumably trying to chgrp to root. That is, I'm getting rsync failed: Operation not permitted (1) My /etc/rsyncd.conf file is (from the destination machine): uid root gid root use chroot= no max connections = 4 pid file= /var/run/rsyncd.pid timeout 600 [mnt] path = /mnt comment = /mnt files read only = FALSE I'm getting 50-60 MB/sec copies, so I'd like to get this working, but I'm not sure how to either 1) change whatever permissions to allow chgrp to work 2) use the correct uid/gid to start with 3) something else I haven't thought of? Here's my rsync command from the source machine: rsync -av --stats --progress /mnt/user/Media/BluRay/5* rsync://<dest ip>/mnt/disk1/BluRay/ thanks for any help that can be provided. [edit] This is with unRAID 6 beta 9. If someone can move it, that would be great. Sorry. [/edit]

Thanks for the (prompt) reply! That seems to be working. I looked for the format option before I started the array, but didn't think to look for it after the array was started. Is there a reason why does the array need to be started to format?

I'm sure this has been posted before, but I'm not finding it. Short version: Can someone give me the shell commands to format a reiserfs disk to xfs to unRAID recognizes it? Longer version: I created a new unRAID array on beta7 (reiserFS) and copied everything from my old unRAID over. Given the write issues, I'm now wanting to start over with beta9. Since Tom mentioned going with xfs as the default OS going forward, I figured I'd try that. So I format my usb and put beta9 on it, without modifying my drives. When I enable shares and try to create one, it fails saying mkdir cannot create /mnt/users/Music. If I go into the webgui and change to reasierfs, the previous shares show up. So I've convinced myself I need to clear the drives and format as xfs, but I don't see how in the webgui. Thanks!

Thanks for the reply - it sounds like q35 is better, and the usb controllers are specific to using q35. That said, I'm not sure why you are suggesting I try i440fx.... As for the hostdev change, that is the xml syntax corresponding to the lines I had, right? So it should work with q35 or i440fx?

Sure. The Win7.xml seems to work for GPU and single usb device (usb dongle for wireless keyboard with trackpad). The .working version was what I had previously, which supported GPU passthru (this was before I tried passthru on USB). Win7.xml.txt Win7.working.xml.txt

I guess I wasn't clear on what I was after. I'm not looking for it to be simpler. I'm confused on this particular xml tag, and so I'm looking for an explanation. If you check the controllers section of the libvirt API, it says So I didn't include any controllers in my xml, but that caused the error about no usb bus. What seems odd is that there is a fixed string of xml that works (and needs to be added by me), but the string isn't dependent on my hardware. Even more confusing there are several versions with slightly different parameters (again, not tied to hardware). Hopefully you see why I find that confusing. The API (linked above) doesn't help (me) much, and google didn't point me to an answer either. So was just hoping for an answer here. Nothing's broken on my end (virtualization-wise - I copied a bunch of stuff over to reiserFS on beta7 from another unRAID server and thus potentially having corrupt copies). I was just looking for information.

I can certainly try those changes. I was hoping this part could be a little less "black magic", so if you could point me to something that describes where these parameters come from, that would be appreciated. Note, passthru is at least working with the settings I provided. Thanks for the suggestion and all of the progress on KVM!

I was trying to pass a specific usb device thru to my VM, but received an error about not having a usb bus to connect to. I added the following lines to my xml, which fixed the problem: <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0' bus='0' slot='4' function='7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0' bus='0' slot='4' function='0' multifunction='on'/> </controller> However, I've seen different values for slot and function online, and I don't know what ich9-ehci1 and ich9-uhci1 are. Can someone shed some light on what these mean, and how to decide what the values should be? Thanks!

Turns out it was a hardware issue not a software issue. I looked in the BIOS, and it listed vt-d as unsupported. Turns out the i5 3570 supports vt-d, but the i5-3570k does not. Guess which one I have. I put another CPU in, one that supports vt-d, and now I have passthru of my GPU! Note, I had included VNC in the xml config for installing windows before I tried passthru. I had to turn this off with GPU passthru, otherwise the windows vm os would complain that the GPU didn't have the necessary resources.

If anybody has suggestions for how to troubleshoot or isolate the problem, that would be greatly appreciated. I'm not sure how describe the problem to google, so I'm not getting much help from searches. TIA

Here's what it is now: root@Tower2:/sys/kernel/iommu_groups# cat /boot/syslinux/syslinux.cfg default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append intel_iommu=on,igfx_off vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot label Xen/unRAID OS Safe Mode (no plugins) kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot unraidsafemode root@Tower2:/sys/kernel/iommu_groups# I've tried without igfx_off as well, and without pcie_acs_override as well. This is an intel board/cpu (i5). I think the root of the problem is that there aren't any iommu_groups, but I'm not sure why they aren't getting created. As in, /sys/kernel/iommu_groups is an empty dir, even after running vfio-bind.

I'm trying to get video passthru working, but my updated domain.xml is producing errors. If I exclude the following lines, I can run virsh create Win7.xml and it creates/starts the VM and it runs fine. So things are working until I try passthru. With these lines included <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/> </qemu:commandline> I get the following error: root@Tower2:/mnt/btrfs/VM/Win7# virsh create Win7.xml error: Failed to create domain from Win7.xml error: internal error: early end of file from monitor: possible problem: 2014-09-01T13:46:20.027666Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error no iommu_group for device 2014-09-01T13:46:20.027706Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed. 2014-09-01T13:46:20.027723Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized root@Tower2:/mnt/btrfs/VM/Win7# I've run vfio-bind on 0000:01:00.0 and 0000:01:00.1, although I guess I'm not sure how to test if it worked. I'm running b7, and it looks like qemu.conf already includes groups 1-32, so changes described in other posts don't seem to apply anymore. I'm not sure how to troubleshoot. In case it helps, here are the devices I'm trying passthru on: root@Tower2:/mnt/btrfs/VM/Win7# lspci -knn | grep -i nvi 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108 [GeForce GT 430] [10de:0de1] (rev a1) 01:00.1 Audio device [0403]: NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1)

Ok, decided to go with btrfs instead, so I followed these instructions. I have win7 installing now, after finding the virtio-win drivers here. I wanted to make sure I could get things working without experimenting with passthru until I had the basics down. Can I start making passthru changes on the existing VM (if I shut it down)? Or does that jack up the VM, and I should create a new one? Thanks again for the help.