comfox Posted December 23, 2015 Author Share Posted December 23, 2015 I will try the suggestion from JonP posted here http://lime-technology.com/forum/index.php?topic=41922.0 SUCCESS!!! Using JonP's suggestion I am now able to get my devices working. This is great news! Onward and upward I go. Quote Link to comment
comfox Posted December 23, 2015 Author Share Posted December 23, 2015 So now that I have some USB drives connected I am trying to copy things from them but the copy is only going at 2MB/s up to 37MB/s. This is through USB 2.0. Is that normal? Seems super slow to me. Quote Link to comment
JonathanM Posted December 23, 2015 Share Posted December 23, 2015 So now that I have some USB drives connected I am trying to copy things from them but the copy is only going at 2MB/s up to 37MB/s. This is through USB 2.0. Is that normal? Seems super slow to me. You would probably get better results passing through an entire USB controller card rather than passing individual devices. Quote Link to comment
comfox Posted December 23, 2015 Author Share Posted December 23, 2015 So now that I have some USB drives connected I am trying to copy things from them but the copy is only going at 2MB/s up to 37MB/s. This is through USB 2.0. Is that normal? Seems super slow to me. You would probably get better results passing through an entire USB controller card rather than passing individual devices. I would love to do that, but unfortunately can't figure out how to do it. The last time I tried I ended up losing connectivity to my USB and my entire unRAID config including VM's got wiped. I followed the GUIDE (http://lime-technology.com/forum/index.php?topic=36768.75) and asked a question in it but haven't got a response so far... Hey all, hoping you can help me out. Is it possible to have the motherboard switch USB buses for the USB slots? I am following the instructions and moving a ADATA 32GB USB3.0 stick from slot to slot but no matter what slot I put it in the lsusb command lists the stick on buss two. Even when I pull out a usb cord from a slot that lsusb lists as bus 1 as soon as I plug in the ADATA stick lsusb then lists it as buss 2. This is making it impossible for me to figure out what USB buss to passthrough. Model: Custom M/B: MSI - Z87-G45 GAMING (MS-7821) CPU: Intel® Core™ i7-4790K CPU @ 4.00GHz HVM: Enabled IOMMU: Enabled Cache: 1024 kB, 256 kB, 8192 kB Memory: 16384 MB (max. installable capacity 32 GB) Network: eth0: 1000Mb/s - Full Duplex eth1: not connected Kernel: Linux 4.1.13-unRAID x86_64 OpenSSL: 1.0.1p Quote Link to comment
JonathanM Posted December 23, 2015 Share Posted December 23, 2015 So now that I have some USB drives connected I am trying to copy things from them but the copy is only going at 2MB/s up to 37MB/s. This is through USB 2.0. Is that normal? Seems super slow to me. You would probably get better results passing through an entire USB controller card rather than passing individual devices. I would love to do that, but unfortunately can't figure out how to do it. The last time I tried I ended up losing connectivity to my USB and my entire unRAID config including VM's got wiped. I followed the GUIDE (http://lime-technology.com/forum/index.php?topic=36768.75) and asked a question in it but haven't got a response so far... Hey all, hoping you can help me out. Is it possible to have the motherboard switch USB buses for the USB slots? I am following the instructions and moving a ADATA 32GB USB3.0 stick from slot to slot but no matter what slot I put it in the lsusb command lists the stick on buss two. Even when I pull out a usb cord from a slot that lsusb lists as bus 1 as soon as I plug in the ADATA stick lsusb then lists it as buss 2. This is making it impossible for me to figure out what USB buss to passthrough. Model: Custom M/B: MSI - Z87-G45 GAMING (MS-7821) CPU: Intel® Core™ i7-4790K CPU @ 4.00GHz HVM: Enabled IOMMU: Enabled Cache: 1024 kB, 256 kB, 8192 kB Memory: 16384 MB (max. installable capacity 32 GB) Network: eth0: 1000Mb/s - Full Duplex eth1: not connected Kernel: Linux 4.1.13-unRAID x86_64 OpenSSL: 1.0.1p What model PCIe USB card are you trying to pass? I don't see a controller card listed in your specs. Quote Link to comment
CHBMB Posted December 23, 2015 Share Posted December 23, 2015 FWIW I passed through a built in motherboard USB bus.. comfox, at the risk of repeating myself, it would help immensely if you actually posted what steps you've tried, what outputs to various commands and what alterations to the XML you've made in regard to USB passthrough.. Look at this post for an example of what I'm trying to say... Quote Link to comment
comfox Posted December 24, 2015 Author Share Posted December 24, 2015 What model PCIe USB card are you trying to pass? I don't see a controller card listed in your specs. I am trying to passthrough the built in motherboard bus. Quote Link to comment
JonathanM Posted December 24, 2015 Share Posted December 24, 2015 What model PCIe USB card are you trying to pass? I don't see a controller card listed in your specs. I am trying to passthrough the built in motherboard bus. What I tried to say earlier and obviously didn't communicate very well, was that it would probably work much better if you passed through an entire USB controller card, instead of trying to split up the motherboard ports. Quote Link to comment
comfox Posted December 24, 2015 Author Share Posted December 24, 2015 What I tried to say earlier and obviously didn't communicate very well, was that it would probably work much better if you passed through an entire USB controller card, instead of trying to split up the motherboard ports. OK thanks for the tip. I am quickly running out of PCIe ports though so would like to get the motherboard ones to work. Quote Link to comment
comfox Posted December 24, 2015 Author Share Posted December 24, 2015 Morning everyone. So I powered down the unRAID to put in a Vantec 4-Channel 6-Port PCIe to SATA card and when I went to power up the VM I get this error. internal error: early end of file from monitor: possible problem: 2015-12-24T20:28:49.798343Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver. 2015-12-24T20:28:49.798357Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1 2015-12-24T20:28:49.798362Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed 2015-12-24T20:28:49.798367Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized Also unRAID isn't showing the Corsair drive that I have connected to the Vantec in the unassigned drives spot. unRAID does see the card though under PCI devices. 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06) 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06) 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06) 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06) 00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06) 00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05) 00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04) 00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05) 00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05) 00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5) 00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5) 00:1c.6 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #7 (rev d5) 00:1c.7 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #8 (rev d5) 00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05) 00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05) 00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05) 00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05) 01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1) 01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) 02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11) 04:00.0 Ethernet controller: Qualcomm Atheros Killer E220x Gigabit Ethernet Controller (rev 13) 05:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection 06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) Here is my IOMMU grouping /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.1 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:03.0 /sys/kernel/iommu_groups/4/devices/0000:00:14.0 /sys/kernel/iommu_groups/5/devices/0000:00:16.0 /sys/kernel/iommu_groups/6/devices/0000:00:1a.0 /sys/kernel/iommu_groups/7/devices/0000:00:1b.0 /sys/kernel/iommu_groups/8/devices/0000:00:1c.0 /sys/kernel/iommu_groups/9/devices/0000:00:1c.3 /sys/kernel/iommu_groups/10/devices/0000:00:1c.6 /sys/kernel/iommu_groups/11/devices/0000:00:1c.7 /sys/kernel/iommu_groups/12/devices/0000:00:1d.0 /sys/kernel/iommu_groups/13/devices/0000:00:1f.0 /sys/kernel/iommu_groups/13/devices/0000:00:1f.2 /sys/kernel/iommu_groups/13/devices/0000:00:1f.3 /sys/kernel/iommu_groups/14/devices/0000:04:00.0 /sys/kernel/iommu_groups/15/devices/0000:05:00.0 /sys/kernel/iommu_groups/16/devices/0000:06:00.0 ******************EDIT BELOW****************** Added syslog as attachment in case it is needed. I may have answered my own questions but hoping someone will validate. I arranged my devices a little differently and see that the new SATA controller was assigned to IOMMU group 1. That is the problem right? /sys/kernel/iommu_groups/0/devices/0000:00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06) /sys/kernel/iommu_groups/1/devices/0000:00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06) /sys/kernel/iommu_groups/1/devices/0000:00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06) /sys/kernel/iommu_groups/1/devices/0000:01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1) /sys/kernel/iommu_groups/1/devices/0000:01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1) /sys/kernel/iommu_groups/1/devices/0000:02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11) /sys/kernel/iommu_groups/2/devices/0000:00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06) /sys/kernel/iommu_groups/3/devices/0000:00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06) /sys/kernel/iommu_groups/4/devices/0000:00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05) /sys/kernel/iommu_groups/5/devices/0000:00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04) /sys/kernel/iommu_groups/6/devices/0000:00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05) /sys/kernel/iommu_groups/7/devices/0000:00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05) /sys/kernel/iommu_groups/8/devices/0000:00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5) /sys/kernel/iommu_groups/9/devices/0000:00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5) /sys/kernel/iommu_groups/10/devices/0000:00:1c.6 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #7 (rev d5) /sys/kernel/iommu_groups/11/devices/0000:00:1c.7 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #8 (rev d5) /sys/kernel/iommu_groups/12/devices/0000:00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05) /sys/kernel/iommu_groups/13/devices/0000:00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05) /sys/kernel/iommu_groups/13/devices/0000:00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05) /sys/kernel/iommu_groups/13/devices/0000:00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05) /sys/kernel/iommu_groups/14/devices/0000:04:00.0 Ethernet controller: Qualcomm Atheros Killer E220x Gigabit Ethernet Controller (rev 13) /sys/kernel/iommu_groups/15/devices/0000:05:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection /sys/kernel/iommu_groups/16/devices/0000:06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) syslog.txt Quote Link to comment
CHBMB Posted December 24, 2015 Share Posted December 24, 2015 What are you trying to do with the SATA controller? Pass it through to a VM? Quote Link to comment
saarg Posted December 24, 2015 Share Posted December 24, 2015 You have to activate the PCIe ACS Override in settings --> VM manager and reboot your server. As you have found out, the sata card got in the same group as the GPU. Quote Link to comment
comfox Posted December 24, 2015 Author Share Posted December 24, 2015 What are you trying to do with the SATA controller? Pass it through to a VM? I am trying to add four more drives to unRAID. I am not trying to pass it through to Windows. You have to activate the PCIe ACS Override in settings --> VM manager and reboot your server. As you have found out, the sata card got in the same group as the GPU. Darn it all, that was my fear. When I turn on ACS Override the VM spontaneously crashes. Is there any other way to get it out of group one? Quote Link to comment
saarg Posted December 24, 2015 Share Posted December 24, 2015 You could try to change the pcie slot... Quote Link to comment
comfox Posted December 24, 2015 Author Share Posted December 24, 2015 You could try to change the pcie slot... Thanks...I did try that already. Didn't do anything. My guess is becuase it is in a PCIe x16 the mobo is going to keep it with all the other PCIe x16 devices. Quote Link to comment
comfox Posted December 26, 2015 Author Share Posted December 26, 2015 Aside form the ACS override is there any other way to remove a PCI device from a group? Quote Link to comment
saarg Posted December 26, 2015 Share Posted December 26, 2015 Aside form the ACS override is there any other way to remove a PCI device from a group? Buying a new motherboard Apart from moving the card around and the ACS override, there isn't any options as far as I know. Quote Link to comment
bungee91 Posted December 26, 2015 Share Posted December 26, 2015 I'm surprised that setting the override to on is the root of your instability. While it is certainly something that could cause issues, I don't know of anyone having issues right from the get go while using it. Overall it is typically a "safe" thing to use, just be aware of the issues with possible peer-to-peer DMA between devices that could cause problems. Quote Link to comment
comfox Posted December 26, 2015 Author Share Posted December 26, 2015 I'm surprised that setting the override to on is the root of your instability. While it is certainly something that could cause issues, I don't know of anyone having issues right from the get go while using it. Overall it is typically a "safe" thing to use, just be aware of the issues with possible peer-to-peer DMA between devices that could cause problems. Thanks encouraging to hear Bungee91. I am hopeful I can get it to work then. Unfortunately I am not skilled enough figure out the root of my stability problem then. When I turn ACS Override on, the VM won't stay up for longer than a couple dozen seconds, a few minutes if I am lucky. Any advice on where to look for the cause of the crashing would be greatly appreciated. Quote Link to comment
goober07 Posted December 28, 2015 Share Posted December 28, 2015 This may make no difference at all, but check to see if your 970 is listed a separate video and audio devices in the webUI. My 770 is. It just seems unintuitive (to me) to pass through the video-only portion of an HDMI device - who knows what the nVidia drivers expect to see. I doubt they considered the possibility of having access to part of the device blocked by the host environment. You should be able to add the onboard audio passthrough after passing the entire video card through. Quote Link to comment
comfox Posted December 28, 2015 Author Share Posted December 28, 2015 Thanks for the suggestion, I will try that. Quote Link to comment
goober07 Posted December 28, 2015 Share Posted December 28, 2015 I would definitely pass through the entire card. Your card is identified just like mine: 01:00.0 VGA 01:00.1 Audio device But Now that I looked at your IOMMU groups, it might not help at all. Group 1 contains both entries above PLUS a SATA controller and a pair of PCI bridges: 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06) 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06) 02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11) Without ACS Override or getting the GPU in another IOMMU group, everything I've read says passthrough won't work. Saarg already nailed it regarding the SATA controller - might need another motherboard. Keep digging in BIOS, but if you can't break group 1 up, I don't think you can proceed. Quote Link to comment
comfox Posted December 28, 2015 Author Share Posted December 28, 2015 This may make no difference at all, but check to see if your 970 is listed a separate video and audio devices in the webUI. My 770 is. It just seems unintuitive (to me) to pass through the video-only portion of an HDMI device - who knows what the nVidia drivers expect to see. I doubt they considered the possibility of having access to part of the device blocked by the host environment. You should be able to add the onboard audio passthrough after passing the entire video card through. Well it seems this may have made ALL THE DIFFERENCE. I am now running with ACS Override turned on and have passed through the audio and video portion of the GTX 970 and so far the VM is hasn't shut down. I am not calling this solved but considering I could only get seconds to a minute before and I am now running for 34 minutes I am going to say I am cautiously optimistic. I will keep it running for a bit and put some stress on it. Would be great if this solved it, then I can move forward to passing through the Onboard USB Controller. Thanks Goober07, seems so blatantly obvious to me now that you pointed it out. Quote Link to comment
goober07 Posted December 30, 2015 Share Posted December 30, 2015 Thanks Goober07, seems so blatantly obvious to me now that you pointed it out. Excellent! Is everything still running now that it's been a couple of days? Quote Link to comment
comfox Posted December 30, 2015 Author Share Posted December 30, 2015 Thanks Goober07, seems so blatantly obvious to me now that you pointed it out. Excellent! Is everything still running now that it's been a couple of days? So far so good! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.