GPU in IOMMU Group with 4 other devices...

[johmei]

I just had figured almost everything out with my VM and it's setup, and I was finally ready to switch from VNC to graphics card.  I thought this one will be easy!  It has a drop down box and configures itself.  Simple!  Naw, vm crashes and I can only guess it's related to my video card being in a very not group of 5 total devices.  Here are my devices and IOMMU Groups respectively

 

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)

00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)

00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)

00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 04)

00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)

00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)

00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)

00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)

00:1c.3 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 4 (rev c4)

00:1c.4 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c4)

00:1c.6 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 7 (rev c4)

00:1c.7 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 (rev c4)

00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)

00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)

00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)

00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)

01:00.0 VGA compatible controller: NVIDIA Corporation GF110 [GeForce GTX 580] (rev a1)

01:00.1 Audio device: NVIDIA Corporation GF110 High Definition Audio Controller (rev a1)

02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

03:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)

06:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03)

07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller (rev 10)

07:01.0 Multimedia audio controller: Creative Labs SB X-Fi

08:00.0 Network controller: Qualcomm Atheros AR9485 Wireless Network Adapter (rev 01)

09:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.1

/sys/kernel/iommu_groups/1/devices/0000:01:00.0

/sys/kernel/iommu_groups/1/devices/0000:01:00.1

/sys/kernel/iommu_groups/1/devices/0000:02:00.0

/sys/kernel/iommu_groups/2/devices/0000:00:02.0

/sys/kernel/iommu_groups/3/devices/0000:00:14.0

/sys/kernel/iommu_groups/4/devices/0000:00:16.0

/sys/kernel/iommu_groups/5/devices/0000:00:19.0

/sys/kernel/iommu_groups/6/devices/0000:00:1a.0

/sys/kernel/iommu_groups/7/devices/0000:00:1b.0

/sys/kernel/iommu_groups/8/devices/0000:00:1c.0

/sys/kernel/iommu_groups/9/devices/0000:00:1c.1

/sys/kernel/iommu_groups/10/devices/0000:00:1c.3

/sys/kernel/iommu_groups/11/devices/0000:00:1c.4

/sys/kernel/iommu_groups/11/devices/0000:06:00.0

/sys/kernel/iommu_groups/11/devices/0000:07:00.0

/sys/kernel/iommu_groups/11/devices/0000:07:01.0

/sys/kernel/iommu_groups/12/devices/0000:00:1c.6

/sys/kernel/iommu_groups/13/devices/0000:00:1c.7

/sys/kernel/iommu_groups/14/devices/0000:00:1d.0

/sys/kernel/iommu_groups/15/devices/0000:00:1f.0

/sys/kernel/iommu_groups/15/devices/0000:00:1f.2

/sys/kernel/iommu_groups/15/devices/0000:00:1f.3

/sys/kernel/iommu_groups/16/devices/0000:03:00.0

/sys/kernel/iommu_groups/17/devices/0000:04:00.0

/sys/kernel/iommu_groups/18/devices/0000:05:00.0

/sys/kernel/iommu_groups/19/devices/0000:08:00.0

/sys/kernel/iommu_groups/20/devices/0000:09:00.0

 

and of course, the error that pops up

 

internal error: early end of file from monitor: possible problem:

2016-01-28T06:09:59.583927Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

2016-01-28T06:09:59.583950Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

2016-01-28T06:09:59.583957Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed

2016-01-28T06:09:59.583965Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

I'm about to collapse and have to give up for the night, but hopefully someone will have some fresh ideas for me to try after work tomorrow   Thank you very much!

[saarg]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

[johmei]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

 

Actually, I did.  People like you can't seem to grasp the concept that sometimes it's difficult to figure out exactly how to search for what you're looking for.  I do not appreciate your attitude, nor do I deserve it.  You also apparently have a difficult time grasping the concept of similar/same problems being expressed in several different ways which makes it EASIER for people using Google of forums search to find the solution to their problems.  Then there's also the concept of a person being dead tired, staying up trying to fix an issue and just can't figure it out so they ask in a forum, only to be smacked in the face with an attitude like yours.

[CHBMB]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

 

Actually, I did.  People like you can't seem to grasp the concept that sometimes it's difficult to figure out exactly how to search for what you're looking for.  I do not appreciate your attitude, nor do I deserve it.  You also apparently have a difficult time grasping the concept of similar/same problems being expressed in several different ways which makes it EASIER for people using Google of forums search to find the solution to their problems.  Then there's also the concept of a person being dead tired, staying up trying to fix an issue and just can't figure it out so they ask in a forum, only to be smacked in the face with an attitude like yours.

 

Saarg, if you'd care to look at his post history is one of the more active and helpful members around here, and maybe, just maybe, you are reading too much into his post....

 

As someone who also posts a significant amount in the KVM forum helping people like yourself, I know there are an awful lot of posts in here with similar problems as I also post the same things over and over again, as does Saarg.

 

Now this is a friendly place, and we don't have attitude in here by and large, which is one of the hallmarks of the Unraid forums that they are a friendly place.  I'm willing to put your post above down to you being tired and frustrated, and suggest that you also step back and realise that maybe Saarg didn't mean it in quite the way you took it. (I know English isn't his first language) and if you're tired and frustrated then sleep on things and approach it with a fresh mind the next day.

 

Plus, don't upset the very people you may well find are the ones you depend upon for help....    After all he did post a possible solution.....

 

 

 

[bungee91]

Back on topic.............

 

So you have 2 devices (actual devices, don't care about the bridge) in group 1 and UnRAID grabs this NIC and is resulting in your errors.

Since you have other NIC's for UnRAID you can do this 2 ways (but both start about the same).

You need to "stub" the NIC 02:00:00 in your syslinux.cfg file as such

pci-stub.ids=(add device info here) from the output of this command

lspci -n

It should look something like this pci-stub.ids=8086:8c31

Add this after the append but before initrd=/bzroot in the syslinux.cfg file.

Details are in this thread for stubbing https://lime-technology.com/forum/index.php?topic=39638.0

 

This will prevent UnRAID from using this NIC.

 

Now you can either just set your VM and it should fire up with the GPU you assigned or (more useful) you pass both the GPU and NIC (since it is effectively doing nothing at this point) to the VM and rejoice.

 

Let us know if that solves your issues.

[johmei]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

 

Actually, I did.  People like you can't seem to grasp the concept that sometimes it's difficult to figure out exactly how to search for what you're looking for.  I do not appreciate your attitude, nor do I deserve it.  You also apparently have a difficult time grasping the concept of similar/same problems being expressed in several different ways which makes it EASIER for people using Google of forums search to find the solution to their problems.  Then there's also the concept of a person being dead tired, staying up trying to fix an issue and just can't figure it out so they ask in a forum, only to be smacked in the face with an attitude like yours.

 

Saarg, if you'd care to look at his post history is one of the more active and helpful members around here, and maybe, just maybe, you are reading too much into his post....

 

As someone who also posts a significant amount in the KVM forum helping people like yourself, I know there are an awful lot of posts in here with similar problems as I also post the same things over and over again, as does Saarg.

 

Now this is a friendly place, and we don't have attitude in here by and large, which is one of the hallmarks of the Unraid forums that they are a friendly place.  I'm willing to put your post above down to you being tired and frustrated, and suggest that you also step back and realise that maybe Saarg didn't mean it in quite the way you took it. (I know English isn't his first language) and if you're tired and frustrated then sleep on things and approach it with a fresh mind the next day.

 

Plus, don't upset the very people you may well find are the ones you depend upon for help....    After all he did post a possible solution.....

 

 

 

 

 

Maybe english isn't his first language, but I've read several of his posts and I would never have known that if you hadn't told me.  He knew exactly what he was saying, he even rolled eyes at me.  That's not looking too much into it, that's what it was plain and simple.  I will not allow someone to speak to me that way and my response was appropriate.  I did not cuss, scream, use caps, or otherwise insult him.  I very simply stood my ground said I do NOT appreciate being talked to that way, and I don't.  The way he spoke to me was rude and inappropriate and I am absolutely shocked that you are defending him and are unable to see how he was being rude.  I spent HOURS and HOURS trying to get my VM running right; watching videos, reading what I could find, and searching both Google and the forums.  It is a slap in the face when after all the time I spent trying to figure this out, he basically says I didn't bother to even try...it's clear to me that neither you nor saarg are anyone I wish to interact with in the future.  I will not apologize for defending myself.  My initial post was friendly, and then the very first response I got was a rude answer.  I did nothing wrong, nor am I doing anything wrong defending myself against you with this post.

[CHBMB]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

 

Actually, I did.  People like you can't seem to grasp the concept that sometimes it's difficult to figure out exactly how to search for what you're looking for.  I do not appreciate your attitude, nor do I deserve it.  You also apparently have a difficult time grasping the concept of similar/same problems being expressed in several different ways which makes it EASIER for people using Google of forums search to find the solution to their problems.  Then there's also the concept of a person being dead tired, staying up trying to fix an issue and just can't figure it out so they ask in a forum, only to be smacked in the face with an attitude like yours.

 

Saarg, if you'd care to look at his post history is one of the more active and helpful members around here, and maybe, just maybe, you are reading too much into his post....

 

As someone who also posts a significant amount in the KVM forum helping people like yourself, I know there are an awful lot of posts in here with similar problems as I also post the same things over and over again, as does Saarg.

 

Now this is a friendly place, and we don't have attitude in here by and large, which is one of the hallmarks of the Unraid forums that they are a friendly place.  I'm willing to put your post above down to you being tired and frustrated, and suggest that you also step back and realise that maybe Saarg didn't mean it in quite the way you took it. (I know English isn't his first language) and if you're tired and frustrated then sleep on things and approach it with a fresh mind the next day.

 

Plus, don't upset the very people you may well find are the ones you depend upon for help....    After all he did post a possible solution.....

 

 

 

 

 

Maybe english isn't his first language, but I've read several of his posts and I would never have known that if you hadn't told me.  He knew exactly what he was saying, he even rolled eyes at me.  That's not looking too much into it, that's what it was plain and simple.  I will not allow someone to speak to me that way and my response was appropriate.  I did not cuss, scream, use caps, or otherwise insult him.  I very simply stood my ground said I do NOT appreciate being talked to that way, and I don't.  The way he spoke to me was rude and inappropriate and I am absolutely shocked that you are defending him and are unable to see how he was being rude.  I spent HOURS and HOURS trying to get my VM running right; watching videos, reading what I could find, and searching both Google and the forums.  It is a slap in the face when after all the time I spent trying to figure this out, he basically says I didn't bother to even try...it's clear to me that neither you nor saarg are anyone I wish to interact with in the future.  I will not apologize for defending myself.  My initial post was friendly, and then the very first response I got was a rude answer.  I did nothing wrong, nor am I doing anything wrong defending myself against you with this post.

 

That's your opinion and you're entitled to it.  And I am also entitled to mine.  But, you're relatively new to the forum and both of your threads I have responded to have been with you expressing some degree of anger, so forgive me if that hasn't created the best of impressions with myself.  So let's just both agree to leave it there shall we.

 

EDIT: Sorry, just seen that you have decided you don't wish to interact with me.  I will of course make a point of avoiding to do so in the future.  Which is a shame, as I'd like to think I'm pretty helpful around here sometimes.  As I tried to be when I responded to your original thread. 

[johmei]

Back on topic.............

 

So you have 2 devices (actual devices, don't care about the bridge) in group 1 and UnRAID grabs this NIC and is resulting in your errors.

Since you have other NIC's for UnRAID you can do this 2 ways (but both start about the same).

You need to "stub" the NIC 02:00:00 in your syslinux.cfg file as such

pci-stub.ids=(add device info here) from the output of this command

lspci -n

It should look something like this pci-stub.ids=8086:8c31

Add this after the append but before initrd=/bzroot in the syslinux.cfg file.

Details are in this thread for stubbing https://lime-technology.com/forum/index.php?topic=39638.0

 

This will prevent UnRAID from using this NIC.

 

Now you can either just set your VM and it should fire up with the GPU you assigned or (more useful) you pass both the GPU and NIC (since it is effectively doing nothing at this point) to the VM and rejoice.

 

Let us know if that solves your issues.

 

Thanks!  I'll check out that thread to see what more I can learn about stubbing.  I found out that it simply would not work with my GTX 580, so I bought a GTX 950 and it's working (I had also enabled PCIe ACS Override) but I'm having trouble with my PCI Auzentech X-Fi Prelude 7.1 so maybe stubbing can help with that.  It looks like it passes through, but when I go to install the drivers, it says that windows found driver software for your device but encountered an error while attempting to install it.  When I boot directly to a HDD on the same machine with a windows install on it, the very same drivers work without issue so I can only imagine it's related to how unRAID is passing it through somehow.  But that may be for another thread.

[saarg]

I guess you didn't read this or search the forum 

You have to enable the ACS override in the VM Manager settings and reboot your server. This will make your devices go in their own iommu groups.

 

Actually, I did.  People like you can't seem to grasp the concept that sometimes it's difficult to figure out exactly how to search for what you're looking for.  I do not appreciate your attitude, nor do I deserve it.  You also apparently have a difficult time grasping the concept of similar/same problems being expressed in several different ways which makes it EASIER for people using Google of forums search to find the solution to their problems.  Then there's also the concept of a person being dead tired, staying up trying to fix an issue and just can't figure it out so they ask in a forum, only to be smacked in the face with an attitude like yours.

 

Saarg, if you'd care to look at his post history is one of the more active and helpful members around here, and maybe, just maybe, you are reading too much into his post....

 

As someone who also posts a significant amount in the KVM forum helping people like yourself, I know there are an awful lot of posts in here with similar problems as I also post the same things over and over again, as does Saarg.

 

Now this is a friendly place, and we don't have attitude in here by and large, which is one of the hallmarks of the Unraid forums that they are a friendly place.  I'm willing to put your post above down to you being tired and frustrated, and suggest that you also step back and realise that maybe Saarg didn't mean it in quite the way you took it. (I know English isn't his first language) and if you're tired and frustrated then sleep on things and approach it with a fresh mind the next day.

 

Plus, don't upset the very people you may well find are the ones you depend upon for help....    After all he did post a possible solution.....

 

 

 

 

 

Maybe english isn't his first language, but I've read several of his posts and I would never have known that if you hadn't told me.  He knew exactly what he was saying, he even rolled eyes at me.  That's not looking too much into it, that's what it was plain and simple.  I will not allow someone to speak to me that way and my response was appropriate.  I did not cuss, scream, use caps, or otherwise insult him.  I very simply stood my ground said I do NOT appreciate being talked to that way, and I don't.  The way he spoke to me was rude and inappropriate and I am absolutely shocked that you are defending him and are unable to see how he was being rude.  I spent HOURS and HOURS trying to get my VM running right; watching videos, reading what I could find, and searching both Google and the forums.  It is a slap in the face when after all the time I spent trying to figure this out, he basically says I didn't bother to even try...it's clear to me that neither you nor saarg are anyone I wish to interact with in the future.  I will not apologize for defending myself.  My initial post was friendly, and then the very first response I got was a rude answer.  I did nothing wrong, nor am I doing anything wrong defending myself against you with this post.

 

I'm sorry to hear that this was such a bad experience. That was not the intention.

I think you put to much into the smiley, interpreting it as insulting and rude. I guess that is the problem with Smiley's. They are not interpreted the same by the writer and reader some times.

 

From the info you gave us in your post it didn't look like you had read up on the subject as there were almost no info about what you have tried up to the point of the error message. I can't assume what you have tried or not, so therefore I start with the basic steps.

 

So, should we get back to fixing your problem?

What have you tried so far?

An XML of the VM would be nice to have.