Does Bus & Slot Really Matter?

[PeteyBoPetey]

Hi Guys, (Space Invader One impersonation)

 

I know nothing about VM's or XMLing, but I love Unraid. I'm the sought of person who likes to root around under the bonnet when the car won't start and manage to get it going even though I don't know what I'm doing. I'm doing the same with Unraid.

I've noticed that when I create VM's using bus='0x00' and slots higher than slot='0x10' Nvidia throws the dreaded code 43 even though I've placed functions 1 to 3 in the same slot. Just recently I've had a Edimax bluetooth dongle spit out a code 43 error on one VM, but work perfectly fine on another?. The VM with the problems has it GPU on slot 9, I don't know if the USB hub the dongle it plugged into is on a slot higher than 10 or not, but if it is, it could be causing my code 43 problem.

I've noticed that the VM template uses up bus=0x00 slots 1 to 4 & slot 7 for something (don't know what, but all the VM's have those slots) and the VM's use up the following slots:
VM 1 GPU: Slot 5
VM 2 GPU: Slot 6
Taken by all VM: Slot 7
Taken by all VM: Slot 8
VM 3 GPU: Slot 9

I've run out of slots below 10. Can I put VM 1 on bus=0x00 slots 1-6, VM 2 on bus=01 slots 1-6 and VM 3 on bus 0x02 slots 1-6 ?

TIA

 

Peter

[ghost82]

5 hours ago, PeteyBoPetey said:

I've run out of slots below 10. Can I put VM 1 on bus=0x00 slots 1-6, VM 2 on bus=01 slots 1-6 and VM 3 on bus 0x02 slots 1-6 ?

Sorry, I'm not understanding

 

First, you need to take into consideration the 2 machine types: i440fx and q35.

They have different layouts: i440fx has only bus 0 (defined as pci-root) and all the devices attaches to bus 0, so every target address will be bus=0x00, slot=0xX, function=0x0Y.

q35 defines pcie-root as its bus 0, and you can attach to it pcie root ports, pcie to pci bridges, other things that you will unlikely (for example pcie-switch-upstream-port and pcie-expander-bus) and devices.

So, in q35, devices can be attached to bus 0 (in this case they are "built-in") or to other buses (1, 2, 3,..) defined by the Index value of the pcie root ports, pcie to pci bridges, etc.

Both i440fx and q35 support multifunction devices, same bus, same slot, different function.

 

Sometimes it's important to attach devices in a predefined target address, a couple of examples:

1. in a mac os q35 vm, a network device should be attached on bus 0 (built-in), so that it will be seen as builtin in the os, otherwise apple services will not work

2. in a mac os q35 vm, analogic audio should be attached on bus 0 (built-in), so that it will be seen as builtin in the os, otherwise analogic audio will not work

3. in all vms, a multifunction gpu should have target address defined as multifunction, otherwise it may not work because of the drivers.

[PeteyBoPetey]

2 hours ago, ghost82 said:

3. in all vms, a multifunction gpu should have target address defined as multifunction, otherwise it may not work because of the drivers.

I've followed SpaceInvaders tutorial on making sure all 3 functions are in the same slot 0=video 1=sound 2=usb controller, 3=usb port and added multifunction='on'

I don't have any problems doing this until I put them into slots 10 or higher, then I get code 43 errors. Considering slots 1-4 and 7-8 are taken by default it only leaves 5,6 and 9 available for GPU's. Given that I get code 43 errors from GPU's in slots 10 or higher, is it possible that other devices in slots higher than 10 give code 43 errors?. If so, what would be the best bus/slot allocation?. 

[ghost82]

3 minutes ago, PeteyBoPetey said:

I don't have any problems doing this until I put them into slots 10 or higher, then I get code 43 errors

Strange, never heard of such issue.

 

3 minutes ago, PeteyBoPetey said:

Considering slots 1-4 and 7-8 are taken by default it only leaves 5,6 and 9 available for GPU's

But on which bus, and on which machine type?

for q35 you have bus 0, 1, 2, .....x and each bus have slot 0, 1, 2, ...Y.

i440fx is older than q35 chipset, q35 is optimized for pcie passthrough devices, maybe the issue is with your gpu coupled with i440fx.

[PeteyBoPetey]

14 minutes ago, ghost82 said:

Strange, never heard of such issue.

 

But on which bus, and on which machine type?

for q35 you have bus 0, 1, 2, .....x and each bus have slot 0, 1, 2, ...Y.

i440fx is older than q35 chipset, q35 is optimized for pcie passthrough devices, maybe the issue is with your gpu coupled with i440fx.

Sorry, I read your post but didn't take it in. I'm currently running i440fx on all 3 VM's. Would it be better to change to q35 and put VM 1 GPU on bus 1 slot 5, VM 2 GPU bus 2 slot 5 and VM 3 bus 3 slot 5?

[ghost82]

4 minutes ago, PeteyBoPetey said:

Would it be better to change to q35 and put VM 1 GPU on bus 1 slot 5, VM 2 GPU bus 2 slot 5 and VM 3 bus 3 slot 5?

Do you have 3 gpus in one vm?

The layout is correct, but again I have no issue in putting a gpu in whatever bus/slot (bus different than 0 in a q35 machine).

And yes, I prefer q35.

[meep]

the bus/slot assignments within each vm are independent of other VMs and relate to virtual hardware of that specific vm only.

 

You seem to be saying that you are striving to use unique slots across different vms? This is not necessary. You can use bus 0, slot 5 in all vms if you want. You only need to take care that you do not try to pass the same physical device in your server to more than one vm that you intend to run simultaneously.