ccowin
-
Posts
8 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by ccowin
-
-
I’ve tried both with no success either way
-
I got it running on the old 2080ti but I cant get it working on the new 4090. Always a black screen.
-
So I've tried for hours and hours and hours to get this passthrough to work and it just never fucking shows any video at all through the GPU. I've tried with two GPUs. I'm really really hoping someone here can throw me a bone and help me troubleshoot, I feel like I'm shouting into the void and have no idea what else to try. I feel like I've tried literally every single possible option of settings and it just doesn't work.
I recently got a new GPU and was able to slot it in to my 6.11 build pretty seamlessly. It worked great in my existing VM and all was good. Well then 6.12 released and I messed something up when transferring the vdisk off my cache to convert it to ZFS. Well that's annoying but all my data is still there otherwise so just needed a new OS set up. However now I can't for the life of me figure out how to get the passthrough working again. I've tried also going back to the old GPU and getting that working and now that doesnt even work.
This is my running ist of things I've tried:
All my BIOS settings are fine CSM and IOMMU are enabled
The vfio log says everything is succesfully attatched
Adding multifunction='on' and changing the sound card to match the bus of the video card
Tried OVM/SeaBIOS - No idea which one is better
Tried UEFI/Legacy - I don't know which one is better or not, literally will be two forum replys with completely different info on which one works better
Tried no vBios, dumped vBios and one from techpowerup with the header removed (wasnt 100% sure I have the TUF 4090 and Techpowerup only had one for the strix but I assumed they were close enough).
Have a HDMI and DP monitor attatched to GPU (I read somewhere it might need hdmi?)
Syslinux:
kernel /bzimage
append initrd=/bzroot vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream,multifunction video=efifb:offEverything seems to launch fine, and I can access the VM from RDP, but the GPU is not recognized by the VM and nothing shows up on any of the monitors.
Thanks for any help you can provide.
Diagnostics for Arch is 1950
Diagnostics for W10 is 1956
tower-diagnostics-20230625-1950.zip tower-diagnostics-20230625-1956.zip
-
1
-
-
Hello,
So I recently got a new GPU and was able to slot it in to my 6.11 build pretty seamlessly. It worked great in my existing VM and all was good. Well then 6.12 released and I messed something up when transferring the vdisk off my cache to convert it to ZFS. Well thats annoying but all my data is still there otherwise so just needed a new OS set up. However now I can't for the life of me figure out how to get the passthrough working again.
- All my BIOS settings are fine CSM and IOMMU are enabled
- Adding multifunction='on' and changing the sound card to match the bus of the video card
- Tried also adding this other bit to the xml for GPUs with high memory amounts:
- Tried OVM/SeaBIOS
- Tried UEFI/Legacy
- Tried no vBios, dumped vBios and one from techpowerup with the header removed (wasnt 100% sure I have the TUF 4090 and Techpowerup only had one for the strix but I assumed they were close enough).
- Have a HDMI and DP monitor attatched to GPU
-
Syslinux:
-
kernel /bzimage
append initrd=/bzroot vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream,multifunction video=efifb:off
-
kernel /bzimage
Everything seems to launch fine, and I can access the VM from RDP, but the GPU is not recognized by the VM and nothing shows up on any of the monitors.
Thanks for any help you can provide.
e: As I was writing this I think the SeaBIOS version actually worked but now it wont boot at all.. I'll play around with it more and report back. I unfortunately didnt attatch any USB devices because I just wanted to make sure it works. -
Good Morning,
I had some issues a few days ago and had to do a hard reset of my unraid pc. Spent all night doing the parity check and when I woke up it finished the parity check but then didnt start the array, so I started it again but then it wanted to do an entire other parity check. I stopped it because it was a waste of uses on the drive (and i didnt really feel like waiting 12 hours again). The array then started but I am now getting this error: "unmountable: no file system" which is not true there 100% is a file system on there so I don't really know what happened or how to get my cache drive to mount again. This has happened once already and I formatted the drive and moved everything back over and thought it was working for a while but then this returned. I'd format it again but I don't think that is the problem.
Thanks for any help you can provide.
This diagnostic is without any dockers/vms running. If you think that would be helpful I can provide those as well, but thought this would be more concise without them since the issue is happening before docker/vms start.
tower-diagnostics-20230606-0958.zip
e: Tried formatting the cache drive and think I found the issue but I'm not sure how to solve it quite yet. Here is the Log:Jun 6 10:25:00 Tower emhttpd: shcmd (373): mount -t btrfs -o noatime,space_cache=v2,discard=async -U 081bc21f-e9a3-4e24-91cd-961901efaaa1 /mnt/cache Jun 6 10:25:00 Tower root: mount: /mnt/cache: /dev/sdb1 already mounted or mount point busy. Jun 6 10:25:00 Tower root: dmesg(1) may have more information after failed mount system call. Jun 6 10:25:00 Tower emhttpd: shcmd (373): exit status: 32 Jun 6 10:25:00 Tower emhttpd: /mnt/cache mount error: No file system Jun 6 10:25:00 Tower emhttpd: shcmd (374): umount /mnt/cache Jun 6 10:25:00 Tower root: umount: /mnt/cache: target is busy. Jun 6 10:25:00 Tower emhttpd: shcmd (374): exit status: 32 Jun 6 10:25:00 Tower emhttpd: shcmd (375): rmdir /mnt/cache Jun 6 10:25:00 Tower root: rmdir: failed to remove '/mnt/cache': Device or resource busySo obviously the second ssd in the pool was having issues so I removed it and reformatting the cache drive, but not sure why this happened or how to prevent it in the future. The second ssd passed SMART. Hopefully something I'm doing wrong is obvious.
-
I was unable to dump my own as the script wouldn't recognize my card. But I was able to get the copy online and edit with the hex editor but it still didnt work
-
Hello,
As the title states, on paper, I should be able to passthrough my GPU to a VM but it just isn't working. I've followed multiple guides but when they select the video and boot their VM it boots and mine just doesnt.
unRAID: Latest version, installed on Sunday
M/B: Micro-Star International Co., Ltd. B450 GAMING PRO CARBON AC (MS-7B85) Version 1.0 - s/n: J316294684
BIOS: American Megatrends Inc. Version 1.B0. Dated: 11/08/2019
CPU: AMD Ryzen 7 2700X Eight-Core @ 3700 MHz
GPU: Nvidia 2080 TI (MSI)
Steps:
- Set up unRAID
-
VM Manager settings:
-
PCIe ACS override: Downstream
-
Downloaded VirtIO drivers for windows
-
- Installed VM w/o passthrough (Windows 11)
- Bound IOMMU group to vfio
- Rebooted
- Changed video/sound to 2080ti
- start VM, no boot
- sad me
When that didnt work i looked on the forms and found folks adding vfio_iommu_type1.allow_unsafe_interrupts=1 and video=efifb:off to the syslinux config but neither worked independently nor together. I also tried dumping my GPU vBIOS with spaceinvaderone's script but it was saying my GPU wasn't bound when unRAID said it was. I'm going to update my bios as I noticed it was pretty old but I doubt that will do anything.
I've tried everything I can think of and everything seems to say its working but I don't know enough about this to be sure so any guidance would be greatly appreciated. Happy to provide any more logs, just not sure what you might need.
IOMMU Bind log:
Quotetext error warn system array login
Loading config from /boot/config/vfio-pci.cfg
BIND=0000:26:00.0|10de:1e07 0000:26:00.1|10de:10f7 0000:26:00.2|10de:1ad6 0000:26:00.3|10de:1ad7
---
Processing 0000:26:00.0 10de:1e07
Vendor:Device 10de:1e07 found at 0000:26:00.0IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:26:00.0/iommu_group/devices/0000:26:00.0
/sys/bus/pci/devices/0000:26:00.0/iommu_group/devices/0000:26:00.1
/sys/bus/pci/devices/0000:26:00.0/iommu_group/devices/0000:26:00.2
/sys/bus/pci/devices/0000:26:00.0/iommu_group/devices/0000:26:00.3Binding...
Unbound 0000:26:00.2 from xhci_hcd
Successfully bound the device 10de:1e07 at 0000:26:00.0 to vfio-pci
---
Processing 0000:26:00.1 10de:10f7
Vendor:Device 10de:10f7 found at 0000:26:00.1IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:26:00.1/iommu_group/devices/0000:26:00.0
/sys/bus/pci/devices/0000:26:00.1/iommu_group/devices/0000:26:00.1
/sys/bus/pci/devices/0000:26:00.1/iommu_group/devices/0000:26:00.2
/sys/bus/pci/devices/0000:26:00.1/iommu_group/devices/0000:26:00.3Binding...
0000:26:00.0 already bound to vfio-pci
0000:26:00.1 already bound to vfio-pci
0000:26:00.2 already bound to vfio-pci
0000:26:00.3 already bound to vfio-pci
Successfully bound the device 10de:10f7 at 0000:26:00.1 to vfio-pci
---
Processing 0000:26:00.2 10de:1ad6
Vendor:Device 10de:1ad6 found at 0000:26:00.2IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:26:00.2/iommu_group/devices/0000:26:00.0
/sys/bus/pci/devices/0000:26:00.2/iommu_group/devices/0000:26:00.1
/sys/bus/pci/devices/0000:26:00.2/iommu_group/devices/0000:26:00.2
/sys/bus/pci/devices/0000:26:00.2/iommu_group/devices/0000:26:00.3Binding...
0000:26:00.0 already bound to vfio-pci
0000:26:00.1 already bound to vfio-pci
0000:26:00.2 already bound to vfio-pci
0000:26:00.3 already bound to vfio-pci
Successfully bound the device 10de:1ad6 at 0000:26:00.2 to vfio-pci
---
Processing 0000:26:00.3 10de:1ad7
Vendor:Device 10de:1ad7 found at 0000:26:00.3IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:26:00.3/iommu_group/devices/0000:26:00.0
/sys/bus/pci/devices/0000:26:00.3/iommu_group/devices/0000:26:00.1
/sys/bus/pci/devices/0000:26:00.3/iommu_group/devices/0000:26:00.2
/sys/bus/pci/devices/0000:26:00.3/iommu_group/devices/0000:26:00.3Binding...
0000:26:00.0 already bound to vfio-pci
0000:26:00.1 already bound to vfio-pci
0000:26:00.2 already bound to vfio-pci
0000:26:00.3 already bound to vfio-pci
Successfully bound the device 10de:1ad7 at 0000:26:00.3 to vfio-pci
---
vfio-pci binding completeDevices listed in /sys/bus/pci/drivers/vfio-pci:
lrwxrwxrwx 1 root root 0 Apr 11 01:54 0000:26:00.0 -> ../../../../devices/pci0000:00/0000:00:03.1/0000:26:00.0
lrwxrwxrwx 1 root root 0 Apr 11 01:54 0000:26:00.1 -> ../../../../devices/pci0000:00/0000:00:03.1/0000:26:00.1
lrwxrwxrwx 1 root root 0 Apr 11 01:54 0000:26:00.2 -> ../../../../devices/pci0000:00/0000:00:03.1/0000:26:00.2
lrwxrwxrwx 1 root root 0 Apr 11 01:54 0000:26:00.3 -> ../../../../devices/pci0000:00/0000:00:03.1/0000:26:00.3VM Log:
Quotetext error warn system array login
2023-04-11 01:58:22.147+0000: Starting external device: TPM Emulator
/usr/bin/swtpm socket --ctrl type=unixio,path=/run/libvirt/qemu/swtpm/2-W11-swtpm.sock,mode=0600 --tpmstate dir=/var/lib/libvirt/swtpm/68b39582-4815-b61c-0574-370045ab8fe7/tpm2,mode=0600 --log file=/var/log/swtpm/libvirt/qemu/W11-swtpm.log --terminate --tpm2
2023-04-11 01:58:22.181+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.17-Unraid, hostname: Tower
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME=/var/lib/libvirt/qemu/domain-2-W11 \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-2-W11/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-2-W11/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-2-W11/.config \
/usr/local/sbin/qemu \
-name guest=W11,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-2-W11/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/68b39582-4815-b61c-0574-370045ab8fe7_VARS-pure-efi-tpm.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-i440fx-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
-m 12800 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":13421772800}' \
-overcommit mem-lock=off \
-smp 14,sockets=1,dies=1,cores=7,threads=2 \
-uuid 68b39582-4815-b61c-0574-370045ab8fe7 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=36,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pci.0","addr":"0x7.0x7"}' \
-device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pci.0","multifunction":true,"addr":"0x7"}' \
-device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pci.0","addr":"0x7.0x1"}' \
-device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pci.0","addr":"0x7.0x2"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x3"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/W11/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win11_22H2_English_x64v1.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.229-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-1-format","id":"ide0-0-1"}' \
-netdev tap,fd=37,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:fd:38:a5","bus":"pci.0","addr":"0x2"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=35,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-chardev socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/2-W11-swtpm.sock \
-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:26:00.0","id":"hostdev0","bus":"pci.0","addr":"0x5"}' \
-device '{"driver":"vfio-pci","host":"0000:26:00.1","id":"hostdev1","bus":"pci.0","addr":"0x6"}' \
-device '{"driver":"vfio-pci","host":"0000:26:00.2","id":"hostdev2","bus":"pci.0","addr":"0x8"}' \
-device '{"driver":"vfio-pci","host":"0000:26:00.3","id":"hostdev3","bus":"pci.0","addr":"0x9"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/006","id":"hostdev4","bus":"usb.0","port":"2"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/1 (label charserial0)-
1
Not worth the money. Fairly unhappy with my purchase.
in General Support
Posted
I just am somewhat upset with my decision to purchase unraid. In hind sight and if anyone in the future asks my recommendation I'll point them to Proxmox, the price just is not worth the very minimal level of improvement in quality.
Unraid has just been incredibly fragile, hard to understand what the problem is when there is a problem, and just incredibly time consuming. Every day it seems I'm spending more time figuring out what has randomly just stopped working with no explanation. One day my VMs just stop booting and I have no idea why, the next day my entire Docker configuration somehow gets corrupted and I have to redownload and set up everything.
This honestly feels like open source software level of quality but with a $60 price tag. One of the biggest "features" unraid offers is essentially done for Lime by the users through the community applications. I truly don't understand how Lime feel comfortable advertising that this software "supports" GPU passthrough when in my experience with multiple different GPUs it takes HOURS of tinkering, watching youtube videos, and messing with XML files to get it to work correctly and then will just break at the drop of a hat.
Didn't see anywhere to leave feedback so I did it here.