cap089

@bastl There was no overlapping and I thought isolating means the cores are permanently reserved for this specific VM (?) But anyway I pinned the Dockers to specific cores/ threads and it didn't changed anything. Even a restart of the whole server did not help. Yesterday there were also Dockers running in the background like Krusader, Plex, Makemkv.. I don't know what changed overnight. Just the fact, that the file transfer is completed.

Hi! Ok everything seemed to work so far, until today... Yesterday the file transfer was running and gaming in the Windows KVM was no issue. Today I set up some Docker-Apps and recognized an overhead in the Windows KVM. I just tested Battelfield V and moving the mouse was lagging as hell... It is no GPU issue, I have constantly about 90-100 fps in this game, as I had before on the baremetal system. Yesterday I already set up the CPU isolation for the gaming VM and also used the "MSI tool". Actually I have no idea why the overhead is now present or rather so intense. Should I try to switch from i440fx-4.0 to Q35? That is what I found on my research in the web and the forum...

Hi! I scanned my music folder and it missed some songs. How can I figure out which files are corrupted or not readable? Actual it left out three songs. When I connect to iTunes it seems to be that seven songs are missing.

Ah ok. So I got you right. Thats the way I have done it the whole time. EDIT: Ok sometimes solutions are that easy that you cant see them at first sight.🙈 I have higher quality active speakers which I normally plugged in via optical cable. Besides that they have normal AUX and USB! So I just use them via USB now and have sound in my VM! For now I let the machine running and let unRAID finish the Parity-Sync and hope that everything runs stable. After that I can kick on the big media file transfer..

Sorry I don't get it. What do you mean? For the USB controllers I used the XML view. And for the soundcard I used the Web UI. So you think I should try it here also with the XML view? Anyway I will check out soundcards these days...

Yes I need them. I want to use hot plug with USB devices and I have a dedicated Ethernet for the VM and for the unRAID Server. And with the USB controller I had no problems so far.

Thats an argument . But well the problem with the onboard sound is more important to fix and if I buy an PCI Soundcard then I first have to find a card that fits my needs.

I want everything in one case and I just need it for Makemkv. Yeah I know there are some Docker versions of Makemkv but I also "cross-read" about some weird problems.

Buying some extra stuff was not my plan. This week a new BIOS update for my motherboard was released but here I have the same issue with the onboard audio.. The next thing which annoys me on unRAID is that it seems to be impossible to just passthrough a blu ray drive like any other SATA drive... So I have to think about, if I can live with these compromises.

UPDATE: I just tested out v 6.8 rc4 and now the GPU passthrough seems to work. Also a restart was no problem, so far. But now I got another problem: When I try to passthrough the onboard soundchip the system crashes. Its also the origin for the error which I posted before: Tower kernel: vfio-pci 0000:0b:00.4: not ready 1023ms after FLR; waiting I tried to unbind it manually via "pci-pci.ids" in the boot parameters, as I had done succesfully with the Realtek ethernet controller, but this did not help. And of course the soundchip is in his own IOMMU group and also I selected it as the second audio device (first one is the GPU audio).. Any ideas, please? EDIT: I actually tried this method, but didn't work..

Hi, I checked out the latest 6.8 rc3 with the new custom kernel modules which are provided in this thread: At all 6.8 seems to run smoother and robust but now it seems to be that the latest BIOS for my motherboard (from: 10/07/19) which make its impossible to passthrough any device. I tried out without the GPU passthrough and just my SSD, USB controllers and onboard sound and in the system-log I get errors, like: Tower kernel: vfio-pci 0000:0b:00.4: not ready 1023ms after FLR; waiting I guess I have to wait for a next BIOS update and hopefully then everything will work...

Hi! This week a new BIOS update launched and now also the first beta version of 6.8 came out. So I thought I try it again... In general the CPU performance is worse with the new BIOS update - but thats another site.. With unRAID 6.8 I have the exact same issues. It crashes when I try to start the Windows VM with the GPU passthrough. Even when I manually stop the parity sync it crashes. The GPU unbind script also not work anymore with 6.8 - so there are more changes made... Unbind it manually with the unbind command in /sys/pci/drivers/vfio-pci/ is also not working.

Ok testing is done.... At first the good news: I found an hidden additional setting for legacy support, after enabling it I was able to boot unRAID without UEFI. Directly to the bad news: 1. I made a complete new unRAID stick with v6.6.7 and also erased the disks of my array. After setting up the disk again and parity sync started I got the "well known" kernel panic: kernel panic - not syncing: fatal exception in interrupt [...] At this point I had not even touched the VM tab... 2. Switched to v6.7.2 in legacy mode -> endless loading after rebuild sync started 3. Tried it with v6.7.3 rc4 -> crashes immediately at startup 4. So Ok - tried it again with v6.6.7. Now I got no kernel panic and set up the Windows VM. I activated PCI ACS Override - otherwise it does not fit my needs... I also had to activate again "VFIO allow unsafe interrupts" because otherwise the Windows KVM was not able to boot. With these 2 options I could actually boot into Windows and I just started up a game and ran a Cinebench. So far so good... Then I tried to reboot the WM and after I logged in to Windows the whole WM crashed. In the unRAID Web UI I had again endless loading on the "WM" tab and after a while unRAID crashed completely. I tried three reboots. Two times it stucked/ crashed during booting and at the third time it booted up but the Web UI was not reachable under the shown IP adress. Clearing the browser cache did not help. 5. Finally I just remembered that sometimes an additional boot-parameter is necessary on Ryzen CPUs. So I again installed everything new and added the parameter: rcu_nocbs=0-23 But with this the system was even more unstable and crashed again. At all I think it doesn't make sense to go on with testing (?) My plan is to wait for unRAID 6.8 and BIOS updates. And if nothing happens during the next two months and I take this as a lesson and go back to a stable Synology NAS.

Thanks for your reply. Ok then I will do some testing in the next days: - disable "VFIO unsafe interrupts" - disable PCI ACS override, but I thinkt thats what I need because otherwise I can forget about to passthrough any USB controller... Regarding the BIOS settings. Yes I know the manual is not worth it... Right from the beginning IOMMU support was enabled or rather it was on "Auto" so I switched the setting to "Enabled" just to be sure. The same thing I made with the "legacy" boot support. But thats what I mentioned earlier, I think here the BIOS of my mainboard behaves buggy or something else. But I will check it out again.

Hi I tried it without UEFI at first and during my tests with other versions of unRAID sometimes I forgot to set the checkmark "Allow UEFI boot" in the USB Creator app and every time I missed the checkmark the stick did not show up in the Boot menu... Ok at first the IOMMU groups. I enabled PCIe ACS Override and set it to "Both", also I allowed the VFIO unsafe interrupts. IOMMU group 0: [1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 1: [1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1483 IOMMU group 2: [1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 3: [1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 4: [1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1483 IOMMU group 5: [1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 6: [1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 7: [1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 8: [1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1484 IOMMU group 9: [1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1482 IOMMU group 10: [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1484 IOMMU group 11: [1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1484 IOMMU group 12: [1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1484 IOMMU group 13: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 14: [1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1440 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1441 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1442 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1443 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1444 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1445 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1446 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1447 IOMMU group 15: [1022:57ad] 01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57ad IOMMU group 16: [1022:57a3] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 17: [1022:57a3] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 18: [1022:57a3] 02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 19: [1022:57a4] 02:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 IOMMU group 20: [1022:57a4] 02:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 IOMMU group 21: [1022:57a4] 02:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 IOMMU group 22: [8086:2723] 03:00.0 Network controller: Intel Corporation Device 2723 (rev 1a) IOMMU group 23: [10ec:8125] 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. Device 8125 IOMMU group 24: [8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 25: [1022:1485] 06:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1485 IOMMU group 26: [1022:149c] 06:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Device 149c IOMMU group 27: [1022:149c] 06:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 149c IOMMU group 28: [1022:7901] 07:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 29: [1022:7901] 08:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 30: [10de:1b80] 09:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1) IOMMU group 31: [10de:10f0] 09:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) IOMMU group 32: [1022:148a] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 148a IOMMU group 33: [1022:1485] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1485 IOMMU group 34: [1022:1486] 0b:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1486 IOMMU group 35: [1022:149c] 0b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 149c IOMMU group 36: [1022:1487] 0b:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1487 IOMMU group 37: [1022:7901] 0c:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 38: [1022:7901] 0d:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Then my XML for my Windows 10 KVM. I passthrough the Realtek Ethernet Controller to the VM, the 2 USB controllers in the IOMMU groups 26 (06:00.1) and 27 (06:00.3). Also of course the GTX 1080 with its sound chip and in addition the onBoard sound card: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>7d89541c-18c6-c0bc-f056-d63aa5f1f952</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>20971520</memory> <currentMemory unit='KiB'>20971520</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>16</vcpu> <cputune> <vcpupin vcpu='0' cpuset='4'/> <vcpupin vcpu='1' cpuset='16'/> <vcpupin vcpu='2' cpuset='5'/> <vcpupin vcpu='3' cpuset='17'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='18'/> <vcpupin vcpu='6' cpuset='7'/> <vcpupin vcpu='7' cpuset='19'/> <vcpupin vcpu='8' cpuset='8'/> <vcpupin vcpu='9' cpuset='20'/> <vcpupin vcpu='10' cpuset='9'/> <vcpupin vcpu='11' cpuset='21'/> <vcpupin vcpu='12' cpuset='10'/> <vcpupin vcpu='13' cpuset='22'/> <vcpupin vcpu='14' cpuset='11'/> <vcpupin vcpu='15' cpuset='23'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/f59f4ccd-d607-2f2a-049b-51c03e6983fc_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='16' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/ata-Samsung_SSD_850_EVO_1TB_S2RFNX0HA24183T'/> <target dev='hdc' bus='sata'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1TJ06V8'/> <target dev='hdd' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='3'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/domains/gtx1080.dump'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x4'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain> EDIT: During writing this post, unRAID 6.6.7 crashed and I had to "kill" it via power button...

Ok what do you mean with malformend XML config? In my test environment without the parity build etc. is running, it worked like a charm. And there I applied the same steps. And yes, I use UEFI because otherwise unRAID is not bootable. Actually "legacy support" is switched on in the BIOS of my mainboard but I think here the Asus board behaves buggy (?)

Hi! I tested it with version 6.6.7 and there its the same problem. Endless loading and crashing....

Ok. I just can download the version 6.6.7 on the website. So I will try this again these days...

Yes I tried 6.7.2, 6.7 and 6.7.3 rc4 and 6.7.2 with the custom kernel mentioned above. But Ok with this new hardware it may be even more risky to use an old version like 6.5??

Ok, so in normal conditions this way should not be the reason for my issues and is just the incompatibility between unRAID/ Linux Kernel and the X570 mainboards. Thats the only explanation I have for me at this moment. You mean a lack of resources? Okay I dont know... During my tests there were no other Docker apps or any big file transfer running in the background. Ok just the parity build/sync process but this shouldn't be that intensive load. In my configuration unRAID has access to 4 + 4 cores and 16 GB RAM, I think thats more enough.

With crash I mean that the "VM" tab gets unresponsive (endless loading) as soon as I starting the VM with GPU Passthrough switched on. In this case I can switch to "Dashboard" or any other tab but after about 20 seconds everything gets unresponsive and I can perform a hard reset by pressing and holding the power button. And I am NOT passing through the whole SATA controller. I just select in the mask for the primary drive "Manual" and then type: "/dev/disk/by-id/the_desired_SSD". So thats the same way you are using? At first I planned to passthrough the whole SATA controller but I cannot figure out on which port the desired SSD sits. This is needed because the SATA controller on my board have different IOMMU groups but the same PCI-IDs. The trick to "hide" the desired devices as in the case of my USB controller doesn't work because I found no command to see on which ports my drives are plugged in. For USB controllers I got the right command from one of SpaceInvadersOne videos...: for usb_ctrl in $(find /sys/bus/usb/devices/usb* -maxdepth 0 -type l); do pci_path="$(dirname "$(realpath "${usb_ctrl}")")"; echo "Bus $(cat "${usb_ctrl}/busnum") --> $(basename $pci_path) (IOMMU group $(basename $(realpath $pci_path/iommu_group)))"; lsusb -s "$(cat "${usb_ctrl}/busnum"):"; echo; done

Hi! Thanks for your reply. I really want to give it another try but without a „stable GPU Passthrough“ its not the right one for me. And I think for that the Linux Kernel has to be updated first, to run properly with x570 chipset hardware (?)

Final Update: On my first test with the custom kernel, which is available in the other thread, I used the wrong unRAID version so that causes the error... So today I tested again with the stable version 6.7.2 and the custom kernel modules. First run: In a test environment with a smaller SSD. I set up the unbind script via the user.scripts plugin. Installed Windows and after updating, I changed from VNC to the GPU and it worked fine. So time for the next step... Second run: Now using all my drives. Set up the disk, parity and cache devices and deleted the SSD with the "baremetal" Windows installation. Done the exact same steps like in the test environment. After passthrough the GPU, unRAID again completely crashes - WTF ?!!! So finally I am done with unRAID, thats too buggy. And I am lucky that I not already bought the software.

Ah ok. In the end of my "test cycles" I used the 6.7.3 rc4. So then I give it a try with 6.7.2 on this weekend.

Hi! As I mentinoed in my own thread I tried your customized kernel on my Ryzen 3000 build with an Asus Strix x570 E-Gaming but there I just got an error warning "incompatible md version" and the array won't start. Does it mean the signature of the two adjusted file? The custom kernel provides just MD5 signature but on my unRAID boot-stick I have SHA256 signatures for the "bzimage" and "bzmodules"... Otherwise at first I am done with unRAID so far. I just hope that the next updates fixes the issues..