Thank you @ofawx I had the same issue in the past, the vm crashed sometime, when compiling in xcode with all cores near to 100%; after isolating the cpus and optimizing the xml the issue did not come back; anyway if it will come back I will know what to try.

 

I'm currently experiencing some issues with mac os updates, which I cannot track if it0s due to qemu or bootloader or configuration: for example, when performing an ota delta update, once finished the download, the os asks to restart, but it isn't fully rebooting after the first stage; the vm is running, this is what virsh is reporting, but the os is unable to start again, so I need to force the shutdown and boot again manually. After this, the next stages are completed automatically.

Anyone experiencing similar behavior?

 

Moreover, I'm testing qemu on another machine (not unraid) and it seems that machine type pc-q35-6.1 is unable to boot, we should remind this when qemu 6.1 will be part of unraid.

It seems qemu 6.1 introduced hotpluggable capabilities for pcie with some modifications to DSDT, but even if injecting the old DSDT (qemu 6.0) doesn't fix the issue.

Machine type 6.0 works as expected.

Edited September 12, 20214 yr by ghost82

20 hours ago, ghost82 said:

Thank you @ofawx I had the same issue in the past, the vm crashed sometime, when compiling in xcode with all cores near to 100%; after isolating the cpus and optimizing the xml the issue did not come back; anyway if it will come back I will know what to try.

 

I'm currently experiencing some issues with mac os updates, which I cannot track if it0s due to qemu or bootloader or configuration: for example, when performing an ota delta update, once finished the download, the os asks to restart, but it isn't fully rebooting after the first stage; the vm is running, this is what virsh is reporting, but the os is unable to start again, so I need to force the shutdown and boot again manually. After this, the next stages are completed automatically.

Anyone experiencing similar behavior?

 

Moreover, I'm testing qemu on another machine (not unraid) and it seems that machine type pc-q35-6.1 is unable to boot, we should remind this when qemu 6.1 will be part of unraid.

It seems qemu 6.1 introduced hotpluggable capabilities for pcie with some modifications to DSDT, but even if injecting the old DSDT (qemu 6.0) doesn't fix the issue.

Machine type 6.0 works as expected.

I have some issues similar to what you describe but unfortunately I couldn't get to the root of the issue. Tried a lot of combinations and messed up the entire VM.  I ended up doing a clean install with fresh XML definition. I can share my config.plist for BigSur/Monterey if it helps you compare stuff.

44 minutes ago, ab5g said:

I can share my config.plist for BigSur/Monterey if it helps you compare stuff

Thank you, I think I don't need it, just wanted to know if it's specific to my setup or not, maybe qemu related...

43 minutes ago, ghost82 said:

Thank you, I think I don't need it, just wanted to know if it's specific to my setup or not, maybe qemu related...

Ok - I'll observe and report the behavior when Beta7 comes out and I do the install.  BTW do you mind sharing your conf.plist ?

21 minutes ago, ab5g said:

BTW do you mind sharing your conf.plist ?

Here it is, for latest opencore, some parts are REDACTED.

Some parts are customized with addresses referencing to my hardware addresses.

config.plist

18 hours ago, ghost82 said:

Here it is, for latest opencore, some parts are REDACTED.

Some parts are customized with addresses referencing to my hardware addresses.

config.plist 26.49 kB · 3 downloads

I'm curious as to what benchmarks do you get on the VM.

I have an Intel i9-10850K (10 core 20 threads) with no overclock on the cores. I have allocated 8 threads to the VM and 16GB RAM iMacPro1,1.  The cores are isloated for use by the VM. and I am getting 1311 as the Single-Core Score and 7752 for Multi-Core Score in Geekbench5.

I got slight performance bump by 

1. cat /sys/devices/system/clocksource/clocksource0/current_clocksource - changing the clocksource to tsc in bootmenu of the flash drive
2. In the VM XML , allocating CPU threads consecutively (2,3,4,5,6,7,8) as opposed to allocating based on hyper-threaded pairs (2,12,3,13,4,14,5,15)

 

 

 

Edited September 14, 20214 yr by ab5g

55 minutes ago, ab5g said:

I have an Intel i9-10850K (10 core 20 threads) with no overclock on the cores. I have allocated 8 threads to the VM and 16GB RAM iMacPro1,1.  The cores are isloated for use by the VM. and I am getting 1311 as the Single-Core Score and 7752 for Multi-Core Score in Geekbench5.

I think it's what you can expect.

 

My result with Geekbench 5.0.4, 14 cores+ht (28 threads), big sur 11.6 (just updated)

Single core: 603

Multicore: 7403

 

My cpu is older (yours: Q3 2020, mine Q1 2012), higher base frequency for yours (3.6 GHz vs 3.1 GHz), higher boost (5.2 GHz vs. 3.8 GHz).

 

Though my cpu was launched in 2012 is still good enough for me.

 

Depending on the os version single core performance ranges/ranged roughly from 600 to 700, multicore doesn't/didn't change too much.

Edited September 14, 20214 yr by ghost82

1 hour ago, ghost82 said:

I think it's what you can expect.

 

My result with Geekbench 5.0.4, 14 cores+ht (28 threads), big sur 11.6 (just updated)

Single core: 603

Multicore: 7403

 

My cpu is older (yours: Q3 2020, mine Q1 2012), higher base frequency for yours (3.6 GHz vs 3.1 GHz), higher boost (5.2 GHz vs. 3.8 GHz).

 

Though my cpu was launched in 2012 is still good enough for me.

 

Depending on the os version single core performance ranges/ranged roughly from 600 to 700, multicore doesn't/didn't change too much.

 

Hmm, I have a baremetal install of Big Sur on an i9-9900k which gives me a single core perf of 5800+ and multicore near 39000 - I'm wondering if the low scores are normal or is it the KVM penalty or is it because the Geekbench lowers the score as it realizes its on a VM  ?

 

Edited September 14, 20214 yr by ab5g

2 minutes ago, ab5g said:

 

Hmm, I have a baremetal install of Big Sur on an i9-9900k which gives me a single core perf of 5800+ and multicore near 39000 - I'm wondering if the low scores are normal or is it the KVM penalty ?

 

 

Are you sure about these values?

Same version of geekbench? values change if version is different, I think I remember newer versions output higher values.

 

Bare metal:

https://browser.geekbench.com/processors/intel-core-i9-9900k

Edited September 14, 20214 yr by ghost82

23 minutes ago, ghost82 said:

 

Are you sure about these values?

Same version of geekbench? values change if version is different, I think I remember newer versions output higher values.

 

Bare metal:

https://browser.geekbench.com/processors/intel-core-i9-9900k

Gosh ! that was it. I was comparing the wrong versions. thank you

• Geekbench 5 on Baremetal with i9-9900k is 1203 and 8068 on VM with i10-10850(8 threads) is 1311 and 7752  
• Geekbench 4 on Baremetal with i9-9900k is 6310 and 35213 on VM with i10-10850(8 threads) is 6117  and 34138  

1 hour ago, ab5g said:

that was it

Yes, it was the opposite, newer versions, lower values: since newer cpus are more powerful, the new "1000" value reference standard is referenced to a newer cpu.

On 9/12/2021 at 10:30 PM, ghost82 said:

Thank you @ofawx I had the same issue in the past, the vm crashed sometime, when compiling in xcode with all cores near to 100%; after isolating the cpus and optimizing the xml the issue did not come back; anyway if it will come back I will know what to try.

 

I'm currently experiencing some issues with mac os updates, which I cannot track if it0s due to qemu or bootloader or configuration: for example, when performing an ota delta update, once finished the download, the os asks to restart, but it isn't fully rebooting after the first stage; the vm is running, this is what virsh is reporting, but the os is unable to start again, so I need to force the shutdown and boot again manually. After this, the next stages are completed automatically.

Anyone experiencing similar behavior?

I used to have this issue too which was also solved by the VTI trick. Maybe worth trying anyway?

3 hours ago, ofawx said:

I used to have this issue too which was also solved by the VTI trick. Maybe worth trying anyway?

I just set it!I will see in the next installation/update.

Thank you!

 

Update: unfortunately, it doesn't fix the issue

 

Edited September 15, 20214 yr by ghost82

On 9/11/2021 at 9:03 AM, bjornatic said:

Since two days everything works now flawlessly. Before that, I was about seriously considering buying a mac mini...

 

Well... I spoke too soon. The KP as I wake macOS are back. Every time now. And this is the report I get :

mp_kdp_enter() timed-out on cpu 0, NMI-ing
mp_kdp_enter() NMI pending on cpus: 0 1 3 6 7
mp_kdp_enter() timed-out during locked wait after NMI;expected 8 acks but received 4 after 0 loops in 8192004096000 ticks

 

With few variations on CPUs number and count...

 

I tried with isolated CPU's and without, with vti=12, 9 or 14 : same problem.

 

And I really don't get why it worked two days in a row without a single issue (and during that time, the process : macOS sleep / unraid sleep / unraid wake / macOS wake happened at least 8 times (as I was testing it and not believing it was working...)

 

😭

 

10 minutes ago, bjornatic said:

macOS sleep / unraid sleep / unraid wake / macOS wake

Not sure about this, but in our mac os vms we cannot control power states of cpu, even if it's passed through..I was never able to get p and c states in mac os as in bare metal, so playing with states in both the guest and the os should not be recommended..

 

btw, are you mapping your usb ports?passedthrough usb controller(s)?

Edited September 15, 20214 yr by ghost82

14 hours ago, ghost82 said:

I just set it!I will see in the next installation/update.

Thank you!

 

Update: unfortunately, it doesn't fix the issue

 

I upgraded to 11.6 without the shutdown error you encountered. I made an observation that when the clocksource was tsc, the machine had random sleep/wake issues. Setting the clocksource to hpet for Unraid got rid of those issues for me (although I loose some performance in Geekbench). You can try and see if that gets rid of the shutdown error for you.

I'll observe the behavior when Beta6 comes out.  

 

6 hours ago, bjornatic said:

 

Well... I spoke too soon. The KP as I wake macOS are back. Every time now. And this is the report I get :

mp_kdp_enter() timed-out on cpu 0, NMI-ing
mp_kdp_enter() NMI pending on cpus: 0 1 3 6 7
mp_kdp_enter() timed-out during locked wait after NMI;expected 8 acks but received 4 after 0 loops in 8192004096000 ticks

 

With few variations on CPUs number and count...

 

I tried with isolated CPU's and without, with vti=12, 9 or 14 : same problem.

 

And I really don't get why it worked two days in a row without a single issue (and during that time, the process : macOS sleep / unraid sleep / unraid wake / macOS wake happened at least 8 times (as I was testing it and not believing it was working...)

 

😭

 

Going by my own experience on this - I would recommend the following 

1. Check to make sure your USB patches are correct. Did you copy the kext from somewhere and doesn't know how it works ? Here's an excellent resource to map your USB properly.- https://github.com/corpnewt/USBMap
2. Check to make sure that whatever you inject as Kernel Properties in OC have the correct device path. You can use Hackintool then PCI to see the true device path of PCI devices and match that with what you are injecting. I realized that if you mess with the XML or change the machine type the device paths can change which in turn mess up with sleep wake.
3. Check for errors in your config.plist. Compare with the sample on this page.
4. Try adding tlbto_us=0 in the OC boot argument along with vti=12. Details here - https://www.nicksherlock.com/2020/08/solving-macos-vm-kernel-panics-on-heavily-loaded-proxmox-qemu-kvm-servers/
5. Try CpuTscSync kext - it helps avoid crashes from certain wake-from-sleep scenarios. 
6. Check the clocksource in Unraid here - cat /sys/devices/system/clocksource/clocksource0/current_clocksource. I got some stability by changing that to 'hpet' instead of 'tsc'. You can change that by adding a boot argument in the flash drive 'clocksource=hpet'.
7. Isolate the cores that the VM runs on by adding a boot argument in the flash drive. Isolate and allocate cores consecutively (2,3,4,5,6,7,8) as opposed to isolating and allocating based on hyper-threaded pairs (2,12,3,13,4,14,5,15)

11 hours ago, ghost82 said:

btw, are you mapping your usb ports?passedthrough usb controller(s)?

I'm passing though : GPU, nVme drive controller and usb controller (Fresco Logic FL1100 USB 3.0 Host Controller). And I'm only using the ports on the Fresco card... do I still need mapping  and USB patches ?

 

 

5 hours ago, ab5g said:

Going by my own experience on this - I would recommend the following 

...

 

Thanks ! I'll try what you suggest and I'll report back.

8 minutes ago, bjornatic said:

And I'm only using the ports on the Fresco card... do I still need mapping  and USB patches ?

I would do so, mapping + power properties.

If you have virtual controllers ehci/uhci I would map also that ports.

On 9/12/2021 at 2:30 PM, ghost82 said:

I'm currently experiencing some issues with mac os updates, which I cannot track if it0s due to qemu or bootloader or configuration: for example, when performing an ota delta update, once finished the download, the os asks to restart, but it isn't fully rebooting after the first stage; the vm is running, this is what virsh is reporting, but the os is unable to start again, so I need to force the shutdown and boot again manually. After this, the next stages are completed automatically.

 

It seems a bug in libvirt or qemu...libvirt 6.5.0+qemu 5.1.0 is ok (unraid 6.9.2).

Expect similar behavior with next versions if the issue isn't tracked upstream.

Edited September 18, 20214 yr by ghost82

On 9/18/2021 at 6:12 PM, ghost82 said:

It seems a bug in libvirt or qemu...libvirt 6.5.0+qemu 5.1.0 is ok (unraid 6.9.2).

Expect similar behavior with next versions if the issue isn't tracked upstream.

Oh damn....no, it's related to OVMF! There's something wrong in newer versions (I would say release versions released since one year ago...)...going to their bugtracker to see if they can fix this.

It's the second time in one year that they break something

 

OVMF 202011: no issue

OVMF 202102: issue

...

OVMF 202108: issue

 

Edited September 20, 20214 yr by ghost82

Hi All,

 

Can't get external USB 3.0 drive to appear/be mounted in unRaid Monterey!

 

Everything else mostly works, except all UBS 3.0 inside the Monterey VM in latest unRaid 6.10.0-rc1.

 

I wonder if someone can have a look at my files ? Also is there any optimization needed ?

 

This machine is used to simultaneously run Microsoft Win 10 64bits + Apple Mac OS Monterey, each with half CPU cores of i9-9900K and half of 32GB RAM, each with own USB + SATA + Video controllers passthrough.

 

(All serial numbers are generated).

 

Thanks.

 

unRaid Monterey.zip Real.zip

Edited September 21, 20214 yr by augustopaulo

5 hours ago, augustopaulo said:

Hi All,

 

Can't get external USB 3.0 drive to appear/be mounted in unRaid Monterey!

 

Everything else mostly works, except all UBS 3.0 inside the Monterey VM in latest unRaid 6.10.0-rc1.

 

I wonder if someone can have a look at my files ? Also is there any optimization needed ?

 

This machine is used to simultaneously run Microsoft Win 10 64bits + Apple Mac OS Monterey, each with half CPU cores of i9-9900K and half of 32GB RAM, each with own USB + SATA + Video controllers passthrough.

 

(All serial numbers are generated).

 

Thanks.

 

unRaid Monterey.zip 15.85 MB · 2 downloads Real.zip 10.18 MB · 0 downloads

I think your config is way off - unnecessary ACPI patches, kernel patches. Reusing the Designare EFI will do more harm than help you. A better approach is to create an EFI from scratch. There's one config.plist attached on this page or the one before which you can use to create your EFI. Once done you can then tackle the USB problem (which is likely due to wrong USB port mapping - https://github.com/corpnewt/USBMap).

You could also start with an EFI from macinabox and customize that to the newer version. So use macinabox to define a VM, mount the defined volume of this VM  as second disk in Win VM. Extract the EFI customize and then work on it. 

 

17 hours ago, ab5g said:

I think your config is way off - unnecessary ACPI patches, kernel patches. Reusing the Designare EFI will do more harm than help you. A better approach is to create an EFI from scratch. There's one config.plist attached on this page or the one before which you can use to create your EFI. Once done you can then tackle the USB problem (which is likely due to wrong USB port mapping - https://github.com/corpnewt/USBMap).

You could also start with an EFI from macinabox and customize that to the newer version. So use macinabox to define a VM, mount the defined volume of this VM  as second disk in Win VM. Extract the EFI customize and then work on it. 

 

 Hi ab5g,

For your mentioned confi.plist, your mean this one: 

 

The reason why I chose a Gigabyte Z390 Designare EFI is because my motherboard is the Gigabyte Z390 Xtreme, which is very, very similar to the Designare. Only extra is the 10G Aquantia ethernet port, RGD lights and massive overclock ability, I think...

What about my unraid VM .xml file ? Any optimizations required ?

 

16 minutes ago, augustopaulo said:

 Hi ab5g,

For your mentioned confi.plist, your mean this one: 

 

The reason why I chose a Gigabyte Z390 Designare EFI is because my motherboard is the Gigabyte Z390 Xtreme, which is very, very similar to the Designare. Only extra is the 10G Aquantia ethernet port, RGD lights and massive overclock ability, I think...

What about my unraid VM .xml file ? Any optimizations required ?

 

 

The config.plist is on page 36 posted by ghost82 (13th Sept) - you can use that as a reference. The XML should be ok as you are able to boot the VM and passthrough the devices. You are running the VM on Unraid so the Designare config.plist will not apply. The config.plist needs to be KVM specific.

 

Edited September 22, 20214 yr by ab5g