Shit. I guess the only way is two gpus, with one shitty one in the top slot. I have a hard time believing UEFI support is this lacking.

7 minutes ago, Tritech said:

Shit. I guess the only way is two gpus, with one shitty one in the top slot. I have a hard time believing UEFI support is this lacking.

I tried the new BIND method that was documented but Unraid seems to completely ignore it.

Alright status update.. I FINALLY GOT THE DAMN THING TO BOOT.. You must do the vfio-pci.ids= thing but then run the commands below. This tells unraid to release the GPU

 

echo 0 > /sys/class/vtconsole/vtcon0/bind

echo 0 > /sys/class/vtconsole/vtcon1/bind

echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

Awesome! I won't be back home for a few hours to test and see if this even makes an improvement. I guess those few commands could be automated into a script that runs at boot.

 

Well it was an idea. We did figure out UEFI booting. Unfortunately for me, latency is still the same.

Another idea. What version is your installed audio driver? Just an idea, remebering some AMD GPU users had issues passing through there GPUs with the latest drivers. I have a kinda old driver installed which is working from day one until today.

weird, I installed the most recent from asrock and the driver still says from MS.

 

Wait a second. I have 2 Realtek HD Audio Devices, I'am confused right now. Is the upper one part of the 1080ti? I thought the "NVIDIA HD Audio" belongs to the card. 🤔 The other one is the same driver as yours.

 

 

One is probably the video card audio. Mine only emulates as a usb device and doesn't output anything. I've disabled all my nvidia stuff.

1 hour ago, Tritech said:

One is probably the video card audio. Mine only emulates as a usb device and doesn't output anything. I've disabled all my nvidia stuff.

One thing we will have to wait for is the PCI speed fixes that causes latency with the nvidia driver.. No idea how much faster we will get once that happens. In "feature requests" we've asked Limetech to give us a way to either do the patch ourselves or roll the patch into our version of QEMU as its been added to the main branch now.

 

 

Edited February 11, 20197 yr by Jerky_san

I'd be fine with that latency.

 

 

 

Really the only difference in setups is my passed through nvme drive. I haven't tried an image on the drive itself, I just can't imagine that being faster. Probably my project for tomorrow.

Edited February 11, 20197 yr by Tritech

4 hours ago, Tritech said:

I'd be fine with that latency.

 

 

 

Really the only difference in setups is my passed through nvme drive. I haven't tried an image on the drive itself, I just can't imagine that being faster. Probably my project for tomorrow.

I use an NVME as well. Only difference is our boards and processors tbh.. 2990wx and a Zenith.

well after upgrading to 6.7-rc3 I am in the same boat as you guys.

 

already had latency issues before but it was manageable.

 

I tried to go with TSC instead of HPET/Hypervclock. Latency dropped by alot but whole Windows VM became sluggish/unresponsive.

 

As of right now I can't even get back to my 6.6 VM cause with rc3 update my flashdisk got corrupted and old VM xml is lost.

 

Regarding the PCIe speed fix you are talking about, it's the one from level1techs right?

If so, that one is only useful for Q35 machine type.

 

Actually I can't even manage to get my DDR4 Latency back to 80-90ns..always sitting at 130-150ns even so I'm only allocating memory on the numa node where my cpu cores are pinned to.

 

In level1techs forum there was a guy with hell like low Latency

 

Even with his XML not possible to reach those latency on fresh install...sad

@Nooke can you please link the level1techs forums entry? I can't find it.

23 minutes ago, bastl said:

@Nooke can you please link the level1techs forums entry? I can't find it.

sure,

https://forum.level1techs.com/t/increasing-vfio-vga-performance/133443/84

 

there you go

9 hours ago, Nooke said:

sure,

https://forum.level1techs.com/t/increasing-vfio-vga-performance/133443/84

 

there you go

Here is what I don't get.. Everyone talks about 3.0 bringing the fixes like for cache and stuff without the need to use the EPYC CPU fix but QEMU3.0/3.1 in Unraid Doesn't have these fixes. I don't understand why. Its obvious we are running 3.1 on the RC it clearly states it as such but the fixes aren't rolled in even though QEMU 3.1 main branch has them. Makes me wonder whats going on there.

These PCIE fixes gnif talked about aren't in yet. He suggested they will be implemented in 4.x as default and first shown in 3.2 and we are currently on 3.1 in the RC build.

1 Dec '18

Hi, is there any chance these patches reach qemu 3.1 release or other systems involved ?

gnif:
I believe they are trying to get them queued up for 3.1, and will default to full speed in Qemu 4.0. These patches will only apply to platforms that actually have PCIe such as Q35, i440fx is out of the question.

 

lessaj:
Went to do a new build and the patch set failed to apply, seems as of Dec 19 this patch set was committed to qemu master branch. Awesome!

 

gnif:
4.0 is when it will default to using the higher link speeds, last I read however the 3.2 and later builds have these patches but you must specify the link speed. I have not checked as I have been on break however and could have the versioning wrong :slight_smile:

 

Is this true?

Quote

Ensure you have the latest AGESA/BIOS, AMD changed the CPU core indexes in the later versions, instead of being 1,2,3,4 and 9,10,11,12 for the HT cores it is now sequential as 1,2,3,4,5,6,7,8

 

my lstopo says the first is correct. The pinned thread here also mentions sequential pairings in the second post. 

I've tried this with terrible results.

@Tritech With one of the earlier Agesa updates i thing end 2017 or early 2018 Amd changed something thats right. On the first BIOS version on my board (dec 2017) it reported the core pairings differently as on an update a couple months later. Depending on your BIOS version, i guess you will have already the changed one, lstopo showed it always correct and so does unraid. In earlier days people got confused cause everyone had different pairings.

Edited February 12, 20197 yr by bastl

Got it. My board already had a 2.x bios when I got it, and I updated to 3.5 before I really got into most of this. It's kinda confusing with old and new information being mixed in, especially on a pinned thread.

 

Just about to try a new vm with q35 bios. BTW I noticed something with my i440 vm. Remember how you said I forgot a portion of the epic hack. Well turns out I checked my new vm, and it seems like that portion just gets deleted in a i440 .xml.

 

Edit: With q35 the .xml changes, specifically these lines

<cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC-IBPB</model>

stayed in the .xml. In both instances, windows still recognized it as a Epic.

Edited February 12, 20197 yr by Tritech

31 minutes ago, Tritech said:

Got it. My board already had a 2.x bios when I got it, and I updated to 3.5 before I really got into most of this. It's kinda confusing with old and new information being mixed in, especially on a pinned thread.

 

Just about to try a new vm with q35 bios. BTW I noticed something with my i440 vm. Remember how you said I forgot a portion of the epic hack. Well turns out I checked my new vm, and it seems like that portion just gets deleted in a i440 .xml.

what i do is have three machines.. a i440, and two q35s. I'm tweaking the q35's and would switch to it. Then tweak the other config some more and move to it. Use my phone to start the other.

Edited February 12, 20197 yr by Jerky_san

@bastlIf you get a chance, can you show or just tell me where things are plugged into your rear USB.

 

 

18 minutes ago, Jerky_san said:

what i do is have three machines.. a i440, and two q35s. I'm tweaking the q35's and would switch to it. Then tweak the other config some more and move to it. Use my phone to start the other.

I've noticed that as well. But in this case I was only editing a single VM and it was deleted during boot. Probably related to changes that only would apply to q35 and trying to apply it to a i440 vm.

Edited February 12, 20197 yr by Tritech

I MAY be onto something. I can't seem to get the gpu passed through, so I'm not sure yet. I tried some of the things that guy had in his xml at level1techs. This is by far the lowest latency I've seen, but gpu isnt really in the equation. It's throwing an error 43.

 

 

Edited February 13, 20197 yr by Tritech

1 hour ago, Tritech said:

I MAY be onto something. I can't seem to get the gpu passed through, so I'm not sure yet. I tried some of the things that guy had in his xml at level1techs. This is by far the lowest latency I've seen, but gpu isnt really in the equation. It's throwing an error 43.

 

 

Its cause nvidia's driver is causing a lot of latency.. Once we get the PCI-E speed fix that should go away. Wish I could figure out how to compile QEMU 4 for myself on unraid. I tinkered with it a little. Or merge the patches with my current build..

Edited February 13, 20197 yr by Jerky_san