[Anybody planning a Ryzen build?]

On 4/24/2017 at 2:08 PM, Bureaucromancer said:

Yeah, I've tried with and without pinning, ignore msrs is on and iommu=pt.  For the moment I'm going to do some experiments on bare metal and see what I get, there's definitely something up with the cores and I'd like, among other things, rule out a bad cpu.

 

So I've did some playing around with bench marking and my 8 core fx/unraid setup on win10.  With NPT disabled I am seeing what you described as progressively worse multi-threaded performance when adding additional cpus.  It's downright horrible in some tests with all 8 cores.  Single threaded performance, however, seems to be pretty consistent with ~12-15% performance drop across the board.  4 cpus looks like the sweet spot for overall system performance.   It seems to be a mixed bag though.  My cinebench tests did not show this behavior and performance seemed to scale up pretty well as I added additional cores.  I'm also running a Linux Mint 18.1 vm and while I saw some noticeable degradation in win10 (login time, internet browsing, some desktop apps, etc), disabling NPT seemed to improve the overall desktop experience and I was not able reproduce the same weird multi-threaded issues.  

 

So it looks like I'll be dealing with same issues with Ryzen...  for a second I started to wander if I should hold off and wait a while longer to see if some of these things get ironed out before taking the dive. But then I realized that I've been with amd since the k5. What am going to do, go out and buy intel? 

[Anybody planning a Ryzen build?]

13 hours ago, Bureaucromancer said:

And in further weirdness it seems in deleting my pinning I accidently set the vm to a single core and that it works with such.  It does not with the full 12 cores I usually have, experimenting now to see if I can track a hard limit down.

 

Another edit:  what I thought was a hard lock to 100% cpu may not actually be such.  I'm letting it sit witht he pegged cores at the moment on the full cpu set I want, having found that the time to get through the windows splash screen drastically increased every time I increased the core count.  The weird thing is that the progres spinner is moving now and then, so it looks like SOMETHING is happening behind all that CPU activity and its somehow related to number of cores in play.  I almost wonder if it's something odd about handling of the CCX units on Ryzen now?

 

I wouldn't think that pinning/unpinning the cpus would effect whether or not you could boot the vm.  Also, are you isolating any cpus for the host?  If so then you need use pinning, otherwise the vm will try and run all cpus on the host cpu.  For instance if you isolcpus=1-15, then you have specify via pinning which cores between 1-15 you'd like the vm to use, otherwise the system will try to run all vm cores on cpu 0 which is the only core "visable" to the system.  Imagine 12 cores trying to use one core.  This would be a problem and might be causing the slower boot times the more cores you add.  The only reason I have the cpu's unpinned is testing out the performance differences.  Also, are you using iommu=pt and kvm.ignore_msrs=1?  These helped me with win10 boot issues I was having (I think).  

 

 

[Anybody planning a Ryzen build?]

1 hour ago, Bureaucromancer said:

Edit: ok, so quick and dirty fix was to switch to QEMU (from host-passthrough) with NPT off, suddenly the GPU is used properly, but of course my CPU performance has gone to hell.  NPT is obviously going to have some loss, but is there anything much to optimize performance with QEMU or get host-passthrough workign without NPT?

 

Here is the relevant part of my win10 xml.  I'm using host-passthough and I've removed any cpu pinning at the moment.

<os>
    <type arch='x86_64' machine='pc-q35-2.7'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/xxx_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>

 

[Anybody planning a Ryzen build?]

1 hour ago, Bureaucromancer said:

I don't have much experience mucking about with kernel parameters, any particular order my appends should go in syslinux?  My first attempt to use kvm-amd.npt=0 just led to vms eating 100% of the assigned cores and never booting.

 

I ran into this issue as well when tinkering with the params.  Only the vm would usually boot the second try after killing it, and it was only the windows one that had the problem.  I'm not having that issue right now, so maybe adding iommu=pt and/or kvm.ignore_msrs=1 will help with this.  I'm not sure that the order makes a difference, but below is the current configuration that is working for me:

default /syslinux/menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot kvm-amd.npt=0 iommu=pt kvm.ignore_msrs=1

 

1 hour ago, Bureaucromancer said:

Any in depth discussions of this you know of?  I've found one on the redhat forums, that ends up suggesting following the iommu mailing list, but are there any other things I should be following?

 

The redhat vfio-user list is what I'm following,  I did see mention of the iommu mailing list, but I wasn't sure what list that was referring too.  I've not been active in the discussions, just following along...

[Anybody planning a Ryzen build?]

 

On 4/19/2017 at 5:37 PM, Donrenegade said:

These cpu scores seem good with the small amount of performance that is lost by virtualizing. However, this is where it gets strange: the OPENGL score seems to hover around 40 in the vm, and 100 in the regular windows install. I Installed the 3DMark demo that’s on Steam and I ran the Time Spy demo in the regular and vm install of windows.

 

On 4/22/2017 at 8:42 AM, Bureaucromancer said:

I'm actually getting some weird performance issues in games as well.  Some combination of spiking to very low framerates and/or very low resource (both cpu and GPU) utilization with low framerates as well, but yeah, this isn't consistent enough to have traced yet...  I've actually been leaning to there being something about Dirextx 11 that messes with performance on VMs?

 

There is an issue with AMD CPU's gaming/gpu performance and nested page tables.  I'm currently running an fx cpu, but apparently this effects Ryzen processors as well (hope to join you guys with a Ryzen build in the coming weeks!).   NPT can be disabled by appending "kvm-amd.npt=0" to the kernel parameters at boot. My current setup is an fx8320e with a gtx970 and gtx1050 passed through and I was struggling with framerates that were 50-75% worse than bare-metal before disabling NPT.   The trade off is that you will give up the performance benefits of NPT.   I've had a noticeable slowdown in some win10 desktop applications with it disabled.  My hope is that this will be less noticeable on the ryzen platform.