Anybody planning a Ryzen build?


Recommended Posts

Just thought I would chime in here.  Over the weekend I put together a new Unraid PC with a single (for now) Windows 10 Pro Gaming VM.  I have a Gigabyte GA-AX370-Gaming K7 Motherboard with the Ryzen 1800x CPU.  I updated to the newest BIOS before installing anything (I believe it was K3?)  I paired it with a MSI GeForce GTX 1080 in the primary PCI slot.  Passing it through to my VM with no problems.  I have just a couple of dockers running at the moment.

 

I have ACS Override enabled, but it isn't breaking my groups up quite as much as I'd like.  I currently have no sound because my motherboard sound card it grouped in the same IOMMU group as some other items and I can't get it to pass through.  GPU Audio passthrough seems to be working fine, but I don't have a monitor with speakers to try out out on.  It also has most of my USBs controllers in the same IOMMU group, so I can't pass any of those through without passing my unraid USB also.  I'm able to passthrough individual USB devices without any problems.  I've ordered a USB sound card that I should be able to passthrough and get some audio out.

 

So far it seems to be stable as long as I don't try to run any of the Dynamix system stats or temperature plugins.  Ran all weekend from Friday to Monday morning with no crashes.

Link to comment
1 hour ago, jonathanm said:

My crystal ball says "Future unclear but optimistic".

 

Nobody knows until it actually happens.

 

Thanks, ill keep watching.  I am holding off my build pending ryzen and unraid,

 

Unraid takes priority, hardware comes second :D 

Edited by Greygoose
Link to comment

So, finally got around to completing my Ryzen Unraid box. Here is my spec:

 

Ryzen 1800x Stock 3.6ghz

Asus Prime x370 Pro BIOS 0604

G.Skill 2x16GB 3200 F4-3200C14D-32 gtz @ 2400

Samsung 950 Pro 512GB

 

 

Tried to overclock the memory to 2933 with timings 18-16-16-16-36 - System was stable for a few hours before my 1st BSOD, will wait for BIOS updates before stress testing some more. UNRAID 6.3.3 is running perfectly stable for more than a day now with Global C State disabled. Got 2 VMs running at the mo and still UNRAID is rock solid stable.

Will be purchasing my 1080ti for passthrough soon and will feedback my success rate with this. So far, coming from an x99 xeon build, I am very impressed with the overall speed both from UNRAID and my W10 Pro VMs. Happy days so far, now waiting for temperature stats and nvme temps to be added to new UNRAID builds/plugins.

Link to comment
28 minutes ago, Josecitox said:

So a build of 2 gaming VM's in 1 PC is not viable yet with Ryzen ? I mean, is it possible to passthrough everything to build something like that??

 

Though I haven't done it myself, yes it is possible.  1 or 2 users (maybe more) have have already accomplished this.

 

Picking the right motherboard helps, and using AMD graphics seems to make the process a bit easier and perhaps cheaper, though you can go nVidia too, just with a bit more work to do the BIOS passthrough.  If you use HDMI sound passthrough that seems to alleviate much of the issues of bad IOMMU groups on sound devices and crackling on USB sound, but it seems there are solutions for both of those issues as well, all described in this thread.

 

Just be prepared to spend a few hours fiddling with BIOS settings and unRAID settings to get it to work.

 

2 hours ago, ColeBrodine said:

I have ACS Override enabled, but it isn't breaking my groups up quite as much as I'd like.  I currently have no sound because my motherboard sound card it grouped in the same IOMMU group as some other items and I can't get it to pass through.  GPU Audio passthrough seems to be working fine, but I don't have a monitor with speakers to try out out on.  It also has most of my USBs controllers in the same IOMMU group, so I can't pass any of those through without passing my unraid USB also.  I'm able to passthrough individual USB devices without any problems.  I've ordered a USB sound card that I should be able to passthrough and get some audio out.

 

 

In this post here (https://forums.lime-technology.com/topic/55150-anybody-planning-a-ryzen-build/?do=findComment&comment=553895) Donach made the best first post ever, and detailed how he edited his syslinux.cfg file to include the downstream,multifunction parameters on his PCIe ACS Override.  Assuming it works on other motherboards, I think this might be a huge help for those struggling with bad IOMMU groups even with ACS Override enabled.  Donach's IOMMU groupings looked significantly better than any others I've seen on Ryzen.  

 

I haven't tried this myself, still testing up-time with Global C-state Control disabled.  Today I reached 14 days of uptime.

 

-Paul

 

Link to comment

I seem to be having issues in using Unraid for virtualizing gaming pc’s. First up, here are the relevant specs:

 

AMD Ryzen 1700X @ Stock

Noctua DH-14 Cooler

Gigabyte AX-370 GAMING 5
Corsair 32GB RAM “CMK32GX4M2B3000C15” - @ 2133MHZ, for compatibility

Creative Sound Blaster Z – top PCI-E 1x slot.

AMD Radeon HD5870 – top PCI-E 16x slot. (running in 8x mode)

Nvidia Geforce GTX670 – second PCI-E 16x slot. (running in 8x mode)

Corsair RM850x

960 EVO 500GB Drive.

Other misc. HDD’s

 

640GB HDD and 750GB HDD for Unraid for different vm’s. No cache or parity drive.

 

I have set the PC up in such a way that I have a “regular” windows install which has access to all the hardware. If I change the boot order with Unraid at the top, Unraid will automatically boot. To get the most consistent results I’ve disabled 4 cores AND SMT for testing purposes on my regular windows install. When using a virtualized windows install, i’ve setup the pc that it boots with 8 cores, SMT disabled. The virtualized machine only has access to 4 cores. Unraid should still have 4 cores left work with, which should be plenty for what I’m doing with it right now. For clarity, I’m only testing 1 virtualized gaming pc for now.

 

These are (what I think are) the relevant specs of the vm:

 

Ryzen, 4 cores, No SMT.

GTX670

8GB RAM.

500GB HDD space. No Cache.

(High performance energy setting)

 

Everything seems to work correctly. However, I seem to be getting wierd results and sub-par gaming performance. To put this into numbers, I’ve ran Cinebench to get the following results:

 

VM Cinebench Multi Score: 535

VM Cinebench Single score: 138

VM Cinebench OpenGL: 43

Regular Cinebench Multi Score: 559

Regular Cinebench Single Score: 148

Regular Cinebench OpenGL: 100

 

 

These cpu scores seem good with the small amount of performance that is lost by virtualizing. However, this is where it gets strange: the OPENGL score seems to hover around 40 in the vm, and 100 in the regular windows install. I Installed the 3DMark demo that’s on Steam and I ran the Time Spy demo in the regular and vm install of windows.

 

VM GPU score: 1811

VM CPU score: 2850

Regular GPU score: 1783

Regular CPU score: 3733

 

For some reason, when the GPU and CPU are stressed at the same time, the CPU seems to be underperforming greatly in the VM. Has anyone else had this problem before and if so, how can I fix this issue? Or is this just a Ryzen early adopter problem that will soon get fixed?

Link to comment

Finally have my unRaid system up and going. I have a MSI X370 GAMING PRO CARBON motherboard with an AMD 1700x. Currently only have an 8tb HDD with a 1 TB cache drive. I'm willing to dump any information that is needed to put together a data base of useful info.

I finally have a windows 10 VM up and running. Nice and smooth for the most part, but video and audio seem to be the issue. I'm passing through a gtx 1080 for audio and video. Going to play around with combinations of whether its the audio passthrough for hdmi causing issue, as well as try through the motherboard. If anyone has any information to pass my way to pinpoint it, i'd greatly appreciate it. I'll be poking around in the mean time.

 

 

Link to comment

I did discover the other day that the MSI registry edit was necessary on my system.  Seems to have improved GPU latency noticeably, though I haven't been using HDMI audio.  That said, installing the actual drivers for my card does seem to have resolved the odd crackle I was still getting after connecting to a passed through USB controller.

Link to comment
On 4/17/2017 at 3:02 PM, ColeBrodine said:

Just thought I would chime in here.  Over the weekend I put together a new Unraid PC with a single (for now) Windows 10 Pro Gaming VM.  I have a Gigabyte GA-AX370-Gaming K7 Motherboard with the Ryzen 1800x CPU.  I updated to the newest BIOS before installing anything (I believe it was K3?)  I paired it with a MSI GeForce GTX 1080 in the primary PCI slot.  Passing it through to my VM with no problems.  I have just a couple of dockers running at the moment.

 

I have ACS Override enabled, but it isn't breaking my groups up quite as much as I'd like.  I currently have no sound because my motherboard sound card it grouped in the same IOMMU group as some other items and I can't get it to pass through.  GPU Audio passthrough seems to be working fine, but I don't have a monitor with speakers to try out out on.  It also has most of my USBs controllers in the same IOMMU group, so I can't pass any of those through without passing my unraid USB also.  I'm able to passthrough individual USB devices without any problems.  I've ordered a USB sound card that I should be able to passthrough and get some audio out.

 

So far it seems to be stable as long as I don't try to run any of the Dynamix system stats or temperature plugins.  Ran all weekend from Friday to Monday morning with no crashes.

Hello ColeBrodine,

as Pauven said, look at my post on page 13 of this thread - my IOMMU grouping was totally off with the ACS Override enabled also, until I added the "multifunction" option. Since then, works like a charm - I'm able to passthrough anything to the VM, even since the CH6 has 3 PCI-Ex16 ports, I can run 3 VMs, each having it's own GPU.

 

 

However, I am now struggling with some specific problems - I want to passthrough USB busses, so I can plug and unplug USB devices on the go.

 

Another thing is that I have encountered issues running Europa Universalis IV - extreme game stuttering, solely on VMs. I did not see this with any other games yet.

Link to comment

I'm actually getting some weird performance issues in games as well.  Some combination of spiking to very low framerates and/or very low resource (both cpu and GPU) utilization with low framerates as well, but yeah, this isn't consistent enough to have traced yet...  I've actually been leaning to there being something about Dirextx 11 that messes with performance on VMs?

Link to comment
On ‎2017‎年‎4‎月‎12‎日 at 11:05 PM, chadjj said:

Hey Everyone,

 

A quick status update on my setup.  Since I have updated to PRIME X370-PRO BIOS 0604 with Global C-State disabled I have seen positive results.  Current up-time is 17hrs, 10mins which is the longest stretch I have seen.

 

If all continues to go well it seems as though AMD AGESA microcode update and Disabling Global C-State have resolved my unRAID hanging issues.  Once I have had some considerable up-time I will try to re-enable Global C-State and evaluate that result.

 

Hope everyone else is seeing similar success.

 

-Chad

 

Thanks for the update. I am planning upgrade to Ryzen and target same mainboard.

 

Does anyone know Ryzen i.e. 1700 will greatly improve mulitiple file hash throughput, I can't found the answer, thanks

 

Edited by Benson
Link to comment

 

On 4/19/2017 at 5:37 PM, Donrenegade said:

These cpu scores seem good with the small amount of performance that is lost by virtualizing. However, this is where it gets strange: the OPENGL score seems to hover around 40 in the vm, and 100 in the regular windows install. I Installed the 3DMark demo that’s on Steam and I ran the Time Spy demo in the regular and vm install of windows.

 

On 4/22/2017 at 8:42 AM, Bureaucromancer said:

I'm actually getting some weird performance issues in games as well.  Some combination of spiking to very low framerates and/or very low resource (both cpu and GPU) utilization with low framerates as well, but yeah, this isn't consistent enough to have traced yet...  I've actually been leaning to there being something about Dirextx 11 that messes with performance on VMs?

 

There is an issue with AMD CPU's gaming/gpu performance and nested page tables.  I'm currently running an fx cpu, but apparently this effects Ryzen processors as well (hope to join you guys with a Ryzen build in the coming weeks!).   NPT can be disabled by appending "kvm-amd.npt=0" to the kernel parameters at boot. My current setup is an fx8320e with a gtx970 and gtx1050 passed through and I was struggling with framerates that were 50-75% worse than bare-metal before disabling NPT.   The trade off is that you will give up the performance benefits of NPT.   I've had a noticeable slowdown in some win10 desktop applications with it disabled.  My hope is that this will be less noticeable on the ryzen platform. 

Edited by Some Dude
Link to comment
4 hours ago, Benson said:

Does anyone know Ryzen i.e. 1700 will greatly improve mulitiple file hash throughput, I can't found the answer, thanks

 

I think that it will.  I have the ASUS Prime X370-PRO motherboard paired with a Ryzen 1800X CPU.  I have my own multi-threaded image processing program that just screams on a Windows 10 VM, hosted by unRAID but only using half the cores/threads (8) that are available.  If it's CPU bound, chances are it will perform quite well.

 

Pretty much everything on the Windows 10 VM (with GPU passthrough) runs very smoothly, though one thing I did notice is some minor delay when I try to play a video file.  I'm using VLC and the first few seconds will stutter a bit, then the system sorts itself out and the rest of the video runs fine.  I'm not sure if this is an issue with VLC, or some conflict with WIndows 10 itself (kinda new to this OS) ... or something specific to VMs or Ryzen.  It honestly is a minor issue, so I'm not overly concerned and I haven't done much testing with other players.

 

As of now, I have 11+ days of uptime, and am pretty happy with the current setup.  When I have more time I'll dig deeper into the ACS override stuff and see if I can re-arrange my IOMMU groups.  If I can dispense with the USB PCIe card, that would be nice ... maybe save a few watts on power. Experimenting with the Global C-States is still down the road a bit.

 

- Bill

Edited by ufopinball
Link to comment

@ufopinball Thanks your reply, due to you own ASUS Prime X370-PRO, so I already add you to follow member sometimes before.

I have consider a upgrade plan for a long time, I use a rather low end system w CPU G3250, just 2 core.

A low cost and easy way was just upgrade to a 4 core CPU and cost ~$167, but this won't be long term, so I consider Ryzen 1700, but need ~$642 and lot of work / risk. :D

 

Hope I will make it success soon, btw I also target a coming soon GPU gtx1030. I am not a gamer/VM fans, so I still not sure upgrade a storage system to 8C16T worth or not.

Edited by Benson
Link to comment
10 hours ago, Some Dude said:

There is an issue with AMD CPU's gaming/gpu performance and nested page tables.  I'm currently running an fx cpu, but apparently this effects Ryzen processors as well (hope to join you guys with a Ryzen build in the coming weeks!).   NPT can be disabled by appending "kvm-amd.npt=0" to the kernel parameters at boot. My current setup is an fx8320e with a gtx970 and gtx1050 passed through and I was struggling with framerates that were 50-75% worse than bare-metal before disabling NPT.   The trade off is that you will give up the performance benefits of NPT.   I've had a noticeable slowdown in some win10 desktop applications with it disabled.  My hope is that this will be less noticeable on the ryzen platform. 

 

I don't have much experience mucking about with kernel parameters, any particular order my appends should go in syslinux?  My first attempt to use kvm-amd.npt=0 just led to vms eating 100% of the assigned cores and never booting.

 

Any in depth discussions of this you know of?  I've found one on the redhat forums, that ends up suggesting following the iommu mailing list, but are there any other things I should be following?

 

Edit: ok, so quick and dirty fix was to switch to QEMU (from host-passthrough) with NPT off, suddenly the GPU is used properly, but of course my CPU performance has gone to hell.  NPT is obviously going to have some loss, but is there anything much to optimize performance with QEMU or get host-passthrough workign without NPT?

Edited by Bureaucromancer
Link to comment
1 hour ago, Bureaucromancer said:

I don't have much experience mucking about with kernel parameters, any particular order my appends should go in syslinux?  My first attempt to use kvm-amd.npt=0 just led to vms eating 100% of the assigned cores and never booting.

 

I ran into this issue as well when tinkering with the params.  Only the vm would usually boot the second try after killing it, and it was only the windows one that had the problem.  I'm not having that issue right now, so maybe adding iommu=pt and/or kvm.ignore_msrs=1 will help with this.  I'm not sure that the order makes a difference, but below is the current configuration that is working for me:

default /syslinux/menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot kvm-amd.npt=0 iommu=pt kvm.ignore_msrs=1

 

1 hour ago, Bureaucromancer said:

Any in depth discussions of this you know of?  I've found one on the redhat forums, that ends up suggesting following the iommu mailing list, but are there any other things I should be following?

 

The redhat vfio-user list is what I'm following,  I did see mention of the iommu mailing list, but I wasn't sure what list that was referring too.  I've not been active in the discussions, just following along...

Link to comment
1 minute ago, Some Dude said:

The redhat vfio-user list is what I'm following,  I did see mention of the iommu mailing list, but I wasn't sure what list that was referring too.  I've not been active in the discussions, just following along...

 

I did find a post referencing this on the iommu list, but no replies, so right there with you.

Link to comment
1 hour ago, Bureaucromancer said:

Edit: ok, so quick and dirty fix was to switch to QEMU (from host-passthrough) with NPT off, suddenly the GPU is used properly, but of course my CPU performance has gone to hell.  NPT is obviously going to have some loss, but is there anything much to optimize performance with QEMU or get host-passthrough workign without NPT?

 

Here is the relevant part of my win10 xml.  I'm using host-passthough and I've removed any cpu pinning at the moment.

<os>
    <type arch='x86_64' machine='pc-q35-2.7'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/xxx_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>

 

Link to comment

Interesting.  Only bits different are apparently kvm hidden state and that I'm trying to use i440fx.  Tried combinations of those with no changes...

 

Edit: I take it back.  I forgot to turn off pinning in the above, Q35 without pinning seems to have got things going.  i44fx doesn't, and neither does with CPUs pinned.  Weird.  Also amusing to see the system try to sort out cpu time with 100% locked cores that aren't pinned (and I'd say a bloody miracle that it didn't lock up given that everything else under the sun seems to crash my system right now ;).)

Edited by Bureaucromancer
Link to comment

And in further weirdness it seems in deleting my pinning I accidently set the vm to a single core and that it works with such.  It does not with the full 12 cores I usually have, experimenting now to see if I can track a hard limit down.

 

Another edit:  what I thought was a hard lock to 100% cpu may not actually be such.  I'm letting it sit witht he pegged cores at the moment on the full cpu set I want, having found that the time to get through the windows splash screen drastically increased every time I increased the core count.  The weird thing is that the progres spinner is moving now and then, so it looks like SOMETHING is happening behind all that CPU activity and its somehow related to number of cores in play.  I almost wonder if it's something odd about handling of the CCX units on Ryzen now?

Edited by Bureaucromancer
Link to comment
On 4/17/2017 at 4:35 PM, Pauven said:

Picking the right motherboard helps, and using AMD graphics seems to make the process a bit easier and perhaps cheaper, though you can go nVidia too, just with a bit more work to do the BIOS passthrough.  If you use HDMI sound passthrough that seems to alleviate much of the issues of bad IOMMU groups on sound devices and crackling on USB sound, but it seems there are solutions for both of those issues as well, all described in this thread.

 

I haven't tried this myself, still testing up-time with Global C-state Control disabled.  Today I reached 14 days of uptime.

 

-Paul

 

 

For all those who are interested in using Nvidia cards, spaceinvader one on youtube did release a new way where you don't need a second gpu any further. The files from techpowerup will work just fine now. What the files had were special headers for Nvidia flashing cards. The youtube video goes through a guide on opening up a Hex editor to remove all the unnecessary junk so you can pass through the vm. This saved me a bit of frustration as my first attempt i somehow created a blank bios dump.

 

Congrats on the long up time!
 

 

 

On 4/22/2017 at 5:42 AM, Bureaucromancer said:

I'm actually getting some weird performance issues in games as well.  Some combination of spiking to very low framerates and/or very low resource (both cpu and GPU) utilization with low framerates as well, but yeah, this isn't consistent enough to have traced yet...  I've actually been leaning to there being something about Dirextx 11 that messes with performance on VMs?

 

I've been having some of these issues as well where the fps changes all over the place. Tested on two different games. First game I tested was Portal 1, smooth gaming experience. Next game I tried was Borderlands the Presequel and had dips and changes quite often. Moved the game to the cache drive I have installed and still had same issues. Thought cache drive may fix it, but that wasn't the case. Changed resolutions as well hoping that would fix it and didn't. I game on a 2k ultrawide monitor. Has this ever been a problem with the Intel boards?


Post Edit:

Display Port/HDMI sound passthrough works. Never looked too much into getting the sound card on the motherboard to work as I finally found what was causing the issue I believe. Changed the cpu cores from 0-7 to instead use the cpu cores 8-15 for the windows vm. Playing around with dockers and haven't had any issues. Only one so far i've really dedicated too any time is Plex. 

Edited by GViz
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.