Issues with VM's Performance.


Recommended Posts

Good evening all, have been reading through the forums alot, but this is my first post! So to begin, a little backstory...

 

I have been looking into getting into unraid for quite some time, not necessarily because i need to or its hugely beneficial, but mostly because i find it interesting and its something fascinating and not widely adopted.. however, it does also have a purpose, space saving!

 

I currently have a spare room in the house which has been dubbed 'Computer Room' however, it is rather small, and ideally i am to fit 2 PC's in, which is easily possible with 2 towers, however i like my 2 monitors for myself and my partner has a monitor for themself. And its also easily possible by just buying a laptop, in which i do already own two, but again, i prefer the upgradability of desktop PC's. So all in all, yes it would be easier and less hassle, and likely cheaper just to build 2 seperate PC's, however, for one, i dont want too for various reasons as stated above, and also, ive already got the hardware i plan to use.

 

Now that the backstory is out of the way, my hardware for my unraid build..

Intel Core i9 7900X 10-core CPU

Asus Strix-E x299 Mobo

24GB DDR4-3000 Ram (Planned to be upgraded)

1x Samsung 850 Evo SSD 500GB

1x Samsung 850 Pro SSD 512GB

1x 2TB Hdd

1x GTX 1080ti Strix

1x GTX 1060 6GB Strix

EVGA Supernova 850 G2 PowerSupply

USB 3.0 PCI-E card

Bequiet! Dark Base Pro 900 Case 

All Custom loop water-cooled - including GPUs + Mobo VRMs.

UNRAID VERSION 6.3.5

 

So thats my PC, and i have already setup unraid using its free trial key, and it is working, but not properly..

 

What i vision being the perfect setup is basically what i have, which is 2 windows 10 VM's:

1st VM to consist of:

4-Core/8-Thread

12GB RAM

GTX 1080ti

USB Controller Passthrough

Installed on Samsung 850 Pro SSD+portion of 2TB HDD for storage.

 

2nd VM to consist of:

4-Core/8-Thread

8GB RAM

GTX 1060 6GB

PCI-E USB card passthrough

Installed on Samsung 850 Evo SSD+portion of 2TB HDD for storage.

 

The above i currently have running succesfully, and is what im currently typing on right now.. All passthroughs are working as id hoped, with the 1080ti having to have the vbios rom file manually inputted into XML file, which is succesful and USBs also working as hoped with hot-swappable ports (thanks to passing through whole USB controller on both VM's).

 

All sounds good right? Unfortunately no, that is where my joy ends, i will firstly point out that VM1 is going to be used primarily for gaming and general PC taks, and VM2 primarily used for lighter gaming and photo editing with document writing ect also.

 

Gaming is where my issue begins.. the performance is VERY poor compared to what id expect, even the VM in general is slower than id expect from an SSD or even a HDD at some point and even occasionally getting mouse stutters. The symptoms of the gaming performance is, using Player Unknowns Battlegrounds as test game, Running on VM1 (1080ti VM) id be at a rock solid 60fps at any settings (V-sync) but only when standing still, once looking around the FPS drops to around 15-20ish, sometimes worse.. In past experience i believe this to be CPU bottleneck, but i may be wrong. I have tried more cores on VM, less cores on VM, disabling hyperthreading all together, none of which helped.. 

 

After a quick search i come around CPU isolating an CPU pinning, and continued to set that up, however i do not believe anything has changed at all, i will be posting all XML files below (VM1+VM2+Flash config XML)

 

I also believe that even though i have assigned certain cores (paired with relevant threads using CPU pairings in system devices) to each of the VM's, they are affecting eachother, i have tested this by having both running and setting up a CPU cryptocurrency miner.. I started off with the one VM running miner, and it was doing approx 200 hashes (speed) while the second VM was idle. Secondly i tried running the miner on both VM's simultaneously, As soon the second VM started to mine, the 1st VM dropped its speed down to 130 hash, and the second VM sitting around 150 hash.

 

Also as a side note, i have the 2TB hdd in the array by itself, the 850 Evo is currently set as a cache drive and my 850 pro is mounted outside the array/cache using unassigned devices, this setup gave me best general performance when installing windows ect, having both as unassigned or both as cache led to VERY sluggish speeds. I also would prefer to have the SSD's seperated between the  VM's so would prefer not to have both in cache, so can use full 500GB on each VM.

 

Not sure what exactly im doing wrong, i do have a few theories which im hoping to test over this weekend, first of which being BIOS update, which i can easily do, and second being maybe PSU not powerful enough? i do not currently have a way to test this unless i undervolt GPUs ect, but i would have thought 850W plus Gold PSU would be able to handle this hardware.

 

P.s All XML is in the one Word Doc.

Flash+VM1+VM2 XML.txt

Link to comment

Get the tips and tweaks plugin and set the CPU governor to performance and turn Intel turbo mode to on. Also set your VMs to high performance mode. ssh into your server and run "cpufreq-aperf" in the cmdline to see where your CPU is clocking. I think the max all core boost of that chip is 4.0 GHz.

Sent from my SM-G955U using Tapatalk

Link to comment

All of the suggested above has been done, all in performance modes ect, and through ssh all cores have been set to minimum of 4ghz, just to test if its possibly just scaling poorly, but still dont understand why my 2nd vm affects the performance on vm1 cpu wise, they arent shareing cores and accord to unraids system devices they aint shareing threads either, so thats where im currently at, still no improovement on performance though unfortunately, thankyou for your reply, its much appreciated:)

Edited by CoryShepherd
Link to comment

Does anyone have any ideas or solutions to my issues? Im out of ideas and havent found much online to help, i have tried resetting everything up from scratch after reformatting all drives ect and deleting all VM's but end up with same result at the end. I feel like i must be missing something somewhere. Thankyou in advance.

Link to comment
13 hours ago, nuhll said:

Im not sure, but i read somewhere you shouldnt overclock for unraid. But could be wrong. You may test if it helps anything.

 

Also, there are videos on youtube about how to performance setup your unraid VMs. Search for it. (im at work now)

 

 

Woops i have had all my Unraid gaming rigs overclocked. Just not my unraid nas box. Had my 6950x running at 4.4Ghz for ages and never had any issues. 5960x before that.

 

 

14 hours ago, CoryShepherd said:

I will certainly give that a try then! May i ask, what sort of performance are you getting now that your up too 6.4? Essentially im wondering what gpus you have and if your planning on playing games, what games are you getting good fps on ect, just want to be able to compare and see if i can match ect, thankyou:) 

9

 

I got 2 gaming VM's running with 4 cores each. HT on. Let me know what game you want to bench and if I have it I will be happy to bench it and put the results up here. So far the performance compared to my 6950x overclocked it better. not sure if being on 6.4 might have made a difference though. Also, the GPUs perform a little better as well.

 

 

Edited by heratic
Link to comment
On 19/12/2017 at 6:15 AM, heratic said:

 

 

Woops i have had all my Unraid gaming rigs overclocked. Just not my unraid nas box. Had my 6950x running at 4.4Ghz for ages and never had any issues. 5960x before that.

 

 

 

I got 2 gaming VM's running with 4 cores each. HT on. Let me know what game you want to bench and if I have it I will be happy to bench it and put the results up here. So far the performance compared to my 6950x overclocked it better. not sure if being on 6.4 might have made a difference though. Also, the GPUs perform a little better as well.

 

 

 

Very similar to my setup then, what GPU's? As i too have 2 VM's for gaming, one with 1080ti and one with 1060 6GB. I Have upgraded to 6.4, and i have definitely noticed a performance increase, but i am still getting the odd dip in FPS which i believe to be cpu related as it will be the same fps at ultra as it is at medium/low on Playerunknowns Battlegrounds. Going as low as 30fps on my 1080ti which is worse than the 1060 had on bare metal. But Playerunknowns is the main game id like to know, or anything else you may have thats relatively intensive.

 

Do you have a 3rd gpu or are you doing a manually specifying a ROM file like i am?

 

I also noticed that when booting up the PC into unraid, to the screen that says 'Tower login', it specifies that the CPU governer is set to 'power saving' when i have it set too performance in tips and tweaks and it also claims it as being in performance mode within tips and tweaks.

 

Any help is appreciated. Thankyou.

Link to comment
On 18/12/2017 at 1:49 PM, heratic said:

Also my 7900x is the same performance stock as my 6950x was overclocked to 4.3Ghz on all cores.  Can't wait to see how it performs overclocked. Gpu performance also seems to be slightly improved on 6.4 as well

 

Just a heads up, although you may have figured this out already, but for updating video drivers i had to revert back to 6.3.5, constant freezing would occur every single time and vm would lock up and would be unable to navigate GUI forcing a hard reset!

Link to comment

I have two 1080tis. No third GPU just booting off a rom. I haven't had any issues with the graphics card drivers. Also overclocked the cpu to 4.8 on all cores. so far so good.

 

played PUBG for a few hours last night. 3440x1440 on new map i get around 80 to 99fps. FPS are capped @ 99 on my machine. Old map sits around 60fps. all settings maxed out. gpu power set to 90% no overclock. just what i mine at so left it. Both 1080tis are watercooled.

Link to comment
12 hours ago, heratic said:

I have two 1080tis. No third GPU just booting off a rom. I haven't had any issues with the graphics card drivers. Also overclocked the cpu to 4.8 on all cores. so far so good.

 

played PUBG for a few hours last night. 3440x1440 on new map i get around 80 to 99fps. FPS are capped @ 99 on my machine. Old map sits around 60fps. all settings maxed out. gpu power set to 90% no overclock. just what i mine at so left it. Both 1080tis are watercooled.

 

Thats very good to hear, i am having a new issue with the gamestream feature from nvidia, dont know if you know anything about that haha,

 

but your 4.8Ghz overclock, is that stable? As im looking to squeeze abit more from my chip, i assume you had to give voltages a bump ect, how are your temps?

 

But back on topic, i will be getting some time to play tomorrow, will see if i get similar results. Thankyou for your replies.

Link to comment

4.8Ghz is all i can get out of my system and keep them temps under 90c during extended stress testing. volts are set to 1.28 but looking at monitoring software its closer to 1.3v.

 

i don't have g-force experience installed it that's what game stream is part of. i have used steam game streaming no problems though.

 

We should find a few benchmarks that have scores. ie 3d mark, tomb raider, LOTR mordor, company of heroes 2, valley, heaven or whatever and compare scores. no driver optimizations and no overclock on the gpu's, 100% power limit, to get a base line of VM performance. if you could get your cpu stable at 4.8Ghz that would be ideal as returning to stock would be a hassle for me. I will hopefully get a chance over the weekend to do this and post them back in this tread. Let me know what games your interested in comparing.

 

My VM will consist of:

 

4-Core/8-Thread

16GB RAM

GTX 1080ti

USB Controller Passthrough

Installed on Samsung 850 Pro SSD

Link to comment

I've actually been suffering some poor performance myself, though I'm on a Ryzen 1700x build (see signature)

2 Gamers 1 PC, giving each person a CCX or 3 cores of a CCX and leaving the other 2 for Unraid.  Emulatorpins and isolation for the 3core variant are a given.

I still tank below 30fps when roaming in Warframe.  Other games are a grab-bag.  This is infuriating.  

I was curious about the "high performance" mode you mentioned earlier for VMs.  Can you tell me where to find that setting?

One thing I will note, it seems I get low usage out of CPU and GPU despite the low framerates.  Using AfterBurner+RivaTuner to determine this, was well as Unraid's core readout on the dashboard.  Neither is working very hard, and bare metal is fine, so the bottleneck appears to be in Unraid.  

Link to comment
14 hours ago, heratic said:

4.8Ghz is all i can get out of my system and keep them temps under 90c during extended stress testing. volts are set to 1.28 but looking at monitoring software its closer to 1.3v.

 

i don't have g-force experience installed it that's what game stream is part of. i have used steam game streaming no problems though.

 

We should find a few benchmarks that have scores. ie 3d mark, tomb raider, LOTR mordor, company of heroes 2, valley, heaven or whatever and compare scores. no driver optimizations and no overclock on the gpu's, 100% power limit, to get a base line of VM performance. if you could get your cpu stable at 4.8Ghz that would be ideal as returning to stock would be a hassle for me. I will hopefully get a chance over the weekend to do this and post them back in this tread. Let me know what games your interested in comparing.

 

My VM will consist of:

 

4-Core/8-Thread

16GB RAM

GTX 1080ti

USB Controller Passthrough

Installed on Samsung 850 Pro SSD

I will see if i can get my clocks up that high over next few days when i get some spare time. Hopefully wont be too much hassle, but will soon see.

 

And yes definitely, not sure what games to try but im happy to give all the scored benchmarks a shot.

 

And yes my VM is Identical, leaving core/threads 0-1/10-11 for unraid and 2-5/12-15 for this specific VM.

 

Your 850 PRO is that mounted outside of the array using unassigned devices? thats how i have mine set up.

Link to comment
Just now, thenonsense said:

I've actually been suffering some poor performance myself, though I'm on a Ryzen 1700x build (see signature)

2 Gamers 1 PC, giving each person a CCX or 3 cores of a CCX and leaving the other 2 for Unraid.  Emulatorpins and isolation for the 3core variant are a given.

I still tank below 30fps when roaming in Warframe.  Other games are a grab-bag.  This is infuriating.  

I was curious about the "high performance" mode you mentioned earlier for VMs.  Can you tell me where to find that setting?

One thing I will note, it seems I get low usage out of CPU and GPU despite the low framerates.  Using AfterBurner+RivaTuner to determine this, was well as Unraid's core readout on the dashboard.  Neither is working very hard, and bare metal is fine, so the bottleneck appears to be in Unraid.  

Sounds similar to the issue i was having, are you on 6.3.5? or are you on pre-release? as me and heretic found being on pre-release helped our performance issue.

 

I have also heard alot about ryzen not being greatly optimised for unraid yet, although it seems they have been making alot of progress recently, but i cant be sure.

 

As for the high performance mode, not sure if its different for amd or not, but you have to install the 'tips and tweaks' plugin for unraid, accessible through the GUI, once in the plugin, you will have a list of tweaks for your unraid server, many of them i dont fully understand, and it also has details for your current settings down the right side, but one of the tweaks was about CPU Scaling Governor which i set to performance, what this does is sets all your cores to run at max frequency regardless of load. I also had to make sure that my BIOS settings were to allow the OS to control Power Management settings.

Link to comment
7 minutes ago, CoryShepherd said:

Sounds similar to the issue i was having, are you on 6.3.5?

Was on rc10b because it had the ryzen NPT fix, updated to rc18 to see if it helped.

 

7 minutes ago, CoryShepherd said:

As for the high performance mode, not sure if its different for amd or not, but you have to install the 'tips and tweaks' plugin for unraid, accessible through the GUI, once in the plugin, you will have a list of tweaks for your unraid server, many of them i dont fully understand, and it also has details for your current settings down the right side, but one of the tweaks was about CPU Scaling Governor which i set to performance, what this does is sets all your cores to run at max frequency regardless of load. I also had to make sure that my BIOS settings were to allow the OS to control Power Management settings.

Oh so you were talking about the CPU scaling governor.  Unfortunately as of now I don't have a driver for this, but that just keeps my CPU maxed at all times.

 

Re-reviewing SpaceInvaderOne's gaming VM videos to seee if there's anything I missed.

 

On 12/9/2017 at 3:29 PM, david279 said:

Get the tips and tweaks plugin and set the CPU governor to performance and turn Intel turbo mode to on. Also set your VMs to high performance mode

So the second part of this statment (about the VMs) corresponds to what?

Edited by thenonsense
Clarity
Link to comment
Just now, thenonsense said:

Was on rc10b because it had the ryzen NPT fix, updated to rc18 to see if it helped.

Oh so you were talking about the CPU scaling governor.  I thought this was a different setting.  Unfortunately as of now I don't have a driver for this, but that just keeps my CPU maxed at all times.

 

Re-reviewing SpaceInvaderOne's gaming VM videos to seee if there's anything I missed.

 

So the second part of this statment corresponds to what?

I Believe that, or atleast what i took from is, was going into windows power settings and making sure its on the high performance power plan rather than power saving.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.