Hi all,

With the fixes to Unraid+Ryzen gaming compatibility I've swapped back to my shelved Ryzen parts and figured now it's a good time for a full Unraid overhaul.  I come to you because while I have all the hardware I need, I'm not sure how I want to configure it.  Currently my build is:

1700x - split in half for 2 VMs, no cores isolated or for use by Unraid exclusively

MSI Gaming Pro x370 - 1 USB controller passed through to each VM (one onboard, one PCIE)

2x Samsung 1TB SATA SSDs - 1 in the array as fast storage, 1 unused brand new

Intel 128GB M.2 SSD - split in half for our VMs' OS images

2x Seagate 3TB HDDs - other array storage

2x GTX 1080s - 1 per VM

and I have it configured for two gaming VMs (Gilgamesh and Enkidu) for my gf and I.  I've over time noticed some weird frame drops and various slowdowns in gaming and because of this want to re-init the whole thing from scratch.  After that I want to re-optimize the setup but am unsure how.  Adding the TB SSDs to the cache drops their capacity because the cache pool is enforced RAID 1 and the M.2 is there.  I would pull the M.2 out but it's just so damn fast.

Maybe the best config will be to get rid of the M.2, move the SSDs to the cache pool, have a share for the cache pool, put the OS images and games in the share (let Unraid decide disk usage) and have the HDDs only in the array.

However I know my judgement is lacking and I come to those of more experience for their thoughts.  If you have anything better to recommend, or believe the current direction is strong, please leave your comments here.  

 

I'm not expert.

But things I would try.

 

• Leave out 1 core for unraid (core0)
• Have you checked the core pinning to make sure you are giving the vm's the correct cores (not entirely sure if this is important on AMD).
• Maybe consider giving each VM direct acces to 1 of the SSD's and use the m.2 as the cache ?

 

@spirituality,

5 hours ago, spirituality said:

Leave out 1 core for unraid (core0)

I'd rather try and keep the VMs equivalent, for my CDO's sake.  I can drop both down to 3 though, and leave one on each CCX to Unraid, not like it'll care about the latency between the cores.  I tested this and since there's almost no usage on Unraid's end, I think those cores would be wasted.  That's why Unraid doesn't have one now.

5 hours ago, spirituality said:

Have you checked the core pinning to make sure you are giving the vm's the correct cores (not entirely sure if this is important on AMD).

Haven't isolated yet.  If I give up cores to the Unraid gods then I will.

5 hours ago, spirituality said:

Maybe consider giving each VM direct acces to 1 of the SSD's and use the m.2 as the cache ?

Leave the SATA SSDs in the array?  That'd be a viable option.  Not sure how I'd like just 128GB in my cache though.

Edited November 15, 20178 yr by thenonsense

8 hours ago, thenonsense said:

I'd rather try and keep the VMs equivalent, for my CDO's sake.  I can drop both down to 3 though, and leave one on each CCX to Unraid, not like it'll care about the latency between the cores.  I tested this and since there's almost no usage on Unraid's end, I think those cores would be wasted.  That's why Unraid doesn't have one now

 

I understand that. But it's worth trying ! I'd take off the core and maybe run a benchmark, to be honest, you might not even notice... I'm using 3 cores of an I5 and i was surprised how little difference it made in game, you can always switch back, it's just for troubleshooting.

 

As for the core pinning, check out this forum topic. It's worth it and not that much trouble

 

And yeah leaving the SSD's OUTSIDE the array would make the vm's work very well, but you do loose some redundancy I guess and yes 128gb is a little small for cache

@spirituality I assumed you meant having the VMs use the SSDs on [separate?] shares.  Passing them through unassigned is also a nice option, and leaves the computer able to boot from them. 

If I boot to Windows from bare metal (without Unraid) and activate then boot Windows from within Unraid wouldn't my OEM licenses yell at me for the abstracted hardware change?

 

Since the majority of the work (video games) would be off the server-reliant hardware, the cache could remain small.  I like that idea.

 

I'm not sure how I'd go about making backups of the OSs.  When they're images it's a lot easier to copy the vdisks to mass storage.  If you'd have a way for creating backups I'd be willing to try it.

 

The CPU config/testing will definitely happen.  I'll look into this for tonight.

 

Edited November 15, 20178 yr by thenonsense

CPU testing consisted of running the ROTTR bench and the CPUZ bench. It seemed like the best performance was from indeed setting a core aside, as well as assigning SMT cores in the cpu config as 1-3-2 rather than 1-6-1, or 3 cores SMT rather than 6 cores. Also isolating all cores used by the VM. 

I’m going to push forward with the SSD sata passthrough for each vm. Yesterday I couldn’t gain a frame above 70 in HOTS, and it would dip below 30 occasionally. Something’s not right there, and I’d rather test from a clean config. 

Download the Tips and tweaks plugin and set the cpu governor to Performance. I have a 1600x system so thats what Im using. I also OC my cpu using the zenstates script. https://github.com/r4m0n/ZenStates-Linux It allows me to hit the 3.925 on all cores that i was getting in Windows. 

@david279 I have the tips and tweaks plugin, along with Fix Common Problems.  Very useful tools. 

 

What is your cooling solution for 3.925 on a 1600x?  That seems like some huge gain.  I'm still struggling to find a proper temp reader for the CPU, so my overclocking has been modest.

 

As for the script you're pointing at, I'll definitely take a look.  CPU-Z in the Windows VM came up with my single-core performance being at 92% of reference, though, so I'm not entirely certain the issue is there.  

I have a cryorig h40 ultimate. I used to run my system at 3.925 @ 1.365 in windows and temps stayed in the upper 30s and low 60s under load so i know exactly how it performs. And the was with all cores at 3.925. In unRAID if i use the onDemand governor the core flux and only go to 3.925 if I need it. Performance will run all the cores at its set max.

Her is a cpuz screenshot of the bench ran in my VM with 4 core 8 threads. Windows can't really tell the CPU freq that really running of AMD cpus. I had to reference it against the 1800x because the 1600x wasn't in the drop down. Not to bad. 

 

 

Interesting.  So Windows TM doesn't know better.  

Air cooling and 3.925 GHz?  Maybe there is hope for me yet.  I'm running an open air case with air cooling myself.  I'll have to test tonight.  Just curious, how are you getting temps?

Cryorig h40 ultimate is a Water cooler with a little fan on top of the cpu block. Guess for cooling the board and the vrm. Works really well.

Oh.  My b.  Well I'll still review the temps and see what I can do.  What were you using to get the temps from your CPU?  

Really nothing right now. I can run sensors-detect in cmd line and it sees some then use modprobe to try to force them but the info it gives is kinda stale. Unless my CPU is between 33C and 43C at all times.

Sent from my SM-G955U using Tapatalk

Thanks for the tip.  

Restructuring has completed.  Both VMs are passed through from their own 1TB SSDs.  Benchmarking has yielded a few frames and percentage increases.  According to CPU-Z I'm at about 95% native single-thread.  Didn't OC yet.  Next experiment is that, for CPU and GPU.  

 

Gaming-wise I see a frame increase from avg 81 to avg 85 in ROTTR.  However some games like HOTS seem to not change no matter what I do.  Even when I drop settings they stay the same.  That's been a problem since I started with Unraid.  It's so weird.

 

Chrome is now being sluggish.  Webpages themselves are fine but the tabs are extremely laggy.  Click on one, wait a second or two, it shows.  I've done the grudging switch to Edge until I can crack that one.  Maybe hardware acceleration isn't a setting that carries over when you log in to your browser.

 

I don't want to turn this into a debugging thread.  That isn't what it was made for.  I appreciate the help you two have given me @spirituality and @david279.  I'll keep this thread updated over the course of the weekend with results.

 

Nice. I'm really thinking about getting one of those $320 1800x that newegg will be selling thru their Black Friday sale. That would be a big upgrade. Went back clocks for today as I'm testing governor performance. Gonna play with the ondemands governor and see how it works. Also cpufreq-aperf is good way to see just what your clocks are doing in cmd line.

Sent from my SM-G955U using Tapatalk

1800x for $320 is a steal.  Especially considering the premiums on parts right now.  Back in march I bought 16G of 3200 MHz for $116.  What happened to RAM?  

 

I'd be interested in hearing more about your governor adventures.  I remember intel having an issue with the ondemand gov when I first got into linux.  Not sure if it ever applied to AMD, or if it still exists today.  Please post back about what you find.

 

Go for the 1800x!  I think per dollar the 8-core series from Ryzen are the best chips this year.  So much future-proofing.  All unlocked.  What a time to be alive

So strange you are having issues with chrome, any chance you can keep an eye on taskmanager to check out if there is any "strange" peak in some sort of metric? cpu/ram/diskIO/... I've never noticed that on my VM.

 

Have you done any validation on your ram? (memtest for X hours/days ?) it's something I plan to do on my new hardware once I get it.

@spirituality,

Out for this whole week with the family, now back to the grindstone.  Thankfully the issue seemed to simply be Windows updates.  Chrome://gpu threw a bunch of errors that could be traced back to before Win10 anniversary, and once Windows finished updating, it ran like a charm.  

Dropped my clock to 3.7 GHz.  I have some fans coming in today, and if I'm not happy with my OC after installing them, I'll look at an AIO.  The graphics cards are plenty cool, so a custom loop hardly seems necessary.

As of now I consider the system fully functional, just needs some fine tuning.

Aaah good to hear ! an AIO does seem to be a good idea

I can just about run my 1800X at 3.975GHz with air cooling (Arctic Freezer 33 with dual 1200rpm 120mm fans), but it runs close to 75C in Cinebench R15.  When gaming it gets a bit warm due to my 980TI heating up the inside of the case.

 

So, I'm using a Kraken X62 which keeps the CPU down at 50C at most.  The CPU will do 4GHz but with lots of volts and lots of heat, but 3.975 is stable with a lot less volts.  Oddly, my old 1700X did the same 3.975 on a PRIME X370-Pro, but I had terrible issues with that board and CPU, so was replaced by the 1800X and Strix X370-F by Amazon.

Numbers sharing time.

Pulling 3.8 from my 1700X with the Core Frozr L, not a degree above 45 according to

    modprobe nct6775

    sensors

Maybe not trustable numbers, but I can't see a symptom yet and I did a LOT of testing over the weekend.

Being able to stream to Twitch from both gaming VMs is a riot on a single chip.  I love it.  

@spirituality As for an AIO, I believe it isn't necessary unless a deal pops out at me today.  

@HellDiverUK Good numbers.  There shouldn't be any task that really demands you risk your system's integrity and go to 4.0.  As for your kraken, I was eyeballing one of those or the Corsair.  Did you recycle the mounting from another cooler?  I don't think those ship with native mountings for AM4.

5 hours ago, thenonsense said:

 

@HellDiverUK Good numbers.  There shouldn't be any task that really demands you risk your system's integrity and go to 4.0.  As for your kraken, I was eyeballing one of those or the Corsair.  Did you recycle the mounting from another cooler?  I don't think those ship with native mountings for AM4.

 

This Kraken X62 replaces a Corsair H115i GT.  The Corsair was about 3 years old, and making some odd gurgling and rattling noises from the pump, so I decided it was about done.  It originally cooled an Athlon 9590 (the 220+W crazy chip), then a 6700K, 5930K, and a 7700K.

 

My X62 came with the AM4 bracket - you just twist off the Intel one, put on the AM4 bracket.  Then just screw in the standoffs in to the stock AMD backplate, and you're done.

Oh so it comes with the parts. Maybe after christmas I’ll grab one, when everything’s overstocked. 

This build is going on PCpartpicker again once I get the cable management stuff I ordered. I’ll link it. Have either of you pushed a build guide?  I think this setup is worthy of one.

On 11/28/2017 at 5:49 PM, thenonsense said:

Oh so it comes with the parts. Maybe after christmas I’ll grab one, when everything’s overstocked. 

 

Just check the one you buy is the new version with the AM4 bracket.  There's probably old stock out there that doesn't.