X399 and Threadripper


Recommended Posts

Looking for some insight here. I have a Win10 VM set up with 6C/12T and 16GB RAM allocated, and GPU and NVME passthrough.

 

When I run the command below to monitor the clock speeds of different cores, they all appear to be running at correct speeds.

watch grep \"cpu MHz\" /proc/cpuinfo

 

However, when I do video renders in Adobe Media Encoder or Handbrake the clock speeds drop by half or more.

 

System Specs are:

 

TR 1950X (Uraid temps reporting around 60c)

Enermax Liqtech TR4 360

TaiChi x399

64GB (8x8) Corsair Vengeance running at 2934

2 x NVME

3 x GTX970 PCIE 1, 4, 5

9 x HDD

2 x SSD

1 x 4 port SATA expansion PCIE 3

1 x 4 port USB expansion PCIE 2

 

Anybody got any ideas?

Link to comment
45 minutes ago, jspence27 said:

Yes exactly. I just changed my hardware config to do some testing so I can't provide a screenshot. But, Idle I was seeing the clocks registering around 3000-3200 MHz, and when I render they drop to around 1200-1500 MHz.

 

Sorry, but I don't understand how that can be. Are you measuring the clocks the same way or is one inside the VM and the other in unRAID? Why is your processor idling so fast - have you been trying to overclock it and left something set wrong in the BIOS? Do you not have a governor applied? Have you mis-allocated the cores (thread pairs) to the VM?

 

Post your diagnostics.

Link to comment

I'm ssh'ing into the box, running the command above and observing the output. From my understanding the command outputs into the the terminal the current clock speeds of each core/thread. The CPU is running at it's base clock in the bios, 3.4 GHz, no overclock. No governor is applied currently. I tried different governors via the Tips & Tricks plugin to see if they had any affect. I saw the same behavior with each of them. All cores are allocated with their pairs and isolated from unraid to be used by the VM. After some testing it looks as though my pump is bad in the water cooler, so the CPU may be throttling. That seems odd though as the temp reported in unraid is 60c and should be within operating temp. I have a different cooler coming tomorrow to test against.

Edited by jspence27
Link to comment

Booting into the bios the CPU is showing 90c. One of the tubes on the cooler is really hot and the other is room temperature. I found a post on Newegg where someone discovered that meant the pump was malfunctioning. Initially, I was going by the sensor package when I wrote my question. The box has been operating fine, no instabilities or anything. I just noticed my render times had gone up so I started digging. I assume now it was because of the CPU down-clocking under load. Hopefully the new cooler will eliminate the behavior.

Link to comment
  • 3 weeks later...
New cooler fixed it. Switched from Enermax Liqtech 360 to Noctua NH-U14S.

Good choice. I’m using the arctic freezer 33 and I’m idling at 42 or so.

Have many HDDs on a fully enclosed fractal R5 so I don’t think 35 degrees or so idle is achievable.

I’m sure the 14s would drop a few degrees but I’m ok with this as at least it’s not too huge.

It’s pretty crazy how this high core count CPUs are having such Low temps.

My usual non AVX load tops out at 57...madness. Load on my old i5 was at 60 usually.

Hopefully others too are on air cooler for their 24/7 server. Kinda scary to run on water for these sort of systems.


Sent from my iPhone using Tapatalk
Link to comment
On 4/25/2018 at 1:42 PM, jspence27 said:

New cooler fixed it. Switched from Enermax Liqtech 360 to Noctua NH-U14S.

 

I just installed a Liqtech 280 and much better results than my H115i due to the better plate.... I am surprised you found the Noctua better. Send that 360 over my way then... I'll have it ?

Link to comment
On 5/15/2018 at 4:52 AM, methanoid said:

 

I just installed a Liqtech 280 and much better results than my H115i due to the better plate.... I am surprised you found the Noctua better. Send that 360 over my way then... I'll have it ?

 

The 360 worked great when it worked. I bought it in November and the pump has already died.

Link to comment
1 hour ago, Tybio said:

The Noctua TR4 fan is actually beating the AiOs for the most part, just goes to show that design is more important than buzz.  I'll take perfectly designed old tech that just "works" over new-bling that's not as well proven any day.

 

I just don't like the idea of water cooled computers. I mean, what could possibly go wrong?!

  • Like 1
Link to comment

Interesting, I have the Corsair H100i V2 AIO mounted in-case (no direct clean air or exhaust) with 2-120x15mm slim fans running at 650rpm and this thing idles at 38C and goes up to maybe 42 under load (with fans jumping up to 750rpm). All BIOS controlled. Clean air she ran 32C at idle.

 

EDIT - I'm using thermal grizzly hydro I think it was

Edited by ryoko227
Link to comment
On 5/17/2018 at 8:04 PM, John_M said:

 

I just don't like the idea of water cooled computers. I mean, what could possibly go wrong?!

 

Lots of other equipment manages just fine with water cooling, so the concept as such has a quite good track record.

 

It's just that water cooling solutions for PC equipment doesn't seem to be supplied with commercial grade pumps. People don't want to pay maybe $300 for just the pump. And that's probably the reasonable price for a pump designed for 20+ years of service life so the pump can manage a similar low failure rate as the other critical parts (processor, motherboard, PSU, RAM) of the system.

 

And besides that, lots of water cooling solutions has toy-like connectors that doesn't age well. It doesn't help if the pump works well if the system leaks coolant. And where does any leaking coolant go?

 

So in the end - if selecting between commercial-grade fans or semi-toy water cooling, it's important to remember that the standard water cooling systems really are not of the same quality. But they look great. ;-)

Link to comment
  • 5 months later...

Ah, here are my taichi running brethren!  Since you guys seem like you have tried a lot of usb passthrough configs, I was hoping you could help me out with my USB nightmares.... Only seeing 3 usb controllers- and really only one that I could passthrough.  Maybe you can post your controllers and any that worked for passthrough so I can glean some of the info I need to get mine working?

 

 

 

Link to comment
1 hour ago, jordanmw said:

Only seeing 3 usb controllers

For me it's kinda the same. My AsRock Fatal1ty has 2 USB 3.0 and 1 USB 3.1 controller. I guess thats the same for you on the Taichi. For me with the latest BIOS and the latest Unraid build 6.6.3 the IOMMU groups are split really well even without the ACS Override patch. The only issue is that the 3.1 Controller is grouped with all the network devices. If i add the following to the syslinix config i could spilt them all in it's own groups, but i don't really need to, because, i don't passthrough any of these.

pcie_acs_override=downstream,multifunction

The following shows my current USB devices and their groupings without ACS override. BIOS version is the latest 3.30

IOMMU group 13:	[1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02)
	[1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02)
	[1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02)
	[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
	[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
	[1022:43b4] 02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
	[1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
	[1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
	[1d6a:d107] 03:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
	[8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
	[8086:24fb] 05:00.0 Network controller: Intel Corporation Dual Band Wireless-AC 3168NGW [Stone Peak] (rev 10)
	[8086:1539] 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)

IOMMU group 17:	[1022:145c] 09:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller

IOMMU group 37:	[1022:145c] 44:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller

 

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.