System:

Unraid 6.5.1

dual E5-2670 xeon

32g memory (non-ecc)

asus z9pa-d8 motherboard

970 and 660Ti

 

2x 4TB and 1x 1TB Spinners for Array

 

2tb for Movies an TV-Shows

1tb hot spare (will be a 4tb one)

120gb SSD for appdata and download cache

 

512gb Samsung nvme SSD for cache (and storage for the VM's)

 

Use case:

Because i didn't really need this workstation any more I figured: why not make it a server?

Dockers: Plex, Crashplan  Pro, Sonarr, transmission, etc.

VM's: 2 vm's for gaming and I want it to run Lightroom and some Premiere Pro.

 

Problems on the 970 VM:

1. Massive latency - which I can feel with moving the mouse, and typing on the keyboard.

2. GPU not functioning properly: if you move a window you can see the screen building up block by block. As if the GPU can't keep up

3. Stuttering audio with dropouts (probably related to #1)

4. Disk speed only half of rated speed (NVME Samsung cache drive)

5. Programs startup are stupid slow

 

Solutions I tried on the 970 VM:

1. Because of the USB latency i did a PCI passtrough of the entire controller. This seemed to help with the mouse and keyboard being laggy.

2. Ethernet Latency (latencymon told me of this one) this is solved 'somewhat' with passing through one of the gigabit ethernet ports

3. Setting BIOS C states to disabled (no difference)

4. Setting all performance options ON in BIOS. (So no power saving) (no difference)

5. Pinning CPU cores - isolcpus=8-15,24-31  (Didn't seem to make ANY difference)

6. Set emulator cores (Didn't seem to make ANY difference)

7. Set everything to MSI instead of IRQ with the MSI tool (No difference)

8. Disable memory ballooning (no difference)

9. Did the: ' vfio-pci.ids=10de:13c2,10de:0fbb' trick in de Syslinux config file (No difference)

10. Tried different memory sizes (8g to 24g) (No difference)

11. Tried less cores (from 4 to 16) (No difference)

12. I only use 1 NUMA node, and with the correct core groups (Some difference. I assigned cores from both NUMA nodes and this made it worse)

13. I installed the newest Virt-io drivers (no difference)

14. Trying all this I only used this one VM. NO OTHER VM WAS ON WHILE TESTING

15. I used a usb DAC instead of the NVIDIA HDMI audio - stuttering and ticks are less but still there.

 

Second VM:

the VM with a 660ti. I don't now if this is related but when installing the drivers for the 660ti the ENTIRE system crashes.

 

So... I'm on the end of my wits here. 

Is there anything else I could try? Maybe some mad BIOS setting I overlooked?

 

Any help will be much appreciated!

unraiddata-diagnostics-20180511-0958.zip

Win10 met passtrough GPU-USB-ETHERNET.xml

Edited May 11, 20188 yr by robertoal

Some other things I tried:

 

16. Disabled NUMA all together

17. Disabled Power Management all together

18. Disable Hyperthreading

 

These steps combined made it even worse. Now the mouse is lagging almost the entire time. Even keyboard presses don't register real time.

 

So... up. Anybody?

FINALLY!

Progress:

 

QPI (QuickPathInterconnect) seemed part of the problem.

Settings I used:

Memory snoop -> Early Snoop

linkop/linkos/mesagen -> Disabled

 

And I updated to the latest virtio drivers.

 

Still not perfect.. So: anybody? Something more I can look at?

 

I will post my BIOS settings for other people to learn from.