Skylake Unraid Build


Recommended Posts

Hi guys

 

I'm looking to upgrade my current simple build to a Skylake VM capable server.

 

Up until today, I thought that VT-d with IO was only available on the Xeon and Server class CPU's.

 

After reading through the forum, I tend to think I can achieve my goal with a desktop class cpu and motherboard combo. Is this right?

 

Can anyone recommend a tried and testing build which passes all IO's without any patches.

 

I'm probably thinking of the i5 6400/6500.

 

I only want to run 1 VM so hoping this will work.

 

Thanks, any help is appreciated 8)

Link to comment

Goodday KidCastro,

 

it's two ways:

 

CPU should support vt-x and vt-d and the motherboards as well. THere is a site that's lists the intel chipsets that support VT-d as VT-x is supported almost every motherboard.

 

I'l show you the link:

 

http://www.intel.com/content/www/us/en/support/boards-and-kits/desktop-boards/000005758.html?wapkw=vt-d+support

 

The following Intel® Desktop Boards support Intel VT with Directed I/O:

Chipset Desktop Board

H87, Q87, Z87 DH87MC, DH87RL, DQ87PG

B85 DB85FL

X79 DX79SI, DX79SR, DX79TO

H77, Q77, Z77 DH77DF, DH77EB, DH77KC, DQ77CP, DQ77KB, DQ77MK, DZ77BH-55K, DZ77GA-70K, DZ77RE-75K, DZ77SL-50K

B75, Z75 DB75EN, DZ75ML-45K

Q67 DQ67EP, DQ67OW, DQ67SW

H61 DH61AGL

Q57 DQ57TM, DQ57TML

Q45 DQ45CB, DQ45EK

Q35 DQ35JO, DQ35MP

 

I dont see Z170 on that.....

 

It's possible that the above list is outdated. Drop a question at ASRock support for the correct answer. If the manual says VT-d it probably will :P

 

 

Link to comment

Goodday KidCastro,

 

it's two ways:

 

CPU should support vt-x and vt-d and the motherboards as well. THere is a site that's lists the intel chipsets that support VT-d as VT-x is supported almost every motherboard.

 

I'l show you the link:

 

http://www.intel.com/content/www/us/en/support/boards-and-kits/desktop-boards/000005758.html?wapkw=vt-d+support

 

The following Intel® Desktop Boards support Intel VT with Directed I/O:

Chipset Desktop Board

H87, Q87, Z87 DH87MC, DH87RL, DQ87PG

B85 DB85FL

X79 DX79SI, DX79SR, DX79TO

H77, Q77, Z77 DH77DF, DH77EB, DH77KC, DQ77CP, DQ77KB, DQ77MK, DZ77BH-55K, DZ77GA-70K, DZ77RE-75K, DZ77SL-50K

B75, Z75 DB75EN, DZ75ML-45K

Q67 DQ67EP, DQ67OW, DQ67SW

H61 DH61AGL

Q57 DQ57TM, DQ57TML

Q45 DQ45CB, DQ45EK

Q35 DQ35JO, DQ35MP

 

I dont see Z170 on that.....

 

It's possible that the above list is outdated. Drop a question at ASRock support for the correct answer. If the manual says VT-d it probably will :P

 

Thanks so much for all that info, yes I pulled up that list as well.

 

Is there any list of builds that are successful with Skylake? Has anyone done it with unraid on here. I can't see a build log section. Hoping to copy a known working configuration.

 

Also, am I right in saying that the 6400 and 6500 support vtx and vtd?

Link to comment

With earlier CPUs I/O pass-through was provided by the chipset, not the CPU.  Any 4th gen or later Core architecture CPU that has vt-d support will work just fine with the chipsets that support them.  Note that the list above also doesn't include the x97 chipsets or any of the SkyLake chipsets => all of which work just fine with vt-d.

 

In other words -- Yes, your SkyLake system will work just fine as long as the CPU has vt-d support.

 

 

Link to comment

... Also, am I right in saying that the 6400 and 6500 support vtx and vtd?

 

Yes ... you can check support for any CPU by looking at Intel's ARK site

 

e.g. for the 6400:  http://ark.intel.com/products/88185/Intel-Core-i5-6400-Processor-6M-Cache-up-to-3_30-GHz

 

and for the 6500:  http://ark.intel.com/products/88184/Intel-Core-i5-6500-Processor-6M-Cache-up-to-3_60-GHz

 

Just look at the "Advanced Technologies" section to confirm what is/isn't supported

Link to comment

With earlier CPUs I/O pass-through was provided by the chipset, not the CPU.  Any 4th gen or later Core architecture CPU that has vt-d support will work just fine with the chipsets that support them.  Note that the list above also doesn't include the x97 chipsets or any of the SkyLake chipsets => all of which work just fine with vt-d.

 

In other words -- Yes, your SkyLake system will work just fine as long as the CPU has vt-d support.

 

... Also, am I right in saying that the 6400 and 6500 support vtx and vtd?

 

Yes ... you can check support for any CPU by looking at Intel's ARK site

 

e.g. for the 6400:  http://ark.intel.com/products/88185/Intel-Core-i5-6400-Processor-6M-Cache-up-to-3_30-GHz

 

and for the 6500:  http://ark.intel.com/products/88184/Intel-Core-i5-6500-Processor-6M-Cache-up-to-3_60-GHz

 

Just look at the "Advanced Technologies" section to confirm what is/isn't supported

 

Thanks man, appreciate the re-assurance... so I'm definetely thinking ASRock Z170 Extreme6+ and i5 6400  ;)

 

Any way of confirming if the onboard SATA is compatible, I can tell they use 2 x different chipsets.

 

- 6 x SATA3 6.0 Gb/s Connectors by Intel® Z170, support RAID (RAID 0, RAID 1, RAID 5, RAID 10, Intel® Rapid Storage Technology 14 and Intel® Smart Response Technology), NCQ, AHCI and Hot Plug

- 2 x SATA3 6.0 Gb/s Connectors by ASMedia ASM1061, support NCQ, AHCI and Hot Plug

 

ASMedia ASM1061 compatible?

Link to comment

I have just upgraded my unraid to  i7-6700 cpu and ASRock - Z170M Extreme4  motherboard. Similar mboard and chip as you plan to.

I have finally got everything working as i want now but did have a few problems which i thought i would point out to you in case it helps you when you build yours.

 

Firstly I have 3 pcie x16 slots and planned to run 3 graphics cards. I had to enable PCIe ACS Override to get this working as without it

the motherboard grouped the graphics cards on the first and second slot in the same iommu group. Both cards would work this way if only one in use at a time. If i tried to use each on a different vm at the same time the second vm to be started wouldnt work.

Well the aci override sorted this out and i can run 2 gpus at once passed to different vms.

However the 3rd slot with or without the aci override, the mb would always group the usb 3.1 (USB controller: ASMedia Technology) in with it in same iommu group. So i couldnt use the third slot for a gpu. I could try to stubb the controller and try that way, but at the moment i use that usb for my unraid, and passthough the other controller on the board to my windows vm. (the reason i do it this way is because i want to passthrough the controller that has the usb3 ports on the front of my case)

 

My second problem was an acpi error.

Jan 24 21:20:34 Prime kernel: ACPI Error: Method parse/execution failed [\_GPE._L6F] (Node ffff8804854732d0), AE_NOT_FOUND (20150410/psparse-536)
Jan 24 21:20:34 Prime kernel: ACPI Exception: AE_NOT_FOUND, while evaluating GPE method [_L6F] (20150410/evgpe-592)

 

Now this didnt effect the unraid booting. However the error messages keep getting generated constantly, even after the initial burst of them at boot up. So this filled my syslog and caused my cpu to be using 15% at idle when running unraid.

This problem seems to be most likely a bios error which hopefully will be addressed by asrock. Also it may possibly be because skylake support was only put into linux kernal 4.3. and when unraid kernal is updated maybe this error will not happen.

Anyway I tried disabling the acpi in my sysconfig and surely the error didnt occur. But disabling the acpi disables iommu so this wasnt an acceptable solution. so i tried acpi=ht which as i found on https://01.org/linux-acpi/documentation/debug-how-isolate-linux-acpi-issues

acpi=ht

the most like "acpi=off", disables all of ACPI except what is needed to enumerate processors.

If acpi=off works and acpi=ht fails, then the issue is in the ACPI table parsing code itself, or perhaps the SMP code.

using acpi=ht had no effect so this seemed to indicate the above.

My problem had been reported on https://bugzilla.kernel.org/show_bug.cgi?id=105491

The solution was to disable the gpe6f

You do this with this command

echo disable > /sys/firmware/acpi/interrupts/gpe6F

You add it to your go file. Or just to test ssh or telnet to your sever and run it there.

 

Anyway i have everything running perfectly now, but anyone thing of a skylake cpu and a z170 based asrock motherboard this info may help you.

 

Link to comment

I wonder if these are AsRock related or Z170 issues => I suspect they may be Z170.

 

The best choice for a system that you want to pass a LOT of devices through to VM's is to use an E5 series Xeon.    The socket 2011 server-class boards that support the E5's have far more PCIe lanes and much better IOMMU support.

 

 

Link to comment

All desktop i3 chips have had Hyperthreading since at least Sandy Bridge.

 

Generally for desktops:

 

Celeron: Dual core with limited cache

Pentium: Dual core

i3: Dual core with HT

i5: Quad core with turbo

i7: Quad core with HT and turbo.

 

There are exceptions, particularly when Mobile and Embedded chips are involved.

Link to comment

I'll just add, that even some skylake i3s now have vt-d support, so the price point is getting lower if you don't mind a dual core (with vt-d) support.

 

Well holy crap! That's awesome!

I'm curious, what features of skylake are attractive to an UnRAID build?  This is an honest question because I know very little about CPUs down at this level of detail.

Link to comment

Vt-d and ECC support by the way.

OK thanks.  Some quick research tells me that Vt-d does some magic to make VMs better.  VMs are one of the things I want to play around with soon, but currently have no particular use for.  So all Skylake processors have this technology?  What about the previous generation?  Does any motherboard that supports Skylake processors also support Vt-d or should I be on the lookout for that?

 

ECC (Error Correcting Code) for RAM on the other hand is very cool but seems to only be available for Xeon processors and that is out of my price range.

Link to comment

ECC (Error Correcting Code) for RAM on the other hand is very cool but seems to only be available for Xeon processors and that is out of my price range.

 

Don't be so sure of that.  You simply need a server-class motherboard [e.g. http://www.newegg.com/Product/Product.aspx?Item=N82E16813182999 ]

 

... and if you want to stay on the low-end processor-wise you could use a Skylake i3  (the i5's and i7's don't support ECC).  An i3-6100 is very close in performance to the i5-6400 you were looking at [PassMark 5512 vs. 6525]

 

... or you could go with a Xeon 1200 v5 series CPU for higher performance (although these will add to your cost)

 

Link to comment
  • 3 weeks later...

VT-d has been available on garden variety i5 non-Ks since Sandybridge at least. I think starting with i5 4000 series at least some of the K models also had it. Skylake is even better here than you guys are letting on though. The lowly Pentium G4400 and a Skylake celeron no one can actually find even have it. I guess Intel decided not to be so greedy this go around with at least one feature.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.