Hardware recommendations for unRAID servers with VM's


RobJ

Recommended Posts

This is an attempt to collect in one place recommended hardware for VM applications.  There are numerous discussions scattered here and there about the pros and cons of various components.  Collecting it in one place seems like a good idea, *if* users buy into it and contribute.  I'd like to see posts on whole setups that work, as well as posts on a single component that the user has determined works well.

 

For consistency, easier comparison and readability, I'm going to make some simple style and content recommendations, but they are just to get the ball rolling, can change easily, may evolve with time.  I don't want anyone to ignore this just because they don't like the way the info is laid out.

 

Info I would like to see is:

- the item name or list of item names - preferably one per line (bolding each is suggested)

- the best usage for the item or items - a 720 and a 1070 can be equally excellent, for very different usages

- the user's rating of the item or items (suggested: Excellent, Good, Fair, Don't bother) - rating could be for a system as a whole or for each item; could also add note about how well it works for the user's own purposes

- what settings, modules, and tweaks are needed for setup and best performance - obviously this could be the longest by far; but what a user really wants to know is what will it take to make this component work for them as stable and bulletproof as possible, with reliably great performance

- any special notes and comments, and links to actual products (on Newegg, Amazon, eBay, etc)

 

An Example: (this info isn't real!)

RX480

Usage: Light to moderate gaming, any general purpose use

Rating: Excellent; perfect for my Plex and Far Cry

Instructions: use Seabios, i440fx-2.5, do this, do that, see <link> for more info, etc

Note: search eBay, often find used for under $100

 

I'm hoping users with great working systems will contribute here, so any user can search this for the best GPU's for what they need, or the best CPU and motherboard combos for what they want to do, or the best USB cards for passthrough, or whatever other common needs VM users have.

 

I do not feel the need for a list of what *does not* work, or doesn't work without headaches, just what works well, and can be recommended to others.  In other words, we don't need anything rated 'Fair' or 'Don't bother'.

 

Please, no support requests or questions here (like 'would this work?'), I'll probably delete or move them.

 

Edit: we're off to a great start!  nice contribution from 1812!

 


Index to Recommendations


Graphics

USB

Network

Motherboard

 

Edited by RobJ
add mb&gpu
  • Upvote 1
Link to comment

like this?

 

Fresco Logic FL1100 USB 3.0 Host Controller (sold as Inatek KT4006)

Usage: usb disk access/usb3 to ethernet adapter/keyboard-mouse/bluetooth dongles

Rating:Good+, good bandwidth with on card, no aux. power cable needed for card, 1 out of 5 I ordered died within 2 weeks

Instructions: for OS X-plug and play though Sierra 10.12.2, windows not tested but manufacture claims windows 7+ compatibility (come with disc of drivers for windows)

 

 

666172-001 MNPA19-XTR HP 10GB Ethernet Network Card (Mellanox ConnectX-2 EN)

Usage: Moderate to heavy capable

Rating: Excellent/Good if you need to cheaply move data faster than gigabit

Instructions: no OS X support, win10 plug and play but works better with Mellanox drivers (downloadable from their website,) native unRaid support for server to the use card in 6.2.4 (possibly earlier,) I currently use a pair to backup one server to the other.

 

 

Geforce GT 710

Usage: good cheap general desktop card, light gaming, watching movies

Rating: Good

Instructions: Plug and play in OS X through Sierra 10.12.2 as long as smbios set to 14,1 or 14,2 before assigning the card (otherwise black screen occurs,)  did not try to give it nvidia drivers

 

 

Geforce GT 730

Usage: good cheap desktop card, light gaming, watching movies

Rating: Good

Instructions: Plug and play in OS X through Sierra 10.12.2 as long as smbios set to 14,1 or 14,2 before assigning the card (otherwise black screen occurs,)  did not try to give it nvidia drivers

 

 

GTX 760 SC

Usage: mild video editing in fcpx, 3 simultaneous screes utilized

Rating: Good+

Instruction: Plug and play in OS X through Sierra 10.12.2 as long as smbios set to 14,1 or 14,2 before assigning the card (otherwise black screen occurs,) using Nvidia web drivers can be a challenge as incremental updates to OS X can cause the drivers to no longer work with the card.

 

Link to comment

hi, my experience so far:

 

Zotac 1060 3gb mini

Usage: high settings gaming, folding, and any general purpose. It's small length so usable in small cases (although it still uses 2 slots)

Rating: Excellent

Instructions: was able to properly get the rom following the video/written guides. tested perfectly in all slots (including slot 1). confirmed with seabios and ovmf.

Note: did not test in Windows OS, but tried many linux OS's (fedora 25, ubuntu 16.04 - gnome, kubuntu, etc... mint) with great success.

 

interested also for Motherboards?

-d

 

edit: Installed last night Win 10 pro with 1060 passthrough. Installed the nvidia latest drivers. All went smooth, no issues detected so far.

 

Link to comment

motherboard

 

Asrock Fatal1ty X99M Killer

 

Usage  To upto 3 vms at once with separate gpu pass through.

Plus points Great x99 board iommu groups are separated well. No need for acs override.

Minus points Onboard usb controller will not passthough. You will need a separate pcie usb

if you want to pass through a controller. No onboard gpu.

 

GPU

 

MSI  1070  Seahawk

zotac 1070  Founders

Usage  fedora  Windows gaming VMS

Plus points Easy to passthrough with great performance

minus points As with all Nvidia will not work as primary gpu (with no onboard graphics) without a rom dump.

                        Doesn't work in osx

 

GPU

 

MSI GTX 750

 

Usage Fedora Libre elec windows osx

Plus points Easy to passthrough  and works with windows linux and osx

minus points As with all nvidia will not work as primary gpu (with no onboard graphics) without a rom dump.

 

 

 

 

Link to comment
  • 1 month later...

Gigabyte X99 Designare EX

 

Usage:  multi-user VR experiences

Plus points:  

  Plenty of data lanes for multiple GPUs and PCIe devices

  Good power distribution

  Mechanically sound solution for PCIe slots

  Alpine ridge with Thunderbolt and USB3.1 performs well when passed through to VM

  Plenty of connectors and USB3 ports

Minus points:

  Renesys USB hubs don't do passthru very well

  PCIe endpoint 02 is overloaded

  Requires a monitor connected to boot.  Doesn't like to let go of primary GPU (still WIP for me)

GPU: MSI 1070 Quicksilver

Usage: VR gaming GPU

Plus points: 

  Easy installation

  No SW problems

  Can use LED SW to indicate which VMs are located and active on each GPU

  Great cooling and silent operation under load

Minus points:

  Only 1 HDMI out, no VGA (but who does nowadays?)

  No way to get at the covered PCIe slot (even with thin x1 ribbon cable)

 

Link to comment
  • 2 months later...

Asus X99-WS IPMI

 

Usage: UnRaid with plenty of dockers, and single VM. 

Plus points:

Onboard VGA

Able to manage server remotely through browser through IPMI interface (useful if your server, like mine, is hidden away in a cupboard without easy access)

Plenty of SATA ports

5 PCIe ports

Board is well laid out and easy to work with. 

Able to passthrough one of the USB controllers on board without having to fiddle with BIOS settings

Two ethernet ports (I haven't tried passing one directly to a VM). 

Minus points:

Expensive!

If you update the BIOS all setting return to default. 

 

Inno3D - GeForce GTX 1070 8GB iChill X3 Video Card

 

Usage: Gaming on Windows 10 VM

Plus points: 

Easy to passthrough, no issues. 

Minus points:

Very big card, takes up a lot of space. 

Link to comment
  • 8 months later...

USB Card roundup for Windows 10 and High Sierra Virtual Machine Use

 

1st Card - AUKEY 4 Ports PCI-E To USB 3.0 Expansion Card

Chipset: VL805

Findings:  Got an error in my syslog during boot and card was unable to be used either natively in Linux or passed through to Windows or OSX VM

 

2nd Card - Anker Uspeed PCI-E to USB 3.0 2-Port Express Card

Chipset: VL805

Findings:  Got an error in my syslog during boot and card was unable to be used either natively in Linux or passed through to Windows or OSX VM

 

3rd Card - Sienoc USB3.0 7Port PCI Express Card

Chipset: - Renesas D720201

Findings: Worked natively in Linux or passed through to Windows, but not detected by OSX VM

 

4th Card - Inateck KTU3FR-5O2I

Chipset: Fresco FL1100

Findings:  Worked natively in Linux, and worked beautifully in Windows or OSX VMs, able to use my hardware KVM switch with either, but unable to power my server off with the card plugged in, as soon as it powered off, it powered on again.  @aptalca has had the same issue with a Fresco FL1100 card and we both have Supermicro motherboards.

 

5th Card - ORICO USB3.0 4 Port PCI Express to USB3.0 Host Controller Card (PME-4U)

Chipset: Fresco FL1100

Findings:  Worked natively in Linux, and worked beautifully in Windows or OSX VMs, able to use my hardware KVM switch with either, but unable to power my server off with the card plugged in, as soon as it powered off, it powered on again.  @aptalca has had the same issue with a Fresco FL1100 card and we both have Supermicro motherboards.  I ordered this one to try and exclude the possibility it was an inherent Inateck fault, but after testing this one, I think we can conclude there is an issue with the FL1100 chipset and Supermicro boards.

 

6th Card - KALEA-INFORMATIQUE Controller Card PCI Express (PCI-E) to 3.1 USB

Chipset: Asmedia ASM1142

Findings:  Success!!  Works in Windows 10, works in OSX natively both, without installing extra drivers, allows my server to powerdown and stay down.

 

Other possibilities:  Well I'm running out of chipsets to try, only thing I could consider is an expensive Sonnet card which uses the Fresco FL1100 chipset but had four controllers on the card which can in theory be split amongst VMs so one controller to each VM.  I don't hold out much hope this will actually work well given it's got four of the damned chips that won't let me powerdown in the first place. :o

 

One other thing I did try was using the Renesas based chipset to boot from and passed through my native USB ports, that didn't work well.  Not sure why, but I might investigate that a bit more if I have no other options.

 

Conclusion

If youi've got a Supermicro motherboard, then Asmedia ASM1142 based USB cards are the way to go, for any other motherboard, then either the Asmedia ASM1142 chipset or the more common Fresco Logic FL1100 based cards would be fine, but if you notice that you have issues with powering down, the card may well be the culprit and try an Asmedia card instead.

 

It's been..... emotional

Edited by CHBMB
  • Like 6
  • Upvote 1
Link to comment
  • 1 month later...
  • 2 months later...

ASUS ROG STRIX ZE70-F Gaming

Usage: Heavy gaming, VR, general use.

 

Rating: Good. Low number of USB ports makes a PCI-e USB card a must for peripherals. UEFI refuses use of onboard graphics when a GPU is plugged in even though you can select it (resets to auto after selecting onboard.) Use of onboard audio is not possible by default due to it being shared in an IOMMU group with a lot of items that don't have the necessary ID assignments. VT-d enable is hidden deep in the UEFI and requires some exploring to locate it.
 

Instructions:

Video output: Use the cheapest GPU you can find in the top slot.

Audio output: A sound card is the best option. If you don't have one get the cheapest thing you can find. Alternatively you can use the HDMI/DisplayPort audio output of your graphics card if you have audio jacks or speakers available in your monitor.

VT-d: <to be edited with direct location of VT-d option and how to get there>

 

Note: Do NOT pass through "ASUSTek Computer", this causes an issue that freezes up your computer for a second at frequent intervals (similar to BSOD freeze but recovers.)

 

------------------------

 

Innatek KT5001

Usage: Gaming, VR, general use.

 

Rating: Excellent. Oculus seems to think that a driver update is needed but have had no issues thus far. Works exactly as you'd hope for plug-n-play when passed through.

 

Instructions: see this thread.

 

Note: Requires SATA power supply, comes with molex to SATA adapter.

 

------------------------

 

Gigabyte GeForce 210

Usage: Dedicated unRAID video output.

 

Rating: Excellent. Lowest-price graphics card I could find with an HDMI output.

 

Instructions: Install into top PCI-e x16 slot, assign VMs to your beefier graphics card.

 

------------------------

 

Creative Labs Sound Blaster Audigy FX

Usage: Integrated audio alternative.

 

Rating: Excellent. Lowest-price sound card I could find.

 

Instructions: Select from sound card dropdown in unRAID GUI, Windows 10 will automatically install necessary driver.

 

Note: When using headphones with inline volume control, if you hear a buzzing sound simply turn down the volume on the inline control and turn up the volume in Windows. The buzzing will cease.

Link to comment
  • 1 month later...

Gigabyte GA-AX370-Gaming K5

Usage : General Use, Virtualization, Light software development, Gaming.

Rating : Awesome, IOMMU groups are pretty well placed and its easy to perform PCI passthrough with VFIO driver, great UEFI, a lot of SATA ports, 3x PCI-e x16 slots.

Intel PRO/1000 PT Quad Port Server Adapter

Usage : Dedicated Network Controller for Docker & KVM

Rating : Great piece of hardware, really holds strong even during most difficult loads (4 users can hold speed of 100 mbps while copying mixture of huge & small files)

Sapphire Radeon RX 580 8 GB Nitro+

Usage : Gaming, video acceleration in certain software

Rating : Flawless, drivers work like charm, games are running pretty well with NVIDIA's adware (named Gameworks), software uses it to full potential. Only problem is when I shutdown Windows 10 VM, fans go full speed and I have to turn on Linux VM and shut that VM down so fans go back to normal speed.

 

All this hadware is running without problem in OVMF based VM with i440fx 2.11

Link to comment
  • 3 months later...

CPU Ryzen x1800

32GB Ram

Motherboard: Asrock Itachi X370

Usage : 3x VM's, CCTV recording 24/7, Plex dockers etc

Gfx Card 1: Sapphire Radeon WX4100 (trippe screen in windows VM)

Gfx Card 2: Nvidia 1030GT (Ubuntu VM)

 

Note: No issues, the WX4100 was a little tempermental when installing the AMD drivers but leaving with stock windows drivers no issues whatsoever. Althoug i expect the AMD drivers will be fine.

 

Edited by Greygoose
Link to comment
  • 4 weeks later...
On 2/2/2018 at 1:28 PM, CHBMB said:

 

@aptalca

6th Card - KALEA-INFORMATIQUE Controller Card PCI Express (PCI-E) to 3.1 USB

Chipset: Asmedia ASM1142

Findings:  Success!!  Works in Windows 10, works in OSX natively both, without installing extra drivers, allows my server to powerdown and stay down.

(...) 

 if you notice that you have issues with powering down, the card may well be the culprit and try an Asmedia card instead.

Was using a Totovin card in my Supermicro which worked until last update round - - - now the machine refuses to stay powered down.  It uses both an ASM1142 and ASM1543. B074QK68XS

 

Just ordered a Qnine to replace it with as it only uses a asm1142.  Gosh I hope this solves the issue.  B07BJ2HVD2

 

Edit:  powerdown functions as expected with the Qnine USB controller that has only the ASM 1142

Edited by landS
Link to comment

Supermicro - X10SRA-F

CPU: Intel® Xeon® CPU E5-1650 v3 @ 3.50GHz

32 GB Multi-bit ECC

 

2 threads Unraid

2 threads Docker/Crashplan

2 threads Windows 10 

* Use Workstation, 3D modeling, Printer Slicing, Owned media backup, Etc

* ZOTAC GeForce GTX 1060 Mini, ZT-P10610A-10L - N82E16814500405

* ASM 1062 Sata for BD ODD (bootmenu stub required: iommu=pt vfio-pci.ids=1b21:0612,1b21:1242)- B00952N2DQ

* ASM 1142 USB 3.1 (iommu=pt vfio-pci.ids=1b21:0612,1b21:1242) - B07BJ2HVD2

* Logitech Unifying Receiver 

* 8 GB ram, i440fx-2.11, OVMF 

6 threads Windows 10

* Use: Steamlink host

* ZOTAC GeForce GTX 1080 Mini 8GB, ZT-P10800H-10P - B01NCVICRY

* sena ud100-g03 bluetooth class-1 (for Wii controllers in Dolphin as Steamlink direct is too much data) 

* Valve Software Steam Controller (for local play)

* 8 GB ram, i440fx-3.0, OVMF 

 

Great IOMMU groups, easy GPU and USB Device passthrough, driver support, etc.  

Very temperamental for non-gpu PCIe passthrough selection. 

Edited by landS
Link to comment
  • 1 month later...

StarTech.com 4 Port PCI Express (PCIe) SuperSpeed USB 3.0 Card Adapter w/ 2 Dedicated 5Gbps Channels

Chipset: Renesas/NEC - µPD720202

Findings: Purchased this because it had 4 individual USB controllers on the board thinking the extra price would allow for more flexibility with my VMs.  Requires SATA/Molex power, works fine when booting native to Windows 10.  It is recognized in Unraid under Settings -> Hardware however when passing through the PCIE card to a VM, while the card is detected, it always has an error with Code=10.  This is with the standard Windows drivers and also with the drivers from the manufacturer's website.  There are other posts online with the same result (Code=10 error in Device Manager) but not promising workarounds.  I suspect this is something at the Linux level as when I do not pass through the PCIE card and hook up USB devices directly to the card, they also are not recognized by Unraid.  I give up on this card.

  • Like 2
Link to comment

motherboard

 

Asrock TaiChi x399

 

Usage  To up to 4 vms at once with separate gpu pass through.  Great for medium/heavy gaming.

Enough slots and ports to get 4 GPUs, 2 m.2 NVMe, and 1 U.2 port for PLX bridge to extra 4x slot

Plus points Great x99 board iommu groups are separated well.

Minus points Onboard usb controller will not passthough. You will need a separate pcie usb

if you want to pass through a controller. No onboard gpu.

 

GPU

 

EVGA GTX960 SSC

Easy passthrough 

 

GPU

 

EVGA GTX960 FTW

Easy passthrough

 

USB

 

Sonnet Allegro Pro USB 3.0 card

Great easy passthrough of all 4 controllers- works off PLX bridge with U.2 port

  • Like 1
Link to comment
  • 3 weeks later...

Asus x99 Strix ROG Gaming

8 dimm slots, 3 video cards (two dual slot), onboard buttons

 

Xeon E5-2673 v4

20 core / 40 thread

 

Thermaltake Floe Ring 360

Fits great

 

Thermaltake 1050 iRGB PSU

Plenty of power

 

G.Skill DDR4 2666 64GB

Using 4 16GB kits no issues (I know its not officially supported by G.Skill)

 

ATI Radeon X600

Cheap vid card for Unraid to grab for admin console

 

EVGA GTX760sc x2

I have one on each VM, one is a Win 10 and other OSX

 

Inateck 5 port PCIe USB 3 card

Pass though to OSX no issues, no driver or kernel issues, no special process just click the option in the VM settings tab and boom it works!

 

Sabrent USB sound card x2

I have one on each VM, again easy pass though and no driver/kernel issues

 

I have this set up as a 40TB server doing many things, along with running 2 VM desktops with GPU passthough. Each VM is 4 core 8 thread, 16gb ram and 1TB hdd. One is running Win 10 with dual users and other is OSX dual user. They are used as normal desktop stuff, slight gaming and have had no issues thus far, my OSX desktop has full control of the 5 port USB card, this way I do not need to pass anything though. I simply plug what USB items I want to use in the OSX VM and it shows up. Everything listed above works natively and did not require anything special to be done to make it work.

 

***Update***

Now have twin Strix GTX 970s and twin 5 ports cards installed.

3 VMs One controls RGB crap, one for desktop and one for HTPC running over IP hdmi extender with usb to other room

Edited by KLAK
  • Like 1
Link to comment
  • 3 months later...

Gigabyte Technology Co., Ltd. AX370-Gaming 5

American Megatrends Inc. Version F25. Dated: 01/16/2019

AMD Ryzen 7 2700 Eight-Core @ 3200 MHz

32gb DDR4 Ram

Gigabyte GTX 1030 2GB Low Profile (N0 Fans)

(Rom Bios downloaded and modified as per Spaceinvader 1 tutorial on GPU passthrough with only 1  graphics card as primary)

4 port USB card for passthrough to VM

(USB card passed through to Windows VM using Virtmanager in a FreeBSD vm) 

4tb WD Red Parity

1 x 4tb WD Red Array

6 x 2tb WD green Array

2 x Samsung evo 860 500gb Cache

Adaptec 1430SA sata controller

Fractal Design Case

Coolermaster 650w Modular power supply

 

This machine upgrades my 10yo i3 unraid server which was primarily used as a file server.

From the old machine I reused the adaptec sata card and a few 2tb hard disks. 

 

During the build I had issues with GPU passthrough.  I had upgraded the BIOS to F31.  Turns out F30 and F31 make provision for the Ryzen 3 CPU and kills off Pass through.  I reloaded the original BIOS (F23) and it worked.  Settled on F25 which was the last one before the BIOS went to the F30 series.

 

IOMMU configured as per Spaceinvader breaking up IOMMU groups option 3.

 

The Fractal Design case will hold 8 x 3.5" drives.  I put a 2 drive hotswap bay in one of the 5 1/4 slots for the SSD's and a single 3.5" hotswap in the other 5 1/4 bay. Will use this to connect unassigned devices.

 

The motherboard has 8 sata ports and I get another 4 ports from the adaptec card.   Adaptec card is old but it works.

 

Edited by Jessie
Link to comment

ASUSTeK COMPUTER INC. PRIME X370-PRO

BIOS American Megatrends Inc. Version 4207. Dated: 12/08/2018

AMD Ryzen 7 2700 Eight-Core @ 3200 MHz

16gb DDR4 ram

Gigabyte GTX 1030 2GB Low Profile (N0 Fans)

Renasas USB controller passed through to the VM

ASUS gt710 primary GPU (no fans)

Samsung evo 860 500gb ssd cache

Realtec Network controller (2nd lan port)

Wd red 4tb (Parity)

WD red 4tb Array

WD re4 1tb array

WD green 2tb Array

 

Machine runs ok but vm operates "Jerky" occasionally.  System needs a restart to rectify. Rest of machine seems ok. Suspect heat issue With GTX 1030 GPU.  Still investigating.

Link to comment
  • 5 months later...

Did you manage to get things up and running smoothly?

I have a system that is also equipped with this ASUS mainboard and a GT710 gpu board, but with a Ryzen 3 1300X.

I can't get it running. I works for about an hour. Sometimes less, sometimes more, and then it freezes. I checked everything I could, the memory test was ok, I upgraded the BIOS, nothing helps. I hope you could provide me with some tips.

Link to comment
On 12/3/2019 at 11:54 PM, jonathanm said:

Early Ryzen's need C-states disabled or power modified in BIOS.

I have been looking for 'C-states' in the BIOS and haven't found it. I tried several settings, but nothing works. I was hoping that, because Jessie has the same mainboard, he could point out what to change.

And what do you mean by 'power modified'?

Will a different processor help? I am willing to spend some more money if it is absolutely necessary, but only if these problems go away, obviously.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.