Jessie Posted November 11, 2020 Share Posted November 11, 2020 Has anyone built unraid on a x570 platform with 2 gpu's 2 vms etc running simultaneously? Quote Link to comment
Timothyy Posted November 11, 2020 Share Posted November 11, 2020 (edited) Yes I have this working, trying to get a 3rd VM working but running into issues. 2 VMs with passthrough actually perform very well simultaneously in my opinion (4 isolated cores, 4 HTs (makes 8 total) and 16gb of reserved memory per VM). Have it running for a couple of months now daily and did not encounter much problems. Guess I had 2 or 3 incidental freezes and can not really remember it crashing completely. Problems for me start at the point I am trying to use a 3rd gaming vm with passthrough. I am updating the bios to version P3.61 on short notice. I don't use any other things on my rig currently. Motherboard: AsRock x570 Taichi, bios version P3.40 CPU: AMD Ryzen 9 3950x Memory: 128gb DDR4 (4x 32gb Samsung M378A4G43MB1-CTD) GPU: 3x GTX 1080 Ti gaming X 11gb Edited November 11, 2020 by Timothyy Quote Link to comment
Jessie Posted November 12, 2020 Author Share Posted November 12, 2020 (edited) 9 hours ago, Timothyy said: Yes I have this working, trying to get a 3rd VM working but running into issues. 2 VMs with passthrough actually perform very well simultaneously in my opinion (4 isolated cores, 4 HTs (makes 8 total) and 16gb of reserved memory per VM). Have it running for a couple of months now daily and did not encounter much problems. Guess I had 2 or 3 incidental freezes and can not really remember it crashing completely. Problems for me start at the point I am trying to use a 3rd gaming vm with passthrough. I am updating the bios to version P3.61 on short notice. I don't use any other things on my rig currently. Motherboard: AsRock x570 Taichi, bios version P3.40 CPU: AMD Ryzen 9 3950x Memory: 128gb DDR4 (4x 32gb Samsung M378A4G43MB1-CTD) GPU: 3x GTX 1080 Ti gaming X 11gb Are the graphic cards identical? In the old days, the graphic cards had to be different. Edited November 12, 2020 by Jessie Quote Link to comment
Jessie Posted December 15, 2020 Author Share Posted December 15, 2020 (edited) I'll answer my own question in case someone else encounters the same problem. Recently, I built a machine on a Gigabyte X570 gaming x platform and a Ryzen 7 3770x. On this board, you can change a setting in BIOS and the IOMMU groups pretty well work in unraid for usb/gpu passthrough without the need to set ACS override in unraid. It worked flawlessly. Then I tried a Gigabyte B550 Gaming X motherboard with the same processor and tried to pass through a Gigabyte GT130 GPU (No fans) and a gpu bios dump from tech powerup. Black screen. I've built a few machines using this GPU and BIOS dump without problems on series 2 ryzen and intel boards. So I blamed the B550 board. Then I built a machine on an x570 board. Black screen again. So I dumped the BIOS from the actual card as per Spaceinvaders tutorial and it worked. I then added another GT130 card and was able to create a second Windows vm complete with keyboards/ mice and usb passthrough. I am assuming the B550 failed for the same reason. I will do the experiment in the near future. Not really sure why it didn't work. on the series 3 hardware but did on the second gen servers. Maybe the firmware changed in the GT130 card. Edited December 15, 2020 by Jessie Quote Link to comment
Michelle Bausager Posted January 9, 2021 Share Posted January 9, 2021 On 12/15/2020 at 10:12 AM, Jessie said: I will do the experiment in the near future. I'm looking at the B550 too, I usally use a VM as my daily driver, but heard a lot of people criticizing the B550 for not beening able to passthrough. Would love to hear what you recomend :) The B550 or x570 c: Quote Link to comment
Jessie Posted January 9, 2021 Author Share Posted January 9, 2021 1 hour ago, Michelle Bausager said: I'm looking at the B550 too, I usally use a VM as my daily driver, but heard a lot of people criticizing the B550 for not beening able to passthrough. Would love to hear what you recomend The B550 or x570 😄 I've built quite a few Unraid systems over the past few years. I recently went to Ryzen. The first Ryzen3 was on a X570 platform (Gigabyte X570 Gaming X) Brilliant. You can change a setting in Cmos and it optimises the IOMMU allocations for Unraid passthrough. The next one I built was on a B550 platform. (Gigabyte B550 Gaming X) I went that way because the requirements of this machine didn't warrant an X570 and the B550's have a lower cost. I had trouble passing through a single Nvidia GT1030. I had previously followed Spaceinvaders methods for modifying a downloaded BIOS image, but this didn't work. In the end I resorted to the old method which was to install a cheap graphics card to boot unraid and then I passed the vm through to the second GPU. The problem with that was that I lost a full sized PCIe slot that could have been used for something else. Then I built another machine on a X570 and had the same problem. This machine had the same hardware specs as the first one, which worked straight off. I then decided to extract the BIOS from the same card and it worked. I suspect there might have been a firmware change in the graphic card which made the bios image incompatible. So with this new found knowledge, I returned to the B550 machine, extracted the BIOS and it now works perfectly on a single card. BIOS extraction was done using the GPU as a secondary card and command lines. Spaceinvader has now released a script which will do all the hard work for you. You don't even require the second GPU for the extraction. He is a very clever man. (If you don't know who he is, go to youtube and discover his brilliant tutorials on unraid, pfsense and others) So far I have been using the Ryzen 7 3700x and previously 2700 processors. I chose the 3700x over the 3800 because it draws less power and isn't much slower than the 3800. In the next couple of weeks I will build one on a B550 and a Ryzen r5 3600?? The r5 has 12 threads vs 16. 12 will be enough for the anticipated needs of its user. For the near future, I will build on either of these platforms depending on their anticipated workload. To date, I have been holding off going to 6.9 until it went mainstream. I changed my mind last week when chasing a problem on my current machine. 6.9 rc2 seems very stable and the features are too good to go back again. B550 Gaming X has 2 x m.2 slots and 4 Sata ports, so using 4tb drives you can have 12tb with parity plus 2 m.2 cache drives. X570 Gaming X has 2 x .m2 slots and 6 Sata ports, so using 4tb drives you can have up to 20tb with parity plus 2 x m.2 cache drives or with v6.9 12tb and 4 cache drives. I used to use coolermaster cases (Mainly silencio 550's) I feel the new generation doesn't suit unraid. So I have opted for Phantecs P400s cases. There is a gaming version of the P400 but I mainly build these machines for business so "light shows" are not really appropriate. I like the phantec case design. They provide for 2 sata ssd drives, 2 x 3.5" bays and then you can install up to 4 extra 3.5 bays to provision the max number of drives for the unraid setup. They have plenty of room behind the motherboard to hide the cables. A typical spec might use one of the above motherboards with a Ryzen 7 3700x, 32 or 16gb ddr4 3200 ram, gigabyte or asus GT-1030 graphics cards (no fans). possibly an addon pcie usb card for extra passed through usb slots, a pcie lan adapter if the machine is going to run a pfsense firewall vm. Then it will typically run at least 1 operating system. (usually win 10) In the background, a nextcloud / Collabora instance, maybe a pfsense vm, maybe a open vpn docker so they can securely access their system remotely and then the sky seems to be the limit for other docker applications. Quote Link to comment
Kich902 Posted March 26, 2021 Share Posted March 26, 2021 (edited) On 11/11/2020 at 6:21 PM, Timothyy said: Yes I have this working, trying to get a 3rd VM working but running into issues. 2 VMs with passthrough actually perform very well simultaneously in my opinion (4 isolated cores, 4 HTs (makes 8 total) and 16gb of reserved memory per VM). Have it running for a couple of months now daily and did not encounter much problems. Guess I had 2 or 3 incidental freezes and can not really remember it crashing completely. Problems for me start at the point I am trying to use a 3rd gaming vm with passthrough. I am updating the bios to version P3.61 on short notice. I don't use any other things on my rig currently. Motherboard: AsRock x570 Taichi, bios version P3.40 CPU: AMD Ryzen 9 3950x Memory: 128gb DDR4 (4x 32gb Samsung M378A4G43MB1-CTD) GPU: 3x GTX 1080 Ti gaming X 11gb Planning on doing the same. Could you give us ur complete parts list? I intend on using: Processor: AMD Ryzen 9 3900X with a be quiet! Dark Rock Pro 4 MB: Asus TUF GAMING X570-PLUS (WI-FI) RAM: Corsair Vengeance LPX 32 GB (4 x 8 GB) DDR4-3200 GPU: 2x RTX 3070 8 GB Founders Edition PSU: Corsair RMx (2018) 850 W 80+ Gold Case: Lian Li LANCOOL II-X Your thoughts/advice on the above parts will be much appreciated☺️ Edited March 26, 2021 by Kich902 Quote Link to comment
Paul_Ber Posted June 13, 2021 Share Posted June 13, 2021 (edited) I am looking to do the same thing. Need 2 GPU and 1 HBA 9211-8i. Do I need a PCIe x1 GPU(I see some GT710 x1 ones on Amazon) for unRaid to boot into so I still had the 3 full size slots? I looked at the ASUS AMD AM4 Pro WS X570-Ace MB but don't know how good the IOMMU groups are like. $470 CAD 3 PCIe 4. 0 x16 (3-Way x8/x8/x8), 4 SATA ports. I looked at the AsRock X570 Creator but it is really expensive at $662 CAD. AQUANTIA 10G LAN, Intel Gigabit LAN, 3 PCIe 4.0 x16(x8,x8,x4), 3 PCIe 2.0 x1, 8 SATA3, 1 Hyper M.2 (PCIe Gen4 x4 & SATA3), 1 Hyper M.2 (PCIe Gen4 x4) Looking at AsRock x570 Taichi $440 CAD ASRock X570 TAICHI RAZER Edition $570 CAD same as regular Taichi but has 2.5G LAN, wifi(not needed), BT(not needed) Other bit unsure is ECC memory really needed? I use ECC in my current system. It just gets more confusing as to how they describe ECC compatibility. I found this on an ASUS but probably only applies to ASUS: Edited June 13, 2021 by Paul_Ber Quote Link to comment
Paul_Ber Posted June 15, 2021 Share Posted June 15, 2021 Doesn't the bottom PCIe slot go thru the chipset? Probably yes and probably why you cannot pass 3rd GPU. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.