ufopinball

In general I don't have freezing issues. So long as the Windows 10 VM is running, the server stays happy. I've had uptimes in excess of a month, only ended because I wanted to do the BIOS upgrade. Thanks for the tip, though ... I'll try the latest RC builds once I have a chance to experiment with all the new features on a test system. Regarding your GPU pass-through, I'll attach a screen shot of my working VM with GPU passthrough, as well as a copy of the XML. I'm presently running OVMF, because I thought SeaBios was less advanced? This could be your problem, what happens when you run OVMF? Otherwise, I have no experience with NVIDIA graphics cards, nor dumping BIOS. I figure we should at least be able to get your Radeon GPU running. I'm also not seeing the failure in the SYSLOG that you're seeing, but hopefully that goes away if we can figure this out. Take a look at my attachments and let me know what you think... PS: If it matters, this Windows 10 VM is about 9-10 months old, I think? Win10e.xml

Post your build details (CPU, Mobo, RAM, etc.) I have this currently working on my Ryzen 1800X w/ ASUS Prime X370-PRO motherboard. My GPU is only a Radeon 6450 desktop graphics card. Nothing fancy, but the GPU pass-through to my Windows 10 VM works nicely.

Over the weekend, I finally got around to upgrading my ASUS Prime X370-PRO motherboard to the latest BIOS (3402). So far, my uptime is 3+ days with no crashes. In general, I have not had a crashing system, primarily because I run a Windows 10 VM all the time with half the cores/threads dedicated to it from my Ryzen 1800X. I guess that's enough to keep the crash/hang problem at bay. The one thing I noticed about BIOS 3402 is the improved IOMMU groupings available. This is what the IOMMU groups look like without the ACS override enabled: IOMMU group 0 [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 1 [1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 2 [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 3 [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 4 [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 IOMMU group 5 [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 6 [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 7 [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 8 [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452 IOMMU group 9 [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 IOMMU group 10 [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 11 [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467 IOMMU group 12 [1022:43b9] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02) [1022:43b5] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02) [1022:43b0] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02) [1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) [1000:0072] 03:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03) [1b21:1343] 04:00.0 USB controller: ASMedia Technology Inc. Device 1343 [8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 13 [1002:677b] 07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos PRO [Radeon HD 7450] [1002:aa98] 07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 14 [1022:145a] 08:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 15 [1022:1456] 08:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456 IOMMU group 16 [1022:145c] 08:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c IOMMU group 17 [1022:1455] 09:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 18 [1022:7901] 09:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 19 [1022:1457] 09:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457 Previously, I had to use the ACS override to break up the IOMMU groups so I could properly pass my GPU, sound card and a USB controller in to my Windows 10 VM. With BIOS 3402, I no longer need to do that. Here's a screen shot of my VM pass-through settings. Anyway, just really glad AMD/ASUS are working to address the concerns for us VM users. I know it was mentioned during an AMD AMA (Ask Me Anything) on Reddit (and likely other places) when Ryzen was initially launched. It might take a while, but it's nice to know they are listening, and are willing to put forth the effort to help us solve those problems. PS: In order to take the screen shot, I had to shut down my Window 10 VM. In the ensuing 10 minutes it took me to write this post, the server DID crash (unRAID 6.3.5) ... so BIOS 3402 adds nothing to help prevent crashing. Odd since my kid was watching a movie on the Plex that's a docker on the server. I guess that's just not enough activity? Oh well, hard reboot and parity check. 'Night all!

For ASUS Prime X370-PRO motherboard owners, there appears to be yet another BIOS update available. Version 0810 (Dated 2017/08/18) says: 1. Enhance Memory Compatibility for Ryzen CPU 2. Improve LAN stability before entering to OS I just uninstalled my Ryzen 1800X CPU that I bought on launch day (March 2nd) to put in a box for RMA. I tested it and find it's one of the early chips suffering from the dreaded Linux segfault issue. I also see occasional crashes in my Windows 10 VM when performing multi-threaded tasks. They sent me a FedEx label and said they'll ship out the replacement once the tracking system shows I've dropped off the old one, so I'll take care of that tomorrow morning. As such, I won't be able to try out the new BIOS until I get my replacement chip and have vetted it using the current system settings. - Bill

BTW, you can't deselect CPU0 until you select a different CPU first. Probably a requirement that at least one CPU is selected for any given VM. - Bill

Pretty sure I've run into something similar. These days I just double and triple check that my path settings are accurate. Haven't had a problem in many, many months. We'll see if it shows up at the next OS upgrade. I'm still on 6.3.5 right now. - Bill

Three things come to mind: 1) Click the "Settings" tab, then "VM Manager", and then switch to "Advanced View". Ensure there are no errors/warnings on this page. 2) Click "View libvirt log", see if anything in there is helpful. Post it here if necessary. 3) Toward the bottom under "Libvirt volume info", click "Scrub". Sometimes that gets me out of weird behavior. You probably want to be sure all your other VMs are stopped prior to doing this. For completeness, how long has this issue been occurring? Has it ever worked? Do you recall any system changes performed around the time the failure started? - Bill

System specs: OS: unRAID 6.3.5 CPU: AMD Ryzen 1800X GPU: Asus Radeon 6450 1GB (Desktop) Graphics Card RAM: 64GB - Crucial DDR4-2133 MB: ASUS Prime X370-PRO (BIOS 0807) VM Specs: VM-OS: Windows 10 Version 1703 (OS Build 15063.540) VM-CPU: 8 cores / 8 threads VM-GPU: Asus Radeon 6450 1GB (Desktop) Graphics Card (pass-through) VM-RAM: 16GB Cinebench R15.0 test: CB15 CPU: 8 Cores, 8 Threads @ 3.6 GHz (reported) CB15 GPU: AMD Radeon HD 7400 Series (reported) CB15 CPU Score: 760 cb CB15 GPU Score: 12.58 fps Let me know if you'd like any additional details. - Bill

I'm not a gamer. My GPU is the Asus Radeon 6450 1GB (Desktop) Graphics Card. I can run a test like Cinebench from the Windows 10 VM and return results for that, if that's helpful? I don't own any of those high end games, though. - Bill

I have downloaded an installed the 0807 BIOS dated 07/19/2017 Under "Advanced" --> "AMD CBS", there appear to be several new options: Zen Common Options RedirectForReturnDIs L2 TLB Associativity Platform First Error Handling Enable IBS Opcache Control Custom Pstates / Throttling (*) Core/Thread Enablement (*) Streamline Stores Control DF Common Options DRAM scrub time Redirect scrubber control Disable DF sync flood propagation GMI encryption control xGMI encryption control CC6 memory region encryption Location of private memory regions System probe filter Memory interleaving size Channel Interleaving hash UMC Common Options DDR4 Common Options Fall_CNT DRAM Controller Configuration (*) CAD Bus Configuration (*) Data Bus Configuration (*) Common RAS (*) Security (*) DRAM Memory Mapping Chipselect Interleaving BankGroupSwap BankGroupSwapAlt Address Hash Bank Address Hash CS NBIO Common Options NB Configuration (*) NBIO Internal Poison Consumption NBIO RAS Control PSI ACS Enable PCIe ARI Support CLDO_VDDP Control (*) There are more sub-menus under this option, but I didn't dig to the bottom Overall, the BIOS upgrade went smoothly. None of the "dead system hang" boots like in the past couple of BIOS upgrades. unRAID booted normally and all my VMs/Dockers auto-started properly ... including the Windows 10 VM with GPU pass-through. System temperatures are about the same, idling round 45c. Not sure what else to check? - Bill

Guess that makes the difference. The 1700 is a 3.0/3.7 GHz, 65 watt CPU, and the 1800X is a 3.6/4.0 GHz 95 watt CPU. I'm still thinking I could have easily gotten by with a 1700 chip. I doubt I stress out the 1800X enough to justify the price/power costs. I'll likely rethink which model I upgrade to when Zen2 comes out. - Bill

Yes, original post edited for clarity.

EDIT: All drives have been sold Used SATA 3.5" hard drives for sale. Sizes range from 300gb through 2TB. All drives were pulled from working systems and were subsequently wiped without error. 1 x Seagate Barracuda 300GB 7200rpm (ST3300831AS) - $15 10 x Seagate Barracuda 750GB 7200rpm (ST3750640AS) - $20 each 12 x Seagate Barracuda 1.5TB 7200rpm (ST31500341AS) - $25 each 9 x Samsung Spinpoint F4 2TB 5400rpm (HD204UI) - $30 each 2 x Samsung Spinpoint F4 2TB 5400rpm (HD204UI/ST20000DL004) - $30 each 1 x Western Digital Caviar Green 2TB 5400rpm (WD20EARS) - $30 4 x Seagate Barracuda XT 2TB 7200rpm (ST32000641AS) - $40 each Prices do not include shipping. Preferred shipping will be US Priority Mail. I can do UPS or FedEx if you have an account to cover shipping costs. Free shipping for bulk orders (10+ drives) may be available to destinations in the continental USA. Payment can be made via PayPal or US Postal Money Order. Returns accepted within 14 days, but buyer pays return shipping. SMART reports for the listed drives is posted here: http://www.ufopinball.com/FS I will do my best to answer any questions. Bill

Here's my rig/signature: Cortex • unRAID Server Pro 6.3.5 (Dual Parity) • ASUS Prime X370-PRO MB • AMD Ryzen 7 1800X 8-Core 3.6GHz • Crucial CT16G4DFD8213 DDR4 2133 (64GB) • Seasonic SS-660XP2 660W 80 PLUS PLATINUM • Asus Radeon 6450 1GB (Desktop) Graphics Card • 12 x Seagate ST4000DM000 4TB 5400rpm (40TB) • 2 x SAMSUNG 850 EVO 1TB SSD (Cache) • Docker: PlexTV • VM: CentOS (LAMP server) - Bill

What sort of VMs? You looking at multiple desktops, or high end gaming, or more of a cloud sort of solution? My rig is in my signature, it's mainly home/desktop stuff. I run a Windows 10 VM with GPU passthrough, and a LAMP VM that supports several websites with a database. There's a lot of room for future expansion. I should really look into building that Hackintosh VM someday. - Bill

Dunno if/when things are changing regarding the Temperature plugin, however you can use the interim fix, documented here: https://forums.lime-technology.com/topic/55150-anybody-planning-a-ryzen-build/?page=19#comment-563450 Let me know how it goes and what temperatures you're seeing on your rig. - Bill

So I've heard, but this behavior doesn't seem to be specific to AMD, or even Intel CPUs. - Bill

I'm the sort of OCD that likes to fill all memory slots. My last build was built using the ASUS M3A78-T MB, AMD Phenom II X6 1100T 3.3GHz CPU, and 4 x 4GB Mushkin 991762 DDR2 800 (16GB) memory ... ran unRAID (both 5.x and 6.x) from 2012-2017 with zero problems related to memory. My recently upgraded build uses the ASUS Prime X370-PRO MB, AMD Ryzen 7 1800X 8-Core 3.6GHz CPU, and 4 x 16GB Crucial CT16G4DFD8213 DDR4 2133 (64GB) memory ... been running my main server since late March. No problems related to memory that I have encountered. Do you remember what sorts of "difficulties" were encountered with older AMD builds? - Bill

Looks good, which CPU and MB do you have? My Ryzen 1800X idles at 42c (at best) with nothing running but unRAID. All dockers and VMs are stopped. The only thing "loaded" is a window to the web UI on the "Main" tab to show the current temp. The UPS is reporting 45 watts, and that's with all drives spun down. If you're getting lower 30s, I'm wondering what I may be doing wrong? Chip isn't overclocked or anything. I guess 45 watts isn't *bad*, but if it could be better, I'd like to find out what I'm doing differently. My specs are in my signature. Thanks! - Bill

Yes, well sort of anyway. The full details are posted in another thread: https://forums.lime-technology.com/topic/55150-anybody-planning-a-ryzen-build/?page=19#comment-563450 It's definitely an interim solution, though I don't know if/when a more official solution will be made available. Let me know if you have problems getting this working... - Bill

Yes, I am using the "pcie_acs_override=downstream,multifunction" setting so that I can get the IOMMU group breakdown for my Windows 10 VM. Seems to work fine for me under the latest BIOS and AGESA 1.0.0.6 release. - Bill

Upgraded the BIOS image for the ASUS Prime X370-PRO motherboard to the latest revision 0805 which was released earlier this week. This is the release version of the BIOS featuring the AMD/Ryzen AGESA 1.0.0.6 update. Like the previous time, I had to reboot a few times to get things settled after the BIOS upgrade. No idea why, but it seems to be running smoothly now. No noticeable changes to the IOMMU groupings between this and the previous 0803 (beta) BIOS version. Not that I would have expected it, but in case anyone was wondering. - Bill

Thanks for the info on the ACS override, I was not aware that there were such problems. Will keep an eye on it, but so far my Ryzen system has been very stable. I would run more tests, but it's also a production system (runs several websites, and Plex for my family) so I'm not at liberty to take it down to perform testing at the level you describe. - Bill

I may have, but it's been awhile. I originally ran Cinebench and Blender. Are there other benchmarks you suggest? Note that my GPU isn't a gaming model. As for "utilizing other resources" ... is there an easy way to do this? I can start a second VM, assign it all remaining CPU cores, and run Prime95 there. Does that cover what you are asking? - Bill

Is there any downside to using the ACS override feature? Once enabled, my IOMMU groupings are very, very favorable for VM usage. I am currently passing through my GPU, and also a USB 3.0 port from the motherboard. ASUS Prime X370-PRO motherboard, and the Ryzen 1800X (you can probably do very well with a 1700 though!) - Bill