Anybody planning a Ryzen build?


Recommended Posts

30 minutes ago, johnnie.black said:

 

Interesting, I measured 6w from both of these, idling with 8 devices connected.

 

My raw notes read as follows:

 

Power per SATA controller:

92.2w - Bare machine (Asus M3A78-T MB) w/ 4GB RAM and no other peripherals
99.4w - Dell H310 (8 ports)    [7.2 watts x 2 = 14.4 watts]
101w - AOC-SAS2LP (8 ports)    [8.8 watts x 2 = 17.6 watts]
97.4w - Adaptec 430A (4 ports) [5.2 watts x 3 = 15.6 watts]

 

This was taken some months ago.  I don't know what power supply I was using, but the wattage was measured using a Kill-A-Watt device.

 

How did you perform your tests?

 

- Bill

 

Link to comment
1 hour ago, ufopinball said:

How did you perform your tests?

 

My interest was comparing the HBAs with the Intel onboard controller, so I measured with the 8 onboard ports connected at idle and during a read check and repeated with the 8 devices connected on the HBAs, I used a Ubiquiti MFI that from what I've read gives very accurate readings:

 

                    Idle   -   Read check

Onboard  -    41   -        51

SAS2LP   -    47   -        57

LSI 9211  -    47   -        57

LSI 9207  -    50   -        60

 

Rounded to the closest watt.

 

Note: CPU usage represents the biggest part of the increase during the read check, I have no way of measuring just the HBAs.

Edited by johnnie.black
Link to comment

I feel like fixing the Global C-State and NPT bugs, both issues that look like they could be software related, would go a long way towards making Ryzen attractive for Unraid. A 10-20W idle increase over my quadcore haswell Xeon would be acceptable, but an additional 40-50W of needless idle power consumption and mysteriously crippled multicore VM performance is not.

A shame really, looks like I'll have to wait for Skylake-X/Kabylake-X and lose the ECC support or wait a year or so for the refresh of Ryzen.

 

Link to comment

Just join Ryzen party with R7 1700, Asus Prime X370 pro. The build and test already complete. Thanks @Pauven for the C-State tweak, this save lot of time troubleshoot.

 

- Whole system power draw have some increase, but check not a big deal. ( I have UPS, Socket meter, clamp meter, quick measurment )

- RGB LED mode config retailn in main-board , power-off won't lost. (setting first under Windows)

- Quite disappoint the storage throughput, it celling at ~700MB/s. ( LSI 2308, 6 data + 2 parity disks ), CPU usage ~20%, in this moment it seems SW bottoleneck rather then HW issue.

Edited by Benson
Link to comment
5 hours ago, johnnie.black said:

Note: CPU usage represents the biggest part of the increase during the read check, I have no way of measuring just the HBAs.

Maybe not completely accurate, but you could get close by doing a parity check with no devices connected to the HBA you want to measure, and then the same check with drives. I don't think the CPU usage would be that different between a 6 drive parity check (onboard only) and a 14 drive check. The drive draws themselves should be easy to factor out, especially with your stable of power consistent SSD's to test.

 

If you have a way of accurately measuring the temperature over time of the HBA chip, that would go a long way towards showing overall power consumption. As a practical matter, every watt consumed is shed as heat. If the temperature rises during heavy activity vs idle, you know it's drawing more power.

Link to comment
On 5/5/2017 at 10:29 PM, ufopinball said:

Okay, so if the overclock sleep bug is previously fixed, is the new AGESA 1.0.0.6 all about memory compatibility?  Is there something specific you're looking for in this release?

 

No, nothing specific.  The only thing I've seen listed so far for 1.0.0.6 is the memory compatibility enhancements.

 

On 5/5/2017 at 10:29 PM, ufopinball said:

SuperMicro AOC-SAS2LP-MV8 (8 ports) -- 8.8 watts

 

My HighPoint 2760A uses the same chip as the AOC-SAS2LP-MV8 (same driver too).  The difference is that the 2760A has 3 of those chips (for 24 drives instead of 8), and a PLX PCIe hub to allow all three to communicate over the PCIe bus.

 

3 x 8.8w = 26.4w, plus a bit more for the PLX hub, and you're at the same 27w that the 2760A consumes.

 

Going from memory, there was a 2740A with 16 ports, and a 2720A with 8, which would be equivalent to the AOC-SAS2LP-MV8.  I think all three of these are no longer in production.

 

I chose the 2760A to go with my Celeron G1610 because the mini-ITX motherboard I used only had a single PCIe slot, and only a few SATA ports.  This card was pretty much the only way I could get over 20 drives.

 

Now with my new Ryzen motherboard, it is overkill as my motherboard has 10 SATA ports (I think), plus a couple M.2 slots, and plenty of PCIe slots.  I could have gone with a 16 port card, or a couple 8 port cards.  The 2760A is very expensive, so I would only recommend it if you can't find more affordable alternatives.

 

On 5/5/2017 at 10:29 PM, ufopinball said:

Considering your power report I'll probably spend the upgrading money moving to 8TB drives, instead of moving to a whole new case and add/upgrade drive controllers, just to support more 4TB drives.

 

I've done the math over and over and over again, and it always comes out that installing bigger hard drives is the cheaper solution, as you need fewer hard drives, fewer ports, fewer cages, fewer everything.  If I divide the cost of my server (CPU/MB/Memory/GPU/Controller/PS/Case) by the number of drive slots (24), each bay costs about $115, and that's before adding any hard drives.  

 

Fill that up with 24 3TB drives @ $115/ea, and you get a 66TB server (dual parity) at a cost of $83/TB.

 

Fill that up with 24 6TB drives @ $220/ea, and you get a 132TB server (dual parity) at a cost of $61/TB.

 

Fill that up with 24 8TB drives @ $270/ea, and you get a 176TB server (dual parity) at a cost of $53/TB.

 

Fill that up with 24 10TB HGST He10 Helium drives @ $500/ea, and you get a 220TB server at a cost of $67/TB.  These are real nice drives, and should last a long time, with a similar cost to 6TB drives, but certainly not the cheapest path forward at this time.

 

-Paul

Edited by Pauven
  • Upvote 1
Link to comment
15 hours ago, johnnie.black said:

 

Interesting, I measured 6w from both of these, idling with 8 devices connected.

 

Good to know the Dell H310 and SuperMicro SAS2LP 8-port controllers are comparable in power consumption ... at least at idle.

 

I was running on the SAS2LP when I was on unRAID 5.x and never had a problem.  I upgraded to unRAID 6.x and ran into the same problems that others were seeing (drives would suddenly disappear during parity checks, etc.)  LimeTech never offered a fix, but did suggest that the older motherboard (ASUS M3A78-T w/ AM2 socket, purchased in 2008) might be to blame.  So, I moved over to the Dell H310 controllers which worked fine in my existing setup.

 

Much later, after finishing the Ryzen build, I freed up my Win7 workstation (ASUS F2A85-V PRO w/ FM2 socket, purchased in 2012) and ran a test with the SAS2LP board.  I didn't run into problems, so it seems that LimeTech was right on the money for this issue.  If/when I decide to sell the SAS2LP boards, I'll feel less guilty since they appear to be solid so long as your existing setup isn't as ancient as mine was.

 

- Bill

 

Link to comment
Maybe not completely accurate, but you could get close by doing a parity check with no devices connected to the HBA you want to measure, and then the same check with drives. I don't think the CPU usage would be that different between a 6 drive parity check (onboard only) and a 14 drive check. The drive draws themselves should be easy to factor out, especially with your stable of power consistent SSD's to test.
 
If you have a way of accurately measuring the temperature over time of the HBA chip, that would go a long way towards showing overall power consumption. As a practical matter, every watt consumed is shed as heat. If the temperature rises during heavy activity vs idle, you know it's drawing more power.


That's a good idea, I'll try it when I get the chance.
Link to comment

Additional observations about VM-related Freezes:

  • Debian Netinst (non-GUI) freezes when I hit Choose Language.
  • Antergos freezes once you choose your disk partitioning style, but the cursor can still move.
  • When the host freezes, the system is not technically hung; WebUI, SSH and (seemingly) the local console hang, but I can upload an ISO and it stays across the reset.

Still no closer to any progress (I'd test normal linux but I can't get a compiled kernel to boot...), but the last one was something I thought was interesting.

 

EDIT: Because I don't want to double post...

I'm an idiot; I finally decided that since I needed the QEMU Emulated CPU rather than Host Passthrough to get Windows running, I'd try it on Linux.

Tried Antergos, and the installer itself errored out, oh well.

Tried Kubuntu, and it actually succeeded, though it failed to restart I could just Force Stop the VM at that point.

 

Trying to install a 4.11 kernel on Kubuntu made it stop booting, so there's still a good deal to poke at, but at least I get further than everything crashing, it seems.

Edited by dkabot
Link to comment
19 minutes ago, garycase said:

 

Nifty gadget.   Hadn't seen that before, but definitely on my "gotta get one of those" gadget list :D

 

I like them very much but be aware that development and software support were discontinued by Ubiquiti (although they still sell them).

Link to comment

This thread is an interesting read and I decided to place an order.

 

Will see how I get on when it arrives... 

I already own the GPU and 1x 4TB listed here. What's left of my old system will turn into another PC until I buy another GPU.

 

System will be used for things like Plex, tvheadend etc, but also some linux VM's for testing work related stuff.

I'd like to run a windows VM as well and could buy a different GPU for it but will have to see how it effects things like TVHeadend that might be using a PCIe card. But server related tasks are more important.

 

PCPartPicker part list: https://pcpartpicker.com/list/8mhZd6
CPU: AMD - Ryzen 7 1700 3.0GHz 8-Core Processor 
CPU Cooler: Corsair - H100i v2 70.7 CFM Liquid CPU Cooler 
Motherboard: Gigabyte - GA-AX370-GAMING 5 ATX AM4 Motherboard 
Memory: Corsair - Vengeance LPX 32GB (2 x 16GB) DDR4-2666 Memory 
Storage: Western Digital - Blue 500GB 2.5" Solid State Drive 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive 
Video Card: Gigabyte - GeForce GTX 670 2GB Video Card 
Case: be quiet! - Silent Base 800 w/Window (Orange) ATX Mid Tower Case 
Power Supply: Super Flower - Leadex Gold 650W 80+ Gold Certified Fully-Modular ATX Power Supply 
 

Edited by Tuftuf
Link to comment
54 minutes ago, Tuftuf said:

This thread is an interesting read and I decided to place an order.

 

Will see how I get on when it arrives... 

I already own the GPU and 1x 4TB listed here. What's left of my old system will turn into another PC until I buy another GPU.

 

System will be used for things like Plex, tvheadend etc, but also some linux VM's for testing work related stuff.

I'd like to run a windows VM as well and could buy a different GPU for it but will have to see how it effects things like TVHeadend that might be using a PCIe card. But server related tasks are more important.

 

PCPartPicker part list: https://pcpartpicker.com/list/8mhZd6
CPU: AMD - Ryzen 7 1700 3.0GHz 8-Core Processor 
CPU Cooler: Corsair - H100i v2 70.7 CFM Liquid CPU Cooler 
Motherboard: Gigabyte - GA-AX370-GAMING 5 ATX AM4 Motherboard 
Memory: Corsair - Vengeance LPX 32GB (2 x 16GB) DDR4-2666 Memory 
Storage: Western Digital - Blue 500GB 2.5" Solid State Drive 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive 
Video Card: Gigabyte - GeForce GTX 670 2GB Video Card 
Case: be quiet! - Silent Base 800 w/Window (Orange) ATX Mid Tower Case 
Power Supply: Super Flower - Leadex Gold 650W 80+ Gold Certified Fully-Modular ATX Power Supply 

 

Will say with the Corsair H100i v2 make sure if you're going to use the RGB set it in windows or some other OS first. The setting will hold. Should be in good shape with the Corsair 2666, I've been having some difficulty getting the 3200 timings right, stuck at 2666 right now until I decide to mess with that more. I'm running a 1600X pretty snappy but physics scores are limited in 3DMark and trying to work that out.
 

 

Link to comment
6 hours ago, MrMoosieMan said:

How about stability?

 

I've had no problems with stability with it locking up on my 1600X test box with C6 state turned to off. VM passthrough I'm trying to dial in Isolation and cpu pinning in the VM to 6 vCPUs. Threads are paired directly in the system it appears 0->1,2->3 if I'm reading system tools properly. Got the system OC'ed to 4.1 without issue. 

 

Is there a concensus from those running VMs to turn on or turn off NPT? Dialed back my vCPU and having a hell of a time getting it to load properly with 4 cores. No difference in performance between the 2. MSI is "enabled" for my GTX1080 and it still runs with a 3Dmark firestrike extreme of about 6100.

Edited by phbigred
  • Upvote 1
Link to comment

ETA is

9th - Cooler, RAM, SSD and SanDisk Fit 16GB USB. (Arrived)

10th - Case, PSU, Mobo, HDD and CPU

11th - Cooler CPU Bracket...............................................Ordered a few hours before I actually ordered the cooler. Once the free bracket was ordered there was no going back though. I was thinking of using the AMD cooler (I think there is one in the box) but now I know its only for 24hours, I really should wait.

 

Sad part is I'm away all weekend from Friday afternoon and working away 4 days next week. Pretty much have Thursday to get it in a state I can continue with it remotely.

Link to comment
11 hours ago, Tuftuf said:

ETA is

9th - Cooler, RAM, SSD and SanDisk Fit 16GB USB. (Arrived)

10th - Case, PSU, Mobo, HDD and CPU

11th - Cooler CPU Bracket...............................................Ordered a few hours before I actually ordered the cooler. Once the free bracket was ordered there was no going back though. I was thinking of using the AMD cooler (I think there is one in the box) but now I know its only for 24hours, I really should wait.

 

Sad part is I'm away all weekend from Friday afternoon and working away 4 days next week. Pretty much have Thursday to get it in a state I can continue with it remotely.

1st thing to do is update your bios, most boards are being sent out with early revisions. If you didn't get memory with Samsung "b" you may want to hold off on playing with memory timings until you get back. Stock 2133 is enough to get you started. OC with aftermarket cooler should be straightforward and fast to do. And turn off C6 state....

Edited by phbigred
Link to comment
4 hours ago, Greygoose said:

If I go ryzen Unraid setup with the following, will it work without issues. 

 

Unraid 

Plex docker

Deluge docker

Krusader docker 

Windows 10 VM x2

 

Big question is are you doing passthrough of GPUs for the VMs? If so that would likely be the only sticking point as Ryzen doesn't have an integrated GPU on the die. You'll need to follow Gridrunner's video on passthrough if using Nvidia in the first slot. Not a huge deal but also look at isolating the VMs from the OS leaving a few cores for the OS. Also my VMs run decently with 8GB of RAM but figure plex needs about 4GB with the OS to run decently. Isolate plex cores separate from the VM using --cpuset-cpus=0,1 at least for plex to isolate. Otherwise Godspeed and good luck!

Link to comment
4 hours ago, Greygoose said:

If I go ryzen Unraid setup with the following, will it work without issues. 

 

Unraid 

Plex docker

Deluge docker

Krusader docker 

Windows 10 VM x2

 

 

You are asking if using hardware that is not (yet) supported by the software will work perfectly, while there is a chance that it will indeed work free of issues, the risk that it wont do it is much greater. Reading though the thread will inform you of some of the issues with Ryzen and Unraid at the current time.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.