Ryzen 3000 series build / Struggling with PCI-e lanes


Recommended Posts

Hi all

 

Like many here (I assume), I am planning on a Ryzen 3000 series build in the next couple of months, to replace my existing setup.  This brings with it some choices around chipsets, pci-e lanes, etc.  My current setup is:

 

  • Node 804 (limited to Micro ATX or smaller)
  • i7-2600k
  • Supermirco Micro ATX 
  • Broadcom 9207-8i 8 port HBA (PCI-e 3.0 8x)
  • TBS dual DVB tuner (PCI-e 2.0 1x)

 

Switching to Ryzen has certain limitations:

  1. Lack of integrated graphics - meaning I have to fit a GPU on a mATX board reducing the number of available slots for other cards
  2. Only 6 (B450) or 8 (X470) PCI-e 2.0 lanes from chipset.  

 

So the best case would be PCI-e 4x 2.0 for the 9201-8i (in a 16x mechanical slot), meaning it will only be running at 2GB/s total instead of 8GB/s.  This works out to ~ 2Gbit per drive so it is above SATA1 speed and I assume unlikely to limit performance much for mechanical disks - can anyone confirm?  Will the card even function like this?

 

Generally, will Ryzen boards boot (and will unRaid function) if no GPU is present?

 

I am aware that new boards with X570 chipset may solve this problem (12 Gen 4 lanes from chipset), but trying to clarify if it is possible to go with a cheaper chipset.

 

 

Edited by flaggart
Link to comment
  • 2 weeks later...
On 6/11/2019 at 3:51 AM, PSYCHOPATHiO said:

I've been on a Ryzen 1700X for over two years & last night I switched it with a 6700K. Unfortunately The first & second Ryzen gen have memory comparability issues & my server wasn't stable enough.

Just make sure you choose the correct memory that is compatibly with your motherboard of choice.

I am running a 1700x but having many stability issues. My server freezes about every 12 hours, no errors in syslog just hangs up and will not respond via keyboard or mouse or IP address. Is this an issue with these processors? Memtest shows no issues and ive ran this hardware for years as a windows 10 machine.

Link to comment

i switched back to the Ryzen. the main problem was as always the incompatible memory... I have the  Corsair LPX 3200 (Clocked 2933 - 15-17-17-35).

I took a look at its closest compatible module from corsair which is the 3000mhz that was clocked at 2933 according to their memory QVL and set the memory to automatic and changed the voltage and timings manually to the timings matching the listed one.

 

Since then I have never seen a crash yet I'm still in a bit of a worry if it might crash at any given point but managed to get 4 days before I manually restarted to add some cards & drives.

 

3 hours ago, erax0r said:

My server freezes about every 12 hours

This used to happen a lot before disabling the c-state, is it disabled in your case?

Under windows when u install AMD chipset drivers AMD also installs their own power profile to keep the CPU from idling and crashing.

Link to comment
On 6/11/2019 at 2:24 PM, sarbian said:

I actually upgraded my setup yesterday with a 2600 running on a X470D4U in a Node 804. It has an integrated GPU with a VGA output (and remote IPMI). It may help with your GPU.

Have you had any trouble using the USB 3.0 ports with your Unraid USB stick? I thought about getting one but the lack of USB 2.0 kind of scared me off.

  • Like 1
Link to comment

thinking of it myself. 2 things I am looking at

1. I hope they come out with a hba card that utilizes the 4.0 lanes and can do 16 drives, not like the ones now with a max of 6 before additional hardware.

2. I am going to get one anyways, I am looking at a dedicated plex/emby setup anyways and if this does not work with unraid I will just re purpose it for the plex/emby server.

Link to comment
On 6/20/2019 at 5:39 AM, ramblinreck47 said:

Have you had any trouble using the USB 3.0 ports with your Unraid USB stick? I thought about getting one but the lack of USB 2.0 kind of scared me off.

None. The only trouble I had with this MB is that it required UEFI to boot

Link to comment
  • 2 weeks later...
On 6/22/2019 at 2:07 PM, sarbian said:

None. The only trouble I had with this MB is that it required UEFI to boot

Thanks for the info. I still think if I go Ryzen, I'll go with the ASRock X570M Pro4. It's significantly cheaper at $185, has USB 2.0 ports, 8 x SATA ports, and is the only X570 mATX board. It doesn't have IPMI, but I don't really have a need for that anyway.

 

The only thing that had me hesitant at first was the fact that it only has PCIE x16 and a PCIE x4 lanes. My LSI 9211-8i shouldn't see a big bottleneck being limited to a PCIE 2.0 x4 lane. Should be still enough bandwidth for 8 x 5400rpm hard drives I'm guessing? Found this a while ago and it looks like it is still applicable:

 

Dell H310 PCIe gen2 x8 (4000MB/s) - 6w – LSI 2008 chipset, results should be the same as IBM M1015 and other similar cards

4 x 455MB/s

6 x 377.5MB/s

8 x 320MB/s (190MB/s*, 185MB/s**)

 

*on PCIe gen2 x4 (2000MB/s)

**on PCIe gen1 x8 (2000MB/s)

 

Link to comment

I intend to go Ryzen when the 16core CPU come out in September, I'll be moving a 12core into my unRAID server from another system. I'm hoping that any Linux wonkiness will be worked out by then, I see that current up to date Linux kernels are currently having some issues with the new CPU (Phoronix reported on it). The added clockspeed and IPC will be welcomed but I'll be reducing my core count. I believe it'll be a huge improvement!

The adapter issue is a big one for me, likewise video. Most of the mobo I'm seeing have just 3x full length slots and I'll need a video card - need to find a cheapie that can sit in a short slot. Each of my current adapters currently have 8x drives in them and it looks like that's hurting me with shared bandwidth. Finding a higher performance adapter might be nice as would finding a board with more suitable slots!

My first build (a server for Docker Swarm not unRAID*) will be using an Asus  Pro WS X570-ACE but looking at it I'm not sure how suitable it might be for unRAID, it has the slot issue plus I think I'd like a 10gig network connection for the future :( I chose it because doesn't have crap onboard like bluetooth, wifi, or spiffy audio and has some remote management features. It wasn't cheap though and I'm disappointed in the 1gig network port.

Has anyone else seen anything more suitable? Are there any drive better adapter cards that can handle a density of drives like the current crop of cheap SAS cards but not lose so much speed? A nasty PCIE4 card would rock but I'll be surprised if we see something reasonable anytime soon. @johnnie.black did some awesome testing previously, has anything better come along since?

 

I will at least try booting unRAID on my new system to see how it might fail and report on it!

 

* wish I could get Docker Swarm working on unRAID :(

Edited by BLKMGK
Link to comment

@BLKMGK have you looked into getting an intel res2sv240, its a sas extender you can plug 1 or 2 8087 cable(s) from say a 9211-8i to the res2sv240 and have 16-20 (if only using 1 cable) drives in your system i use 1 of these in my current unraid box. You could then have a max of 24 drives connected through a 8i HBA card.

The res2sv240 can be powered from a molex 4 pin or from a spare pcie slot. It only requires the power to work so if you need all your slots it can be placed somewhere else in the case.

 

I'm currently looking at doing the same ASUS board with either a 3900x or 3950x making it my gaming rig(2080ti), plex(p2000), and storage(lsi 9207-8e). i just want to see a few people that jump in first give feedback to see if it's worth it

Link to comment

Hi guys! I just pulled the trigger on a ASUS Pro WS X570-ACE + a Ryzen 9 3900x.  I have a 21 drives Unraid and plan on upgrading my stuff with this new Ryzen 3900x and board.  I currently have 2 9211-8i and 1 GPU (for PLEX transcoding, but in a 1x slot with a Mining Riser that I reuse, works fine).  I might add a GTX 1080Ti for a VM (passthru) when that will be fix on the new platform.  I'll probably run that on a "test" Unraid until everything works fine, pretty sure i'll have some issues :)  Also, i still need to choose RAM for the new system and a NVMe PCIe Gen4 for the Cache drive.

 

I'll keep posting to this topic what my findings are.  Hope to at least be able to boot current 6.7.2 Unraid on my board.

Edited by Pducharme
  • Upvote 1
Link to comment

I ended going with a Gigabyte Aorus Master i could not find any retaill shops online with asus ws ace in stock, and a 3700x (for now, I can upgrade my 1700 after) whilst i wait to see what the 3950x performs like in september. i live in B.F nowhare Australia so i didn't expect a local to have if any or at a decent price so i found a reputable shop on ebay that doesn't price jack when there is a sale on, there was a 10% off ebay sale this week M/B was $629 down to $566 (~usd392) and CPU $519 to $467 (~usd323). all AUD prices incl a 10%gst in the final price.

 

They should be here by next monday-ish, here is a passmark comparison of what im going from and to https://www.cpubenchmark.net/compare/AMD-Ryzen-7-3700X-vs-Intel-Xeon-E5-2660-v4-vs-AMD-Ryzen-9-3900X/3485vs2881vs3493

 

I might have to buy a basic unraid for testing the setup before i jump full in and dismantle my main server, fun times ahead, for testing i might put in gtx 1080(plex) and 680(VM) they will eventually be replaced with my 2080ti(VM's) and p2000(plex), a spare LSI HBA with an old drive or 2 (test out my ds4243 for what will be in the new build) if i don't have to many dramas it runs smooth and plex works as intended i might only upgrade the cpu to the 3900x to give the vm a few more cores.

 

Future plans may include 2x NVME upgrade for the cache (maybe the silicon power 512gb) and OS VM installs(1tb with maybe 2 patitions mostly for windows and small 60GB osx)

 

 

Link to comment
2 hours ago, fr05ty said:

I ended going with a Gigabyte Aorus Master i could not find any retaill shops online with asus ws ace in stock, and a 3700x (for now, I can upgrade my 1700 after) whilst i wait to see what the 3950x performs like in september. i live in B.F nowhare Australia so i didn't expect a local to have if any or at a decent price so i found a reputable shop on ebay that doesn't price jack when there is a sale on, there was a 10% off ebay sale this week M/B was $629 down to $566 (~usd392) and CPU $519 to $467 (~usd323). all AUD prices incl a 10%gst in the final price.

 

They should be here by next monday-ish, here is a passmark comparison of what im going from and to https://www.cpubenchmark.net/compare/AMD-Ryzen-7-3700X-vs-Intel-Xeon-E5-2660-v4-vs-AMD-Ryzen-9-3900X/3485vs2881vs3493

 

I might have to buy a basic unraid for testing the setup before i jump full in and dismantle my main server, fun times ahead, for testing i might put in gtx 1080(plex) and 680(VM) they will eventually be replaced with my 2080ti(VM's) and p2000(plex), a spare LSI HBA with an old drive or 2 (test out my ds4243 for what will be in the new build) if i don't have to many dramas it runs smooth and plex works as intended i might only upgrade the cpu to the 3900x to give the vm a few more cores.

 

Future plans may include 2x NVME upgrade for the cache (maybe the silicon power 512gb) and OS VM installs(1tb with maybe 2 patitions mostly for windows and small 60GB osx)

 

 

Plz keep us posted! I'm looking for a X570 mobo for my gaming nas build. 

Link to comment
2 hours ago, fr05ty said:

 

They should be here by next monday-ish, here is a passmark comparison of what im going from and to https://www.cpubenchmark.net/compare/AMD-Ryzen-7-3700X-vs-Intel-Xeon-E5-2660-v4-vs-AMD-Ryzen-9-3900X/3485vs2881vs3493

 

Here is my passmark comparison :  https://www.cpubenchmark.net/compare/AMD-Ryzen-9-3900X-vs-Intel-Xeon-E3-1240-v3/3493vs1952

 

Pretty solid upgrade :)  I'm thinking about getting the Corsair MP 600 2TB or 1TB for my Cache drive.  The 1TB is out of stock, but not the 2TB...  Still looking for a compatible RAM.  I think that with Unraid, there is no benefits of going ECC RAM ?

Link to comment
12 hours ago, Pducharme said:

Hi guys! I just pulled the trigger on a ASUS Pro WS X570-ACE + a Ryzen 9 3900x.  I have a 21 drives Unraid and plan on upgrading my stuff with this new Ryzen 3900x and board.  I currently have 2 9211-8i and 1 GPU (for PLEX transcoding, but in a 1x slot with a Mining Riser that I reuse, works fine).  I might add a GTX 1080Ti for a VM (passthru) when that will be fix on the new platform.  I'll probably run that on a "test" Unraid until everything works fine, pretty sure i'll have some issues :)  Also, i still need to choose RAM for the new system and a NVMe PCIe Gen4 for the Cache drive.

 

I'll keep posting to this topic what my findings are.  Hope to at least be able to boot current 6.7.2 Unraid on my board.

How many pcie cards do you have installed in your system. I plan on having a gpu via pcie x16, usb controller, and Dell perc in it mode and made another gpu in pcie 1x (riser card). 

Link to comment

Currently, I have :

 

1 card x GTX 1060 6GB in a 1x (riser card) - Dedicated to PLEX

2 cards x LSI 9211-8i in IT mode - For 16 disks (other 5 disks are from onbard SATA controller)

 

Might add :

 

1 card x GTX 1080ti in a x16 slot - Passthrough to a Win10 VM (Not sure yet i'll do that).  I think I would also need a USB card for Keyboard/mouse and I would miss slots because of that ? The ASUS WS Pro X570-ACE  has 3 x x16 slots and 1 x x1 slot.

 

Link to comment

I just pulled the trigger on a Corsair MP 600 1TB NVMe Gen4.   Found it In Stock at Canada Computers (i'm in Canada).  Should receive everything in maximum of 15 days I suppose.   For the RAM, still undecided.  I might use one of my 2 stick (8GB) from my Main PC during testing of Unraid on this new platform.   Still trying to see if they are any benefits for ECC on Unraid (I think not).  At least, my Cache drive will fly ! :)

Link to comment
7 hours ago, fr05ty said:

I ended going with a Gigabyte Aorus Master i could not find any retaill shops online with asus ws ace in stock, and a 3700x (for now, I can upgrade my 1700 after) whilst i wait to see what the 3950x performs like in september. i live in B.F nowhare Australia so i didn't expect a local to have if any or at a decent price so i found a reputable shop on ebay that doesn't price jack when there is a sale on, there was a 10% off ebay sale this week M/B was $629 down to $566 (~usd392) and CPU $519 to $467 (~usd323). all AUD prices incl a 10%gst in the final price.

 

They should be here by next monday-ish, here is a passmark comparison of what im going from and to https://www.cpubenchmark.net/compare/AMD-Ryzen-7-3700X-vs-Intel-Xeon-E5-2660-v4-vs-AMD-Ryzen-9-3900X/3485vs2881vs3493

 

I might have to buy a basic unraid for testing the setup before i jump full in and dismantle my main server, fun times ahead, for testing i might put in gtx 1080(plex) and 680(VM) they will eventually be replaced with my 2080ti(VM's) and p2000(plex), a spare LSI HBA with an old drive or 2 (test out my ds4243 for what will be in the new build) if i don't have to many dramas it runs smooth and plex works as intended i might only upgrade the cpu to the 3900x to give the vm a few more cores.

 

Future plans may include 2x NVME upgrade for the cache (maybe the silicon power 512gb) and OS VM installs(1tb with maybe 2 patitions mostly for windows and small 60GB osx)

 

 

I think I'll go with this one too since it has more m. 2 slots. 

Edited by phat_cow
Link to comment
3 hours ago, Pducharme said:

Still trying to see if they are any benefits for ECC on Unraid (I think not).

Not sure how to answer that. If you have a memory chip error, non-ECC silently corrupts the data, leaving it up to the program logic to catch and correct it. With ECC, the error is trapped, corrected if possible, if not, the system is halted to avoid further corruption.

 

Unraid doesn't need or benefit from ECC any differently than any other OS.

 

It pretty much comes down to risk tolerance and backup strategy. If you have full backups with checksums and don't mind dealing with possible silent corruption, then ECC is probably a convenience instead of a necessity.

  • Upvote 1
Link to comment

I am looking at the following for a middle of the range Unraid as all I want it to do at the moment is host a few VM's one in a while a few dockers and about 8 SATA drivers. and of course Plex.

 

Motherboard    Asrock B450 Pro4 ATX

CPU                 Ryzen 7 270   as this is the same price as the R5 3600 but is an 8 core/16 instead of a 6 core 12 thread

 

side note: do you think it is a good move to be looking at old technology and looking at the latest ? as I feel that the two CPUS are about the same in power and overheads

 

Link to comment

@chris_netsmart ok just going from what wendell says form Level1Linux video (at about 5:50 in video below) iommu doesn't really separate things out in the older boards x370 x470 asmedia chip-set, where the x570 separated it out a lot cleaner, if you look at the asrock board you mentioned i have just pulled this info of the asrock site i dont know anything else about this board

 

AMD Ryzen series CPUs (Summit Ridge and Pinnacle Ridge)
- 2 x PCI Express 3.0 x16 Slots (PCIE2: x16 mode; PCIE4: x4 mode)*
- 4 x PCI Express 2.0 x1 Slots
*Supports NVMe SSD as boot disks If M2_1 is occupied, PCIE4 will be disabled.

 

so if you wanted a nvme in m.2_1 slot, you would only have 1x16 pcie 3.0 slot and the 4 x1 PCIe 2.0 slots for use

A single PCIe 2.0 lane. The 6Gbps SATA spec allows for up to 750MB/s of bandwidth, but the PCIe 2.0 x1 interface limits read/write speed to 500MB/s. so passing things through to a VM might be a little trickier on the x1 slots and drive scrubbing could take a little bit longer if you connect a HBA card to one of these slots.

 

if you fill M.2_2 slot you lose 2 sata ports of the 6 onboard but you retain the use if the pcie 3.0 x4 slot, and the nvme slot runs at half the speed of slot 1 if you get/have one of the 3000MB/s ssd's.

 

- 1 x Ultra M.2 Socket (M2_1),  M.2 PCI Express module up to Gen3 x4 32Gb/s

- 1 x M.2 Socket (M2_2),  M.2 SATA3 6.0 Gb/s and M.2 PCI Express up to Gen3 x2 16 Gb/s

*M2_2, SATA3_3 and SATA3_4 share lanes. If either one of them is in use, the others will be disabled.

**If M2_1 is occupied, PCIE4 will be disabled.

 

i hope this info helps you decide what you want to do

 

on my asus prime x370 pro has

IOMMU group 15:

[1022:43b9] 02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset USB 3.1 xHCI Controller (rev 02)

[1022:43b5] 02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset SATA Controller (rev 02)

[1022:43b0] 02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset PCIe Upstream Port (rev 02)

[1022:43b4] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

[1022:43b4] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

[1022:43b4] 03:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

[1022:43b4] 03:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)

[1000:0072] 04:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

[1b21:1343] 05:00.0 USB controller: ASMedia Technology Inc. ASM1143 USB 3.1 Host Controller

[8086:1539] 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)

 

 

  • Upvote 2
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.