Do i understand PCIE correct?

[Oedzesg]

Hallo Unraid users. 

 

Hope you can explane me if u understand the pcie lanes correctly. 

 

My situation: 

 

My server is on a SuperMicro X12STH-LN4F. 

 

This board has:

 

1 pcie 4.0 x16  (bij cpu) 

1 pcie 4.0 x8 / running on 4 lanes (bij cpu) 

1 pcie 3.0 x8 / running on 4 lanes (bij chipset) 

 

At this moment the x16 slot is used by an Dell perc h310 expantion card and connected to my 8 WD RED HDD Drives. 

 

The Dell is an pcie 2.0x8 card. 

PCIE 2.0 = 500 mb per lane x8 = 4000 mb/s 

Read and write speeds WD RED = 180 mb/s. x 8 disks = 1440 mb/s. 

 

So at this moment no problem.

 

but i think it's a shame to have a pcie 2.0 card with 8 lanes in a full 4.0x16 slot.

 

Then I would rather use the 4.0x16 slot for a fast extra nvme cache.

 

But if i place the Dell card in the one of the slots with 4 lanes there is a max speed of 4x500 = 2gb/s

For now it should work i guess? 

 

But for future updates to SSD NAS drives... 

The WD RED SSD = 560MB/s x 8 = 4480 mb/s 

So thats not possible if i understand correctly? 

 

If i upgrade to an pcie x3 adapter card: 

Pcie 3.0 = 1 gb/s per lane = 4 gb/s. 

 

So thats close en since i dont use all disk at the same time in Unraid it would work? 

 

And would it make any different if i use the chipset slot? 

The chipset is connected with 8 pcie 3.0 lanes to the cpu. 

The only thing that is else is connecteted to the chipset is one nvme drive whitch uses 4 lanes. 

I dond use anny usb devices or whatever. 

So there are 4 lanes left? 

 

I only use this server in my home, so the probability of reading from the disks and writing to the cache being used at the same time is very small.

 

I hope that someone can tell me i understand this all correct or missing something. 

 

Thnx in advance. 

 

 

 

 

 

[JorgeB]

See here for some real world numbers, but yes you can use a PCIe 3.0 HBA with 4 lanes for good performance.

 

11 minutes ago, Oedzesg said:

And would it make any different if i use the chipset slot? 

Chipset slots share the DMI with any other devices using, like onboard SATA, NIC, NVMe, etc, so you need to account for that.

[Oedzesg]

Thnx! 

 

I also thinking about adding a 10 gbe network card. 

Would it be better to install this one into the pcie chipset slot?

[JorgeB]

Same considerations to take into account,  i.e., the share DMI.

[Zonediver]

On 1/10/2023 at 8:28 PM, Oedzesg said:

Thnx! 

 

I also thinking about adding a 10 gbe network card. 

Would it be better to install this one into the pcie chipset slot?

 

I use three PCIe-Cards:

 

.) A PCIe 3.0 x8 HBA in the primary PCIe 3.0 x16 slot (CPU)

.) An Intel GBit-NIC in the PCIe 3.0 x1 slot (PCH)

.) A PCIe 3.0 x4 10 GBit-NIC from Asus in the PCIe 3.0 x4 slot (PCH)

 

There are no bottlenecks so far...

Both NICs are using 5 PCIe-Lanes of the PCH and work with full speed - the 10 GBit-NIC reaches 9480 MBit max.

Edited January 12, 2023 by Zonediver