PCI-e storage controller: CPU or chipset lanes?

[koyaanisqatsi]

Hey All,

 

Did some searches and couldn't find any indication whether placing the storage controller on PCIe lanes that connect directly to the CPU or to the chipset were better for unRAID.  Is there any discussion on this?  If not, I'd love to hear experiences.  

 

I've currently got a Core i3-7350K CPU @ 4.20GHz on a Supermicro C7Z170-SQ, with an 8-port Marvell 88SE9485 SAS/SATA 6Gb/s controller and 8 6TB WD Red EFRX disks.  Right now, the controller is in a slot on CPU PCIe lanes.  But I could move it to a slot that has chipset PCIe lanes if that's a better setup. 

 

It seems like CPU lanes could be faster, esp for calculating parity, since it's more direct to the CPU, possibly resulting in lower write latency.  But the trade-off seems like it could be lost CPU cycles due to the CPU having to manage the data writes to the disk rather than just shuttling data back and forth, and letting the chipset manage the data writes to the disk.  

 

Or maybe this is really just splitting hairs?

[JorgeB]

CPU slot are faster and should be used whenever possible, no disadvantages, PCH slots will also max out @ x4 link. besides sharing the DMI with SATA ports, other PCIe slots, etc.

[koyaanisqatsi]

OK, that's what I figured, but I wanted to ask and be sure.  I'd hate to assume and find out later there's some performance issue because of it.

 

I see what you mean about shared DMI.  My block diagram shows 8GT/s DMI into the CPU from the PCH and 8GT/s PCIe which is dedicated to the storage controller (x8 on a x16, but nothing is using the other 8 lanes).

 

Thanks!

Edited July 19, 2019 by koyaanisqatsi