My server consists of a Ryzen 5 3600, Asus B550 Prime MB, 64gb ram, 2 M.2 2tb SSDs on the MB, and 12 total HDD. 4 are running off the onboard SATA, and 8 are running off of an LSI SAS2008. I just finished up installing two new 2tb m.2 drives cache drive and configuring them in a zfs mirror cache pool. I have an array of 8 drive with one parity. Now that I have my new cache pool up and running, I am planning on swapping out my two oldest 2tb array disks and installing two 12tb disks in their place and reducing the number of disks in the array. After doing that I'm planning on creating another raidz1 pool with four 2tb SSD's for a total of 8 HHDs, 4 SSDs, and 2 M.2's.
Years ago, when I built the system, I wasn't paying much attention to where and how I was hooking things up. Currently I have the parity drive along with 2 other array disks running off of the MB controller and the others on the LSI controller. Am I leaving performance on the table by having these drives split across two controllers?
Another question I've always had is when doing things like moving files from one array disk to another or parity checks. Does the LSI controller move them directly between the drive without passing them up and then back down the PCI lane?