Current Setup:
Super micro server
10th Gen mobo and CPU for quicksync
1 NVME for cache
1 NVME for app/vm drive
1 Network card - 1gig (have 1 gig switches and main pc has one too). 
1 connector card for supermicro backplanes. 

Desire:

I dont game. I dont do Vms other than once in a while to test something. No regular use. 
99.9 percent of the usage is Media server or data storage/backup. 
I rip my blurays to my media server. As such move 100s of gigs at a time to the server regularly. But even good NVME drives caches fill up fast and their performance tanks. 

As such to maintain transfer speed, I got my hands on 
2x Oracle F80 WarpDrive 800GB (4x 200GB ) SSD Flash Accelerator PCIe 
1 x Intel DC P4600 2TB PCIe3.0 NVMe HHHL SSD 

Intent is to switch to a Supermicro X10SRL-F Intel LGA2011   or  a Asus X99-E WS. Both LGA 2011 with 6+ Pcie ports. (cost reasons for going the older route). 

So the plan was the 2x Oracles to be my cache drive with redundancy. And the Intel to be my Vm/App drive. 
and a Nvidia single slot GPU for transcoding since I'd loose Quicksync with older CPU Id have to get. 
So there is the 6 cards for 6 pci slots. 

Anyone have any concerns with the above? With 6 cards being in use? 
More importantly, the loss of overall CPU power by going with a LGA 2011 I5/I7/Xeon (E something)?
Again, this will be just for Media server usage and some data back up storage. 

 

And of the two Mobos, which would you got with. Prices are close enough. 

Edited November 4, 20241 yr by GuardianAli

Please don't double post in different sub-forums. It is a mess for everyone. I hid the other one.

For transfer speeds of large amounts of data (20Gigabytes) or more, you are going to run into a number of bottlenecks. First 1gigabit ethernet is only good for about 128Mbyte per second theoretically and more like 115Mbyte per second factoring in overheads. So first order of business in to upgrade the network infrastructure to 10Gigabit. With that change, your network sustainable transfer rate would be around 1000Mbytes per second. 

 

As far as drive speed, you will always be limited by the end storage drives speed. If you send a huge amount of data then the cache SSD/nvme will fill up waiting for the regualr storage HHD to accept the cached data and the transfer rate will tank to the speed of the HDD. To overcome this you would need enough SSD buffer to cover the entire transfer data amount. On the cheap you could use a smaller SSD cache drive as the total transfer time would be divided by the data injected into the ssd plus the reduced speed when cache fills and is waiting on the slower transfer to the HDD storage.  

 

Configuration of the storage drives can make a bigger difference than a cache drive in that raid configurations of many drives can increase throughput but vastly reduce redundancy. Consider a raid array with many drives (12 total) in a raid 10 configuration. This should increase sustained throughput and help with the huge data transfers. If you have more than 12 available drive interfaces the speed will continue to increase in a raid 10 configuration. 12 drive would yeild roughly 500-600MBytes write speed and nearly double on the read side. IMO I would sell the NVME cache drives as your use case is not supported by the amount of available cache. You are literally only saving the time it takes to fill a cache in the end. IE 2tb NVME to HHD transfer to write out the cache at the end. It's in the backround so you perceive the information is on the HDD. The fast corporate SANS are difficult to replicate on a budget.

 

Final advise would be to go with a purpose built NAS which consumes far less power and will preform perfectly for your intended use. For a 24/7 device power usage is a concern for most people. Remember that caching is meant for lots of smaller accessing or transfers... not for large ones.