Oculink PCI-E 4x to NVMe Gen4?

[spamalam]

I'm looking at a second build but this time a smaller form factor. I am looking at:

https://www.asrockrack.com/general/productdetail.asp?Model=W680D4U-2L2T%2FG5#Manual

Support 3 OCuLink (PCIe4.0 x4), 1 OCuLink (PCIe4.0 x4 or 4 SATA 6Gb/s)

 

My understanding of oculink is its a pci-e expansion port, i was hoping i could have 4x gen4 nvme via oculinks but i've only been able to find gen3 hardware:

https://www.aliexpress.com/item/1005005029291938.html?spm=a2g0o.productlist.main.19.7c506d501C01Ji&algo_pvid=1329707b-6635-4711-8bbf-e30c078d014d&algo_exp_id=1329707b-6635-4711-8bbf-e30c078d014d-9&pdp_ext_f={"sku_id"%3A"12000031388265391"}&pdp_npi=3%40dis!GBP!15.7!15.7!!!!!%402145274c16776245656212959d06ee!12000031388265391!sea!GB!891441194&curPageLogUid=eDwrZWH8yb8Y

 

Does anything exist that would allow gen 4 nvme at full speed via the oculink ports? 

Edited February 28, 2023 by spamalam

[MAM59]

You should not be too demanding for 4.0 hardware. Current NVMe SSDs hardly get over 3.0 speed (when looking at "sustained access", not bursts). We recently had a test here in Germany, the good 3.0 drives were matching or even better than 4.0 ones.

And, not to forget, energy level and heat dissipation is much higher with 4.0 (thats why they often have to slow down to lower the temp).

 

So, with the current versions of the drives, it does not matter which speed you  are running. You won't miss much.

 

[scs3jb]

17 hours ago, MAM59 said:

You should not be too demanding for 4.0 hardware. Current NVMe SSDs hardly get over 3.0 speed (when looking at "sustained access", not bursts). We recently had a test here in Germany, the good 3.0 drives were matching or even better than 4.0 ones.

And, not to forget, energy level and heat dissipation is much higher with 4.0 (thats why they often have to slow down to lower the temp).

 

So, with the current versions of the drives, it does not matter which speed you  are running. You won't miss much.

 

 

This is odd, I already have a setup with an array cache of WD SN850x Gen4 which get close to 7000MB/s in real life workload copying files that fit in DRAM (3x 2TB drives in a RAID), and smaller files were lower but higher than Gen 3 spec, whilst I have WD SN850x Gen4 is a PCI-e 3x expander and they are stuck at around 3500MB/s.   

 

On top of the faster speed I've seen, the endurance rating is much higher on the current models so I think buying gen4 nvme only to cap them at lower speed seems awfully wasteful.  Endurance/Speed/Price is the balance I went for, which is why I went for the SN850x over the Samsungs (had terrible experience with EVOs).

 

Have you published some results somewhere I could take a look at?  Given everything modern is gen4, i wonder what the price/speed/endurance sweet spot is.

Edited March 2, 2023 by spamalam2

[MAM59]

7 hours ago, spamalam2 said:

Have you published some results somewhere I could take a look at?

Not me, it was in a german computer magazine called "c't" 2/2023 page 116 ff.

 

the thing is, that prices come down, 4.0 is almost the same as 3.0. But these days people are looking to save energy and 4.0 is a waste. So it is a question if you like to spend much money for electricity (the ssd and the additional cooling that will be needed) to get very few peaks of high speed. Copying files on and off the cache is more or less a one time thing (access for clients is limited by LAN speed, the SSD is not the capping factor).

 

And, yes, those EVOs are evil, not recommended for server use. But the PRO drives by Samsung are fine and usually on top of the list.

 

 

[C4RBON]

Why not use U.2 drives?  Rated for much higher endurance than consumer M.2 drives (usually 1 DWPD over 5 years), easier to mount, can be had in capacities up to 16TB, and have a built in heatsink.  You can get new "read-intensive" (still 1 DWPD) PCIE gen 4 ones for $100/TB, which isn't much of a premium over the good consumer M.2 drives.

 

Then connect them with an Oculink to SFF-8639 cable.  This one specifically claims it is for PCI-e gen 4.

https://www.amazon.com/PCIe-OCulink-SFF-8611-SFF-8639-Cable/dp/B089QPFMD2/

 

 

[scs3jb]

On 3/2/2023 at 1:19 PM, C4RBON said:

Why not use U.2 drives?  Rated for much higher endurance than consumer M.2 drives (usually 1 DWPD over 5 years), easier to mount, can be had in capacities up to 16TB, and have a built in heatsink.  You can get new "read-intensive" (still 1 DWPD) PCIE gen 4 ones for $100/TB, which isn't much of a premium over the good consumer M.2 drives.

 

Then connect them with an Oculink to SFF-8639 cable.  This one specifically claims it is for PCI-e gen 4.

https://www.amazon.com/PCIe-OCulink-SFF-8611-SFF-8639-Cable/dp/B089QPFMD2/

 

 

I went with this approach and it was a good call, sacrifice a tiny bit of speed for endurance.  

 

Thanks for the suggestion.