Hi everyone,

I'm creating an image from a physical NVMe drive to another NVMe drive.

Both drives are unassigned, both are Gen4.

This is the command I use:

Quote

qemu-img convert -f raw -O qcow2 /dev/nvme8n1 /mnt/disks/SPCC1/990Pro_1tb.img

Dev 7 (direct), to Dev 2 (unassigned share):

As you can see, it transfers @ 160MB/s, should be 6GB/s at least.

When I passthrough the drives in a Win11 VM, they bench as expected, 6/7 GB/s.

Any ideas?

Thanks

Edited September 25, 20232 yr by shpitz461

I've tried a Windows 10 install, same results.

Max transfer between Gen4 NVMe drives is around 1.5GB/s.

Upon further inspection, HWInfo64 reports the drives operate as Gen2 with max link-speed of 1x (5 GT/s):

The Red Hat PCI-to-PCI bridge does show Gen4:

Power mode is set to Performance.

It seems that the HWInfo data is wrong since I'm getting higher speeds than the pcie Gen2 allow.

I've benched the drives using Samsung Magician:

I think I will also reboot the server and do some bare-metal testing outside of Unraid.

Thanks

Also, I'm copying directly via /mnt from one gen3 nvme pool to another gen4 nvme pool, both are btrfs raid-1 filesystems, and this is the speed I'm getting:

When I clear or balance the pools I get the drives rated speed.

Any idea why I'm getting 1/3rd of the speed?

Thanks

While it should probably be a little faster than that, assuming a fairly decent CPU, don't expect to get max device speed during a copy, with any OS, see for example:

 

https://superuser.com/questions/1331298/why-do-windows-file-transfer-speeds-top-out-at-around-1-5-gigabytes-second

 

The above is what I get, around 1.5GB/s.

 

 

Thanks for the reply!

I did expect a little overhead, but not a 1/3rd. I also did the transfer using Krusader within Unraid, no Windows is involved.

The CPU is an Epyc 7742.

I've tested the drive's Q1T1 performance, shows 5GB/s write:

It is what it is I guess.

Like mentioned an actual copy is not the same as running a benchmark, I have a similar CPU in one of my servers, and get max 1.5GB/s when copying data with NVMe devices, with Unraid, TrueNAS or Windows.