ZFS performance within NVMe pool


Recommended Posts

Background: I have 5x 2TB NVMe SSDs in a RaidZ1 pool (non encrypted, some datasets have compression and some do not), used as my cache.  I'm sitting between 80% and 90% utilization at any given time (8TB usable, ~1TB free). 

 

Moving large files within the pool is very slow compared to when I was running the same drives in a BTRFS Raid1 pool (5TB usable).  Moving a 5GB file from folder A to folder B within the cache gives me consistently less than 500MB/s transfer speed (usually much less), while I was seeing around 1.5GB/s on BTRFS.  

 

913611149_Screenshot2024-02-02at8_41_32AM.png.6e3391f9c6cce8089f22510fcefe49a0.png

 

Not sure what I can check here.  Could this be due to the higher pool utilization?  I have a 10th gen i7 and 48GB of RAM, and the system isn't taxed.  The PCIe bus is 3.0 and all NVMe's have DRAM.  

 

Thanks for your suggestions!

Edited by Andiroo2
Reasons
Link to comment
20 minutes ago, Andiroo2 said:

Not sure what I can check here.  Could this be due to the higher pool utilization?

What is the current pool utilization? 

 

I've moved from a btrfs raid5 pool to zfs raidz1 pool and see similar speeds, even a little faster with zfs, I used to get around 1GB/s, now get 1.5GB/s when copying data inside the pool.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.