Hi,

 

I have an unraid setup with 10GbE and a 970 Evo cache drive.  Writes to this machine max out around 650MB/s.

 

iPerf shows 950MB/s over the network.  But, diskspeed shows average writes to the NVME 970 Evo at 650MB/s, which is my limiting factor.

 

Why is this drive performing so slowly?  In a windows machine, it can write at 2.5 GB/s.  Are there settings I should change?

Edited March 8, 20251 yr by Belgian Genius
update

Try using a disk share (or exclusive share)

I tried that - no difference.

Do you have another fast device on the server that you could use to do a local test copy?

No, this is the only SSD on the server.

 

It's plugged directly into an NVME slot on the MB, so I have no doubt that it's unRAID that is somehow hobbling its performance to 650MB/S Write, but I can't figure out what is causing it.  The SSD is easily capable of 4X that performance.

It would be easier to test with another fast device, but if you have enough RAM, you can also copy a large file to RAM then do a local test with pv, e.g.:

 

pv /temp/large.file > /mnt/cache/large.file

 

And post the results

Ok., I replaced my cache drive with something new - a 1TB Crucial P3 Plus drive.

 

iPerf shows ethernet performance as ~ 950 MB/s.

 

Diskspeed now shows SSD write performance at 2.5 GB/s, so SSD bottleneck has been eliminated.

 

BUT, when I transfer a file to the unRAID over the network, I'm still getting 700MB/s max. 

 

WHAT * THE * HELL? 

 

What am I missing here?

Edited March 16, 20251 yr by Belgian Genius

700MB/s is pretty good, you may get more if you use a disk share (or exclusive share).

Agreed, but my Windows server which sits next to it on the same network writes 980MB/s.

 

I'll try the disk share.

 

UPDATE:  I made my cache an exclusive share.  Unfortunately, no difference.

Edited March 18, 20251 yr by Belgian Genius

There should be a difference, user share (FUSE) should always be slower vs native, though sometimes not by much, so something else may be interfering:

 

 

Post a screenshot of the Windows explorer graph during a large transfer, that can help show if the issue is the LAN or the device.

 

 

 

 

 

Here's a graph of transferring a 25GB file.

 

It bounces all over the place from a low of 500MB/s to a high of about 800MB/s.

 

By comparison, transfers to my Windows server on the same network are a straight line around 980MB/s.

Since the initial transfer speed is already low, it usually means the LAN, or source PC, is the issue, if it were a device issue, it would start at full speed while it's being cached to RAM, and then slow down to actual device speed.

Here are some examples to show how the Windows transfer graph can help identify where the issue is.

 

1st one shows optimal speed with 10GbE, 2nd one shows a network issue limiting the transfer, in this case caused by using a very low MTU value, last one shows a device limited transfer, it starts at 1.15GB/s while it's caching to RAM, but then it's limited by the device actual sustained write speed.