Hi all,

My small business recently set up an unRaid server on an old Dell Tower 7910 that we had lying around. It was running an Intel E5-2643 v3 (2014) with 128GB DDR4 ECC RAM and had a 10GbE NIC installed. My boss wanted to use an old PowerEdge R510 instead, so I moved everything over to that hardware instead. It's running an older Intel E5620 (2010) with 32GB DDR3 ECC RAM and the same 10 GbE NIC installed.

On the old hardware, I was able to write to the SMB share I created at over 500MB/s from another 10GbE-equipped workstation. Now, with the new hardware, my writes to the SMB share only max out at around 290MB/s. However, when running a LAN speed test (https://totusoft.com/lanspeed) to the same share using a 9GB packet, I am able to get write speeds over 600MB/s.

My array is made up of 7200RPM HDDs, but I have a cache pool made up of Samsung 860 EVO SSDs. Any idea what might be causing the degradation in write speed? Is my (admittedly) CPU/RAM downgrade that big of an issue where it might be causing a bottleneck? Appreciate any help you can provide!

Edited August 8, 2025Aug 8 by BWPanoramas
Removed diagnostics files for now

Start by running a single stream iperf test in both directions.

On 8/9/2025 at 1:50 AM, JorgeB said:

Start by running a single stream iperf test in both directions.

Here's the results of iperf. Any thoughts?

C:\>iPerf3 -c 192.168.10.244 -bidir

Connecting to host 192.168.10.244, port 5201

[ 5] local 192.168.10.157 port 64807 connected to 192.168.10.244 port 5201

[ ID] Interval Transfer Bitrate

[ 5] 0.00-1.01 sec 1.10 GBytes 9.35 Gbits/sec

[ 5] 1.01-2.01 sec 1.10 GBytes 9.46 Gbits/sec

[ 5] 2.01-3.01 sec 1.10 GBytes 9.48 Gbits/sec

[ 5] 3.01-4.01 sec 1.00 GBytes 8.61 Gbits/sec

[ 5] 4.01-5.01 sec 1.10 GBytes 9.47 Gbits/sec

[ 5] 5.01-6.01 sec 1.10 GBytes 9.48 Gbits/sec

[ 5] 6.01-7.02 sec 1.10 GBytes 9.42 Gbits/sec

[ 5] 7.02-8.01 sec 1.10 GBytes 9.47 Gbits/sec

[ 5] 8.01-9.01 sec 1.10 GBytes 9.46 Gbits/sec

[ 5] 9.01-10.01 sec 1.11 GBytes 9.48 Gbits/sec

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval Transfer Bitrate

[ 5] 0.00-10.01 sec 10.9 GBytes 9.37 Gbits/sec sender

[ 5] 0.00-10.02 sec 10.9 GBytes 9.37 Gbits/sec receiver

That looks OK, but I prefer to run it in one direction then the other, in separate tests, use -R for reverse.

42 minutes ago, JorgeB said:

That looks OK, but I prefer to run it in one direction then the other, in separate tests, use -R for reverse.

Here it is using separate tests:

C:\>iPerf3 -c 192.168.10.244

Connecting to host 192.168.10.244, port 5201

[ 5] local 192.168.10.157 port 49363 connected to 192.168.10.244 port 5201

[ ID] Interval Transfer Bitrate

[ 5] 0.00-1.00 sec 1.10 GBytes 9.47 Gbits/sec

[ 5] 1.00-2.00 sec 1.10 GBytes 9.44 Gbits/sec

[ 5] 2.00-3.00 sec 1.11 GBytes 9.50 Gbits/sec

[ 5] 3.00-4.01 sec 1.11 GBytes 9.49 Gbits/sec

[ 5] 4.01-5.01 sec 1.11 GBytes 9.48 Gbits/sec

[ 5] 5.01-6.01 sec 1.11 GBytes 9.48 Gbits/sec

[ 5] 6.01-7.01 sec 1.10 GBytes 9.43 Gbits/sec

[ 5] 7.01-8.01 sec 1.11 GBytes 9.50 Gbits/sec

[ 5] 8.01-9.01 sec 1.10 GBytes 9.46 Gbits/sec

[ 5] 9.01-10.01 sec 1.11 GBytes 9.49 Gbits/sec

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval Transfer Bitrate

[ 5] 0.00-10.01 sec 11.0 GBytes 9.47 Gbits/sec sender

[ 5] 0.00-10.02 sec 11.0 GBytes 9.47 Gbits/sec receiver

iperf Done.

C:\>iPerf3 -c 192.168.10.244 -R

Connecting to host 192.168.10.244, port 5201

Reverse mode, remote host 192.168.10.244 is sending

[ 5] local 192.168.10.157 port 49374 connected to 192.168.10.244 port 5201

[ ID] Interval Transfer Bitrate

[ 5] 0.00-1.01 sec 1.11 GBytes 9.39 Gbits/sec

[ 5] 1.01-2.02 sec 1.11 GBytes 9.48 Gbits/sec

[ 5] 2.02-3.00 sec 1.09 GBytes 9.49 Gbits/sec

[ 5] 3.00-4.00 sec 1.11 GBytes 9.49 Gbits/sec

[ 5] 4.00-5.00 sec 1.11 GBytes 9.49 Gbits/sec

[ 5] 5.00-6.01 sec 1.11 GBytes 9.49 Gbits/sec

[ 5] 6.01-7.01 sec 1.10 GBytes 9.48 Gbits/sec

[ 5] 7.01-8.01 sec 1.11 GBytes 9.50 Gbits/sec

[ 5] 8.01-9.01 sec 1.11 GBytes 9.49 Gbits/sec

[ 5] 9.01-10.01 sec 1.10 GBytes 9.49 Gbits/sec

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval Transfer Bitrate Retr

[ 5] 0.00-10.01 sec 11.0 GBytes 9.48 Gbits/sec 0 sender

[ 5] 0.00-10.01 sec 11.0 GBytes 9.48 Gbits/sec receiver

iperf Done.

That looks great. Please post a Windows Explorer transfer graph for one or more large files; that can sometimes give a clue on what the issue is.

21 minutes ago, JorgeB said:

That looks great. Please post a Windows Explorer transfer graph for one or more large files; that can sometimes give a clue on what the issue is.

This is the transfer of a 13.5GB .psb file.

Transfer to RACKNAS (unRAID)

Transfer to our other NAS (not unRAID)

Unfortunately, I don't have a transfer graph for the original unRAID hardware, which showed transfer speeds over 500MB/s.

Edited August 11, 2025Aug 11 by BWPanoramas
Added context

Both of those suggest a LAN issue, despite the good iperf results; see the examples below:

1st one shows the expected line speed for 10GbE, the 2nd one shows a LAN issue, and the 3rd one shows a device limit, since the initial transfer is fast, while it's being cached to RAM, meaning the LAN is OK, then it's limited by the actual device write speed, since you are seeing slow speed from the start, it suggests a LAN problem, assuming the source device is fast enough, of course.

1 hour ago, JorgeB said:

Both of those suggest a LAN issue, despite the good iperf results; see the examples below:

1st one shows the expected line speed for 10GbE, the 2nd one shows a LAN issue, and the 3rd one shows a device limit, since the initial transfer is fast, while it's being cached to RAM, meaning the LAN is OK, then it's limited by the actual device write speed, since you are seeing slow speed from the start, it suggests a LAN problem, assuming the source device is fast enough, of course.

The files are being written to the cache pool, which is a pair of Samsung 860 EVO SATA SSDs. Those should be capable of hitting 500+ MB/s, and they were on the old hardware. But they wouldn't be able to max out the bandwidth of a 10GbE connection. The three major differences I can see between the old hardware and the new hardware are: CPU, RAM, SAS controller. The old hardware had a much newer CPU (E5-2643 v3 vs. E5620), 4x as much RAM (128GB DDR4 vs. 32GB DDR3), and an LSI3008 SAS controller. The controller in the new hardware is an older Dell PERC H200 running in IT mode. Could any of those be the culprit?

Edited August 11, 2025Aug 11 by BWPanoramas

Assuming the source is fast enough, try writing directly to a disk share (or create a cache exclusive share) to see the difference.

6 hours ago, JorgeB said:

Assuming the source is fast enough, try writing directly to a disk share (or create a cache exclusive share) to see the difference.

I set up a cache only share and the result was the same. Usually hovers around 290 MB/s. The source disk is a RAID 10 array made up of Samsung 990 Pro NVMe SSDs, so it should have no problem saturating a SATA SSD I would think. I'm stumped :(

4 minutes ago, BWPanoramas said:

I set up a cache only share and the result was the same.

But did you set it to exclusive? Or used the disk path? If not it will still go through FUSE.

25 minutes ago, JorgeB said:

But did you set it to exclusive? Or used the disk path? If not it will still go through FUSE.

The share was set to primary: Cache, secondary: none. That makes it exclusive, correct?

EDIT: Nevermind, Permit Exclusive Shares is set to No in Global Share Settings, so that must not be enough. I'll stop the array, enable that setting, and try again.

Edited August 12, 2025Aug 12 by BWPanoramas

Not necessarily; you need to enable exclusive shares, they are disabled by default, and confirm it's enabled for that share:

Settings - Global Share Settings - Permit Exclusive Shares

Then confirm for that share:

15 minutes ago, JorgeB said:

Not necessarily; you need to enable exclusive shares, they are disabled by default, and confirm it's enabled for that share:

Settings - Global Share Settings - Permit Exclusive Shares

Then confirm for that share:

Updated setting and made the cache exclusive share. Confirmed exclusive access was yes. Here is the Windows transfer chart now:

Initial speeds were over 600MB/s, so we're definitely heading in the right direction. I assume this involves the pseudo-SLC cache on the 860 EVO, which is 22GB and improves initial write speed. However, the file is only 14GB, so I'm not sure why it wouldn't be able to sustain the increased write speed for the whole file.

The graph now suggests a device limit, 300 MB/s sustained write speed for the 860 EVO is about right in my experience.

4 minutes ago, JorgeB said:

The graph now suggests a device limit, 300 MB/s sustained write speed for the 860 EVO is about right in my experience.

Any idea why the old NAS using the same storage hardware would be able to sustain the increased write speeds for longer? Is it a RAM issue?

The pseudo SLC cache on the EVOs will vary with the space used; maybe it's smaller at the moment, or needs a TRIM, but the pseudo cache is not very large on those devices, so those speeds looks good to me, it's what I saw with the same devices.

1 minute ago, JorgeB said:

The pseudo SLC cache on the EVOs will vary with the space used; maybe it's smaller at the moment, or needs a TRIM, but the pseudo cache is not very large on those devices, so those speeds looks good to me, it's what I saw with the same devices.

OK, appreciate your help with this. Did the PRO versions of the SSDs have the same limitations re: write speed?

Not sure, don't really have experience with those, but if you need more than 300MB/s sustained, I would recommend using NVMe devices, and good quality ones, since they also have a pseudo cache.