Hi all,

I'm trying to diagnose persistently slow write speeds on my three Unraid servers. Despite a full 10GbE network and various hardware upgrades, my write speeds are stuck at a stable 70-80MB/s when writing directly to the array.

I'd be grateful for any insights into what I might be missing.

The issue is: when I transfer large files, the speed starts fast (approx. 215MB/s) until the server's RAM cache fills, then it drops to a consistent 70-80MB/s for the remainder of the transfer.

Test Method:

• Transferring 1TB batches of large video files (5-10GB each).

• The 500GB cache drive is disabled for these tests (since my transfers are often 10TB+).

• The transfer is from a Windows 11 PC via a "Map Network Drive" (SMB) connection.
  

Core Hardware:

• Motherboard: Supermicro X10SDV-4C+-TLN4F

• CPU: Intel Xeon D-1518

• RAM: 32GB DDR4 ECC

• Disks: 18TB Toshiba Enterprise

• HBA: Supermicro AOC-S3008L-L8e (SAS3)

• OS: Unraid (no Docker or VMs running)

Network Setup:

• Client: Windows 11 PC with a 10GbE network card.

• Servers: Using the integrated 10GbE ports.

• Infrastructure: All connected via a 10GbE switch and CAT8 cabling.

I've tried multiple upgrades to fix this, with puzzling results:

1. Mobo/CPU/RAM: Upgraded all servers from a DDR3 platform to the DDR4 setup above. (No change in write speed).

2. HBA: Replaced original SAS2 HBAs with the current SAS3 models (AOC-S3008L-L8e) on two servers.

3. Backplane: Replaced the SAS2 backplane with a SAS3 backplane on one of those servers.

4. Cabling: Swapped HBA-to-backplane cables and connection methods.

The upgrades had almost no effect on file transfer speeds (maybe a 10MB/s increase at most).

However, on the one server with both the SAS3 HBA and SAS3 backplane, the parity check speed more than doubled.

• Before: ~90MB/s average (52-54 hours)

• After: ~210MB/s average (26 hours)

This proves the new hardware is working and capable of high speeds, but it only affects parity checks, not my SMB file transfers.

Note: The server with the new backplane has 19 disks (2 parity, 16 data). The other two servers have 30 disks (2 parity, 28 data).

Unraid Settings:

• All servers have "write reconstruct enabled" (this is the md_write_method).

• Disk and share settings are identical across all servers (screenshots were attached in my original post).

I suspect this might be a software limit or an inefficient transfer method, but I'm not sure what to check next. Why would the parity check speed improve so dramatically, but not the array write speed?

Thanks for any help you can provide!

Just now, Senect said:

then it drops to a consistent 70-80MB/s for the remainder of the transfer.

That suggests the disks can't keep up, note that the disks become much slower on the inner sectors, and they are all full, though I would expect closer to 100MB/s when full.

16 minutes ago, JorgeB said:

That suggests the disks can't keep up, note that the disks become much slower on the inner sectors, and they are all full, though I would expect closer to 100MB/s when full.

Thanks for the reply, but I don't think that is the issue, as you can see this server has 2 empty disks, it's full SAS3 setup (the one that reaches 215M/s average during parity check)
And still this one is the same, the initial burst in speed is related to the RAM, because I can see the speed directly on Unraid and they are not going at 200MB/s+ initially, they have always a stable speed, windows transfer at that speed initially until the 32GB of RAM is full, so it lasts only 1m max.

31 minutes ago, Senect said:

a stable 70-80MB/s when writing directly to the array.

That is not unexpected when writing to the parity protected array due to the way Unraid handles writes and the overhead of updating the parity drive.

That is one reason the concept of a ‘cache’ drive was introduced so the initial write could happen at device speed, and the slower transfer to the array happened overnight when the system was typically idle.

If speed is a key criteria then you want to be using pools rather than the main array. This need was one of the main reasons ZFS support was added to Unraid.

56 minutes ago, Senect said:

but I don't think that is the issue, as you can see this server has 2 empty disks,

In that case something else is limiting the speed. If parity check is fast, it should not be a controller bottleneck. I don't have a parity-protected array now, but when I did, I could write at 200MB/s+, as long as I was writing to the beginning of a disk. To rule out any issue with just write speed, is a parity sync/disk rebuild write speed similar to parity check?