Would like to understand this SMB behavior in a file transfer using FreeFileSync on a Windows 11 PC to an Unraid Share.  The share uses reconstruct write and writes directly to array, no cache is used.  The file transfer (example is using larger files so that small file size does not skew the result) starts at 120Mbs and gradually drops to 0, climbs back up and repeats this behavior.  

 

Using htop, on both the main and io tabs, the behavior is also shown, logs do not seem to indicate any disk or network issues (though I may have missed something)  

 

I assume there is some type of cache or resource exhaustion issue but I do not understand it.  Attached are clips of the htop screens, the file transfer screen on the Windows 11 PC, and a diagnostic with my logs.  I attached clips so the drop and recover process could be shown dynamically

 

You can see the drop to 0 at 1.20 mark on the FreeFileSync MP4

 

Any information or insight appreciated.

unraidtower-diagnostics-20241117-1241.zip

many factors form netowring equipemnt inbetween and hard disk cache read write...

This behavior during file transfers using SMB (Server Message Block) on Unraid shares can be attributed to a variety of factors, especially when using reconstruct write mode (commonly referred to as "turbo write"). Let's break it down and understand potential causes and remedies:

 

1. Reconstruct Write and Disk I/O Bottlenecks

Reconstruct write mode writes data directly to the parity-protected array without a cache. It calculates parity dynamically for every write, requiring all array drives (data and parity) to be spun up. This can cause:

Write bursts: Data is written as fast as the slowest drive in the array and parity calculation allows.

Pauses during parity updates: Once the system buffers fill up, writes may pause momentarily to allow drives to sync parity data.

The cyclical nature of the speeds (high to 0 and back) aligns with buffer exhaustion and reallocation.

 

2. Buffer Caching in SMB

Windows SMB transfers often leverage system memory as a buffer. However, if:

The buffer fills up faster than Unraid can flush data to the drives, you'll see a pause.

Once Unraid clears enough space in its buffers, transfers resume at high speeds until the next cycle.

 

3. Disk Fragmentation or Individual Drive Performance

While the array doesn't use a traditional file system like NTFS, fragmented or heavily used drives can slow down writes. This is especially true for slower spinning drives (e.g., 5400 RPM).

 

4. Networking and Protocol Overhead

SMB is a chatty protocol, which can exacerbate bottlenecks when coupled with:

CPU load on the Unraid server during parity calculation.

Latency or bandwidth limitations in the network (e.g., between the Windows PC and the server).

If you're using a 1 Gbps network, 120 MB/s (close to 1 Gbps theoretical limit) represents a fully saturated connection..

 

5. FreeFileSync Behavior

FreeFileSync may impose its own transfer logic or chunk-based copying strategy. For large files, it tries to maximize throughput, which could interact with the SMB caching system in ways that amplify the pauses.

 

Diagnostics to Check

To narrow down the issue:

Monitor Disk I/O in Detail: Use Unraid's disk activity monitors or iostat via the terminal to see if a specific drive is slowing down.

Check Network Stats: Use tools like iperf to ensure your network isn't introducing delays or inconsistencies.

Analyze System Load: htop suggests system resource exhaustion. Check CPU usage for spikes, especially related to the smbd process or parity calculations.

Examine Disk Health: Run a SMART test on all drives to identify any latent hardware issues.

 

SMB extra options to edit / tune:
 

socket options = TCP_NODELAY IPTOS_LOWDELAY
aio read size = 1
aio write size = 1

 

Are you copying to samba or off samba

 

Thanks for the detailed response.  I should have mentioned that I run iperf docker on my unraid and have a 10gb network.  I tested that part prior to posting and the throughput is over 700mbs.  I run automated periodic smart tests on all my drives, checked that but perhaps I missed something so I will look at it again.  Can you expand a little on resource exhaustion you mention?   I will look closer at the htop stats over a longer period of time.  Appreciate your time and comments.

One more thing is the file overhead with the creations of many small files.  (And it looked like that was what was happening in the motion capture video.)  Head movements are time killers. 

 

SMR hard drives are another issue that can effect write times as massive numbers tracks can require a total rewrite of all of the data while the data size of the original write will fit on a single track.  (Some SMR drives handle this situation better than others but it is still there in the end.)

3 hours ago, rwilkers said:

Thanks for the detailed response.  I should have mentioned that I run iperf docker on my unraid and have a 10gb network.  I tested that part prior to posting and the throughput is over 700mbs.  I run automated periodic smart tests on all my drives, checked that but perhaps I missed something so I will look at it again.  Can you expand a little on resource exhaustion you mention?   I will look closer at the htop stats over a longer period of time.  Appreciate your time and comments.

not realy it more watching if your cpu is peged when the copy begins in the old pentium p1 days, everything had to go through the processor. simalry things happen in conatiner and it more watching that the host isn't peged for doing to mush htop give a nice top view simlar to windows task manger.