June 29Jun 29 Symptom:- Copying from Windows to \\server\multimedia\... is inconsistent: ~80–160 MB/s (sometimes ~100 MB/s),even though files ultimately land on the NVMe cache pool (btrfs) and the link is 2.5GbE.- Depending on the path/folder, throughput can vary (sometimes I see ~220–230 MB/s in deeper subfolders).- Copying from the server to Windows is usually ~270 MB/s, so the issue seems specific to writes to the server.Environment:- Unraid 7.3.1 (same behavior observed on older Unraid versions as well)- Cache pool: NVMe (btrfs)- CPU: i5-12500, RAM: 48GB- NIC: Realtek RTL8125B 2.5GbE; tested r8169 and currently using r8125 (OOT driver)- MOBO: MSI MAG B660M MORTAR DDR4- Bonding: bond0 active-backup (fault-tolerance), primary+active slave eth0 @ 2500 Mbps full duplex- Windows client: Windows 10Hard facts / observations:1) The "multimedia" share is not “exclusive” - it exists on multiple array disks (union via shfs/FUSE):/mnt/disk1/multimedia//mnt/disk2/multimedia//mnt/disk3/multimedia/Example: the destination folder "Filmy Dokumentalne" exists on multiple disks at the same time:/mnt/disk1/multimedia/Filmy Dokumentalne//mnt/disk2/multimedia/Filmy Dokumentalne//mnt/disk3/multimedia/Filmy Dokumentalne/2) During slow copies, top CPU consumers are shfs and smbd (highest %CPU), suggesting a bottleneckin the user shares layer (shfs/FUSE/union) + Samba, not the NVMe itself.3) The network interface shows no errors/drops:ip -s link show eth0 -> errors=0, dropped=04) TCP retransmits increase during the copy (nstat):after nstat -r, TcpRetransSegs increases during the transfer (e.g. 272 -> 295),while TcpExtTCPTimeouts = 0.Conclusion (hypothesis):- This looks like a combination of:a) shfs (user shares/union) overhead due to the share being spread across multiple disks,b) plus TCP retransmits, which may reduce SMB write speed to ~80–100 MB/seven when the interface itself shows no errors/drops.If needed, I can post diagnostics ZIP.Additionally, I had the same kind of issues on a different platform:- Motherboard: MSI B250M MORTAR- CPU: i7-7700- RAM: 16GB DDR4 (single stick)- Storage: ~60TB HDD, 1TB NVMe, 1TB SSD Edited June 29Jun 29 by Xwint5
June 30Jun 30 Community Expert There have been other reports with those NICs and performance issues in one direction, see if this helps: https://forums.unraid.net/topic/123594-poor-ssd-cache-read-speed-25gbe/
June 30Jun 30 Author Thanks for the link JorgeB!I tried the same approach from that thread. I temporarily disabled bridging (Settings -> Network Settings -> Enable bridging = No). After applying, br0 disappeared and my server IP (192.168.0.XX) moved to bond0, with the default route also via bond0 (confirmed with: ip -br addr / ip route).Unfortunately SMB write performance to \\server\\multimedia did not improve - it still sits around ~150 MB/s (sometimes lower), so it does not seem to be the exact same br0/bridging routing issue as in that topic.For reference, my routing table already has the local subnet via br0:192.168.0.0/24 dev br0default via 192.168.0.1 dev br0I also noticed TcpRetransSegs increasing during slow copies (via nstat), while ip -s link shows 0 errors/dropped.iperf3 results (Unraid iperf3 server: iperf3 -s)Windows -> Unraid:iperf3.exe -c 192.168.0.XX -P 1 -t 30~2.33 Gbits/sec avgiperf3.exe -c 192.168.0.XX -P 4 -t 302.35 Gbits/sec sender / 2.35 Gbits/sec receiverUnraid -> Windows (-R):iperf3.exe -c 192.168.0.XX -P 4 -t 30 -R2.34 Gbits/sec sender / 2.34 Gbits/sec receiverSo raw TCP throughput over 2.5GbE looks good. SMB write to \\server\\multimedia still fluctuates around ~80-160 MB/s, which suggests the limitation is higher up the stack (SMB + shfs user share/union overhead and/or Samba behavior), not the link speed itself.
July 1Jul 1 Author results (SMB Write from Windows):Direct to cache (NVMe): ~155 MB/sDirect to disk2 (HDD): ~165 MB/s (Parity disk was active during this copy, as expected).To confirm FUSE was fully bypassed, I monitored processes while copying to disk2. smbd was working, but shfs remained at 0.0% CPU:857891 myuser 20 0 92832 20420 15480 S 21.0 0.0 0:14.25 smbd833042 root 20 0 625036 11156 1844 S 0.0 0.0 0:40.74 shfs833029 root 20 0 139356 2052 1732 S 0.0 0.0 0:00.00 shfsSince writing directly to the NVMe cache is still slow (~155 MB/s) and bypassing /mnt/user didn't improve the speed, FUSE is clearly not the bottleneck here.With iperf3 showing a solid 2.35 Gbps and SMB reads hitting ~270 MB/s, what would you suggest checking next? Could it be related to Samba settings or the Windows TCP stack?Results (SMB Write from Windows):UPDATE:Ubuntu Live USB SMB Write Results:Ubuntu 26.04 LTS - same file from same source drive, copying to Unraid via SMB:Direct to cache (MSI M450 1TB NVMe, btrfs): ~127 MB/sDirect to disk2 (HDD): ~150 MB/s Edited July 1Jul 1 by Xwint5
July 1Jul 1 Community Expert Your description makes it sound as if it could be the NVME after all that is limiting speed. Have you done local speed tests on it?
July 1Jul 1 Author iostat monitoring during 80 MB/s SMB write to cache:The NVMe drive was barely working - %util only 2.7%, w_await 2.6ms, and most of the time showing 0.00 activity across all metrics. This clearly shows the bottleneck is not the storage - the MSI M450 NVMe can handle much more but SMB write path is limiting throughput somewhere between the network and filesystem layers.If this was a DRAM-less + BTRFS issue, we'd expect high %util (50-80%) and elevated w_await times, but the drive is practically idle during slow SMB writes.Local write test results:dd if=/dev/zero of=/mnt/cache/test_speed.img bs=1M count=4000 oflag=direct status=progress4194304000 bytes (4.2 GB, 3.9 GiB) copied, 3.84098 s, 1.1 GB/sLocal write: 1.1 GB/s vs SMB write: ~150 MB/sThis confirms the NVMe and BTRFS are not the bottleneck - the issue is specifically in the SMB write path. The drive has 7-8x more capacity than SMB is utilizing.
July 1Jul 1 Community Expert If it's not FUSE there's some other bottleneck or issue; it's certainly not an Unraid limitation. I can get line speed with 25GbE (writing to an exclusive share, though FUSE it drops to around 1.8GB/s with this server)
July 1Jul 1 Author I ordered an Intel I226-V 2.5GbE PCIe yesterday and should have it in about 2 weeks. This SMB write bottleneck has been plaguing me since day one, and I've been unable to resolve it despite extensive testing.I've had this issue on two different configurations, though sometimes I do get around 230 MB/s, especially when writing to deeper directory paths. Using filezilla (FTP) gives me ~165 MB/s to the main multimedia share, but ~220-240 MB/s to deeper subdirectories, even though both locations span multiple disks in the array. Edited July 1Jul 1 by Xwint5
Tuesday at 05:18 PM1 day Author Update: hardware changes + SMB/FTP testing- Changed cache from BTRFS to XFS + docker.img --> Docker Directory- Replaced server realtek 2,5GbE NIC with Intel I226-V (2.5GbE)- Added NVMe Lexar NM790 2TB as separate pool (XFS) for testingTest: always the same .mkv file 16.7GB, Windows --> Unraid.- SMB: first copy after server and Windows restart usually ~230–270 MB/s, subsequent ones (especially to deeper folders) drop to ~80–90 MB/s- FTP: same file goes stable ~250–270 MB/s-SMB: Unraid (reading from Cache) --> Windows (writing) goes stable 275-281 MB/s. With copying from server to windows there was never a problem. - Source in Windows tested from SSD and NVMe - no impact on result- Overall in FTP Intel is faster than Realtek- Problem occurred on both cards (Realtek + Intel), so it wasn't a NIC issue Edited Tuesday at 05:45 PM1 day by Xwint5
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.