ChrisK2 Posted April 1, 2022 Share Posted April 1, 2022 (edited) I have set up a share that uses an SSD as cache. However, something seems to be very wrong. My writing speed to /mnt/user/share are around 25-30 Megabyte/s, even though I'm having an SSD cache and the "Use Cache Pool" setting at "Yes". I noticed that my qBittorrent was always at a maximum of 25 Megabyte/s. I have 750 Mbit/s internet and always get 500 Mbit/s+ in speed tests and reaches speeds at 70 Megabyte/s+ on my other computer. I tried to search and found out more people who seem to have had the same problem, but I couldn't find any solution. I've tried setting Tunable (md_write_method) to reconstruct write, but that doesn't affect speed. (I guess it shouldn't either, as the data is written to cache and the parity is not involved.) My server specs: Motherboard: Supermicro X12STH-LN4F CPU: Intel Xeon E-2356G Memory: 2*32 GB ECC RAM Hard drives: 3*Seagate Exos X16 16 TB 7200 RPM (1 parity, 2 data) - Data is xfs encrypted SSD cache: 1 TB Samsung 970 EVO Plus NVMe M.2 - SSD is btrfs encrypted Unraid version: 6.10.0-rc4 I ran some dd tests and these are the results: # dd if=/dev/zero of=/mnt/user/media/test-user bs=1K count=5000000 conv=fdatasync 5000000+0 records in 5000000+0 records out 5120000000 bytes (5.1 GB, 4.8 GiB) copied, 209.546 s, 24.4 MB/s # dd if=/dev/zero of=/mnt/cache/media/test-cache bs=1K count=5000000 conv=fdatasync 5000000+0 records in 5000000+0 records out 5120000000 bytes (5.1 GB, 4.8 GiB) copied, 8.50469 s, 602 MB/s # dd if=/dev/zero of=/mnt/disk1/media/test-disk1 bs=1K count=5000000 conv=fdatasync 5000000+0 records in 5000000+0 records out 5120000000 bytes (5.1 GB, 4.8 GiB) copied, 23.7756 s, 215 MB/s I've verified that the dd test to /mnt/user is written to the share, and not disk. I've attached the diagnostics, collected after I ran the dd tests. No reboot. Below are some images of my settings. Disk Settings: Disk 1: Cache: Share settings: Global Share Settings: tower-diagnostics-20220401-2004.zip Edited April 1, 2022 by ChrisK2 Formatting Quote Link to comment
ChrisK2 Posted April 1, 2022 Author Share Posted April 1, 2022 (edited) Some other topics with same/similar problem, but without any solution. Edited April 2, 2022 by ChrisK2 Quote Link to comment
ChrisK2 Posted April 1, 2022 Author Share Posted April 1, 2022 (edited) Ran a dd with Tunable (enable Direct IO) turned to yes, with no difference at all: # dd if=/dev/zero of=/mnt/user/media/test-user bs=1K count=5000000 conv=fdatasync 5000000+0 records in 5000000+0 records out 5120000000 bytes (5.1 GB, 4.8 GiB) copied, 189.436 s, 27.0 MB/s Edited April 2, 2022 by ChrisK2 Quote Link to comment
ChrisK2 Posted April 2, 2022 Author Share Posted April 2, 2022 (edited) Some more information, it seems that the shfs process is taking some CPU when doing the dd tests, if that's of any interest. Both these screenshots from when running dd to /mnt/user/. Edited April 2, 2022 by ChrisK2 Quote Link to comment
JorgeB Posted April 2, 2022 Share Posted April 2, 2022 Just a couple of observations, using dd to test /mnt/user doesn't always give relevant results, also it's known that user shares always have some extra overhead vs disk shares, some users see a much bigger difference than others, though 25MB/s would be extra slow, usually only users with 10GbE notice the difference. If you you transfer data from another PC to a share using cache do you also get 25MB/s? Quote Link to comment
ChrisK2 Posted April 2, 2022 Author Share Posted April 2, 2022 Thank you for your answe, @JorgeB! I've setup my PC and ran tests against the share and directly against the cache. Both were actually at 96-98 Megabyte/s, so the problem doesn't seem to be the cache, after all. I also tried local dd, with larger block size, and that seemed to increase the speed as well. Can it be that the small block size, when running locally, increases I/O? And that's hard for the shfs to handle? However, I still seem to be limited to 25-30 Megabyte/s in all torrent clients I've tried. qBittorrent, rTorrent, Deluge. From my other PC, I reach speeds at 60 Megabyte/s+ with no problem. Could it be that the torrent clients has high I/O, and that, as in the dd with a block size of 1kB, leads to poor performance? Quote Link to comment
JorgeB Posted April 3, 2022 Share Posted April 3, 2022 20 hours ago, ChrisK2 said: I also tried local dd, with larger block size, and that seemed to increase the speed as well. Can it be that the small block size, when running locally, increases I/O? Like mentioned dd is not a reliable way to test that. 20 hours ago, ChrisK2 said: However, I still seem to be limited to 25-30 Megabyte/s in all torrent clients I've tried. qBittorrent, rTorrent, Deluge. Sorry, can't really help with that. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.