sdamaged Posted March 5, 2017 Share Posted March 5, 2017 Copying a 30GB movie as a test to my cache pool consisting of the following:- 250GB Samsung 840 250GB Samsung 850 256GB Samsung 850 Pro gigabit network, unRAID server has a link aggregation group with a quad port Intel gigabit network card. Transfers from my desktop PC (i7 6700k, gigabit, ssd etc) - File transfers start quickly, but will plummet to below 5MB/s and then jump up to 60MB, then back down Caching is enabled on the share (all shares in fact) Something isn't good, as my lounge Kodi Intel NUC (gigabit) will also show "buffering" when i copy files to my unRAID server, causing me to have to pause transfers No idea what is going on, the infrastructure i have is pretty good for a home setup TRIM plugin is enabled and runs daily Any ideas? thanks Quote Link to comment
trurl Posted March 5, 2017 Share Posted March 5, 2017 Simplify the network setup on your server and see how it does with just one port. Quote Link to comment
sdamaged Posted March 10, 2017 Author Share Posted March 10, 2017 Ok i'm now using a single NIC with bonding etc disabled on both unraid and the switch Writes to cache drive over the network go between 8-90MB/s averaging around 15MB/s performance is pretty garbage Quote Link to comment
JorgeB Posted March 10, 2017 Share Posted March 10, 2017 Run a write speed test on the cache pool to check if the issue is or not network related, copy the attached script to root of your flash disk and run it like so: /boot/write_speed_test.sh /mnt/cache/test.dat write_speed_test.sh Quote Link to comment
sdamaged Posted March 10, 2017 Author Share Posted March 10, 2017 (edited) Thanks a lot johnnie, much appreciated! How do these speeds seem to you? /boot/write_speed_test.sh /mnt/cache/test.dat writing 10240000000 bytes to: /mnt/cache/test.dat 1401928+0 records in 1401928+0 records out 1435574272 bytes (1.4 GB, 1.3 GiB) copied, 5.00053 s, 287 MB/s 2731752+0 records in 2731752+0 records out 2797314048 bytes (2.8 GB, 2.6 GiB) copied, 10.0038 s, 280 MB/s 3898320+0 records in 3898320+0 records out 3991879680 bytes (4.0 GB, 3.7 GiB) copied, 52.443 s, 76.1 MB/s 4490443+0 records in 4490443+0 records out 4598213632 bytes (4.6 GB, 4.3 GiB) copied, 55.0356 s, 83.5 MB/s 5739534+0 records in 5739534+0 records out 5877282816 bytes (5.9 GB, 5.5 GiB) copied, 60.0389 s, 97.9 MB/s 6891683+0 records in 6891683+0 records out 7057083392 bytes (7.1 GB, 6.6 GiB) copied, 65.0424 s, 108 MB/s 7492912+0 records in 7492912+0 records out 7672741888 bytes (7.7 GB, 7.1 GiB) copied, 222.807 s, 34.4 MB/s 8015470+0 records in 8015470+0 records out 8207841280 bytes (8.2 GB, 7.6 GiB) copied, 225.149 s, 36.5 MB/s 9268564+0 records in 9268564+0 records out 9491009536 bytes (9.5 GB, 8.8 GiB) copied, 230.152 s, 41.2 MB/s 10000000+0 records in 10000000+0 records out 10240000000 bytes (10 GB, 9.5 GiB) copied, 233.37 s, 43.9 MB/s write complete, syncing removed '/mnt/cache/test.dat' [spoiler/] Edited March 10, 2017 by sdamaged Quote Link to comment
JorgeB Posted March 10, 2017 Share Posted March 10, 2017 (edited) You should also post your diags like trurl asked, but if there's nothing there it's slow, looks like the speeds I see on untrimmed SSDs. Are you sure trim is working? Did you check the log, it doesn't work on some controllers, notably LSI2008. If trim is working I would move everything from the cache pool and start over with one SSD only, after running blkdiscard so it's completely clear and trim (or lack there of) is not a factor, if speeds are normal add the other SSDs one at a time after also clearing them. Edited March 10, 2017 by johnnie.black Quote Link to comment
sdamaged Posted March 10, 2017 Author Share Posted March 10, 2017 Thanks Johnnie Well i have the trim plugin enabled and running every day The two raid controllers i have are :- 1 x Del Perc H200 flashed to LSI 9211-8i 1 x LSI 9211-8i How do i find out if TRIM is actually working? (As i agree the speeds are indicative of an SSD that does not have TRIM enabled If it is indeed not working, maybe i should move the SSDs from the hot swap bays in my server to normal SATA ports. Bit of a pain, but can be done if necessary Many thanks for helping ba-unraid-diagnostics-20170310-2325.zip Quote Link to comment
JorgeB Posted March 10, 2017 Share Posted March 10, 2017 Both those controllers use the LSI2008 chipset, I don't see any trim info on your syslog, it should look like this: Mar 8 05:00:41 Tower7 root: /mnt/cache: 696.7 GiB (748054454272 bytes) trimmed Change all 3 SSDs to the onboard controller and run trim manually: fstrim -v /mnt/cache And re-check speeds. Quote Link to comment
sdamaged Posted March 10, 2017 Author Share Posted March 10, 2017 OK cool, tomorrow i will change them to the onboard slots and run it again thanks a lot for the advice and assistance, and ill feed back asap Quote Link to comment
machineshake123 Posted March 11, 2017 Share Posted March 11, 2017 I have the exact same problem, I have a Samsung 850 Evo (500GB) and a 960 Evo (250GB), they are not in a cache pool, I have been testing them and my speeds are those of an untrimmed drive. I have ruled out all other factors like network etc. If I run the trim manually it says xyz bytes trimmed, however if I run the trip again exactly after the command it says the same xyz bytes trimmed, shouldnt it say 0 bytes as the previous trim should have trimmed? Quote Link to comment
JorgeB Posted March 11, 2017 Share Posted March 11, 2017 13 minutes ago, machineshake123 said: I have the exact same problem, I have a Samsung 850 Evo (500GB) and a 960 Evo (250GB) I just started noticing over the last week very slow writing speeds to my NVMe device, I suspect that currently trim is not working with them, but I'll need to do some testing over the weekend to confirm, but it is working with normal SSD devices. Quote Link to comment
machineshake123 Posted March 11, 2017 Share Posted March 11, 2017 I noticed something interesting, if I run the speed test on /mnt/cache I get around 500MB/s for the SSDs, however if I run the test on a user share using the cache (/mnt/user/Movies), my speeds are around 40MB/s. I have triple checked that the share is using Cache and I have also checked on the drive to see if the item was indeed written to the cache SSD not the array, this doesn't make any sense. Quote Link to comment
mikeyosm Posted May 8, 2017 Share Posted May 8, 2017 On 11/03/2017 at 7:38 AM, johnnie.black said: I just started noticing over the last week very slow writing speeds to my NVMe device, I suspect that currently trim is not working with them, but I'll need to do some testing over the weekend to confirm, but it is working with normal SSD devices. JB, are you still getting slow writes on your nvme? I am now running Ryzen and only getting 600MB/s writes according to the write speed test script. I used to get 1.5Gb/s on my x99 rig. I checked that the m.2 slot or nvme drive isn't throttling - seems fine when testing speed on Windows 10 and Magician which leads me to UNRAID being the possible issue. My nvme drive is being used as an array device with a bunch of vms on it. Any clues? Quote Link to comment
JorgeB Posted May 8, 2017 Share Posted May 8, 2017 1 minute ago, mikeyosm said: JB, are you still getting slow writes on your nvme? I am now running Ryzen and only getting 600MB/s writes according to the write speed test script. I used to get 1.5Gb/s on my x99 rig. I checked that the m.2 slot or nvme drive isn't throttling - seems fine when testing speed on Windows 10 and Magician which leads me to UNRAID being the possible issue. My nvme drive is being used as an array device with a bunch of vms on it. Any clues? Nope, don't know what the problem was at the time but now I'm getting >900MB/s using 10GbE, didn't test locally. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.