Copying a 30GB movie as a test to my cache pool consisting of the following:-

 

250GB Samsung 840

250GB Samsung 850

256GB Samsung 850 Pro

 

gigabit network, unRAID server has a link aggregation group with a quad port Intel gigabit network card.

 

Transfers from my desktop PC (i7 6700k, gigabit, ssd etc) - File transfers start quickly, but will plummet to below 5MB/s and then jump up to 60MB, then back down

 

Caching is enabled on the share (all shares in fact)

 

Something isn't good, as my lounge Kodi Intel NUC (gigabit) will also show "buffering" when i copy files to my unRAID server, causing me to have to pause transfers

 

No idea what is going on, the infrastructure i have is pretty good for a home setup

 

TRIM plugin is enabled and runs daily

 

Any ideas?

thanks

Simplify the network setup on your server and see how it does with just one port.

Ok i'm now using a single NIC with bonding etc disabled on both unraid and the switch

 

Writes to cache drive over the network go between 8-90MB/s averaging around 15MB/s

 

performance is pretty garbage

Post diagnostics

Run a write speed test on the cache pool to check if the issue is or not network related, copy the attached script to root of your flash disk and run it like so:

 

/boot/write_speed_test.sh /mnt/cache/test.dat

 

write_speed_test.sh

Thanks a lot johnnie, much appreciated!

 

How do these speeds seem to you?

 

 

/boot/write_speed_test.sh /mnt/cache/test.dat
writing 10240000000 bytes to: /mnt/cache/test.dat
1401928+0 records in
1401928+0 records out
1435574272 bytes (1.4 GB, 1.3 GiB) copied, 5.00053 s, 287 MB/s
2731752+0 records in
2731752+0 records out
2797314048 bytes (2.8 GB, 2.6 GiB) copied, 10.0038 s, 280 MB/s
3898320+0 records in
3898320+0 records out
3991879680 bytes (4.0 GB, 3.7 GiB) copied, 52.443 s, 76.1 MB/s
4490443+0 records in
4490443+0 records out
4598213632 bytes (4.6 GB, 4.3 GiB) copied, 55.0356 s, 83.5 MB/s
5739534+0 records in
5739534+0 records out
5877282816 bytes (5.9 GB, 5.5 GiB) copied, 60.0389 s, 97.9 MB/s
6891683+0 records in
6891683+0 records out
7057083392 bytes (7.1 GB, 6.6 GiB) copied, 65.0424 s, 108 MB/s
7492912+0 records in
7492912+0 records out
7672741888 bytes (7.7 GB, 7.1 GiB) copied, 222.807 s, 34.4 MB/s
8015470+0 records in
8015470+0 records out
8207841280 bytes (8.2 GB, 7.6 GiB) copied, 225.149 s, 36.5 MB/s
9268564+0 records in
9268564+0 records out
9491009536 bytes (9.5 GB, 8.8 GiB) copied, 230.152 s, 41.2 MB/s
10000000+0 records in
10000000+0 records out
10240000000 bytes (10 GB, 9.5 GiB) copied, 233.37 s, 43.9 MB/s
write complete, syncing
removed '/mnt/cache/test.dat'
[spoiler/]

Edited March 10, 20179 yr by sdamaged

You should also post your diags like trurl asked, but if there's nothing there it's slow, looks like the speeds I see on untrimmed SSDs.

Are you sure trim is working? Did you check the log, it doesn't work on some controllers, notably LSI2008.

 

If trim is working I would move everything from the cache pool and start over with one SSD only, after running blkdiscard so it's completely clear and trim (or lack there of) is not a factor, if speeds are normal add the other SSDs one at a time after also clearing them.

Edited March 10, 20179 yr by johnnie.black

Thanks Johnnie

 

Well i have the trim plugin enabled and running every day

 

The two raid controllers i have are :-

 

1 x Del Perc H200 flashed to LSI 9211-8i

1 x LSI 9211-8i

 

How do i find out if TRIM is actually working?  (As i agree the speeds are indicative of an SSD that does not have TRIM enabled

 

If it is indeed not working, maybe i should move the SSDs from the hot swap bays in my server to normal SATA ports. Bit of a pain, but can be done if necessary

 

Many thanks for helping

 

ba-unraid-diagnostics-20170310-2325.zip

Both those controllers use the LSI2008 chipset, I don't see any trim info on your syslog, it should look like this:

 

Mar 8 05:00:41 Tower7 root: /mnt/cache: 696.7 GiB (748054454272 bytes) trimmed

Change all 3 SSDs to the onboard controller and run trim manually:

 

fstrim -v /mnt/cache

 

And re-check speeds.

OK cool, tomorrow i will change them to the onboard slots and run it again

 

thanks a lot for the advice and assistance, and ill feed back asap

I have the exact same problem, I have a Samsung 850 Evo (500GB) and a 960 Evo (250GB), they are not in a cache pool, I have been testing them and my speeds are those of an untrimmed drive. I have ruled out all other factors like network etc. If I run the trim manually it says xyz bytes trimmed, however if I run the trip again exactly after the command it says the same xyz bytes trimmed, shouldnt it say 0 bytes as the previous trim should have trimmed? 

13 minutes ago, machineshake123 said:

I have the exact same problem, I have a Samsung 850 Evo (500GB) and a 960 Evo (250GB)

 

I just started noticing over the last week very slow writing speeds to my NVMe device, I suspect that currently trim is not working with them, but I'll need to do some testing over the weekend to confirm, but it is working with normal SSD devices.

I noticed something interesting, if I run the speed test on /mnt/cache I get around 500MB/s for the SSDs, however if I run the test on a user share using the cache (/mnt/user/Movies), my speeds are around 40MB/s. I have triple checked that the share is using Cache and I have also checked on the drive to see if the item was indeed written to the cache SSD not the array, this doesn't make any sense.

On 11/03/2017 at 7:38 AM, johnnie.black said:

 

I just started noticing over the last week very slow writing speeds to my NVMe device, I suspect that currently trim is not working with them, but I'll need to do some testing over the weekend to confirm, but it is working with normal SSD devices.

 

JB, are you still getting slow writes on your nvme? I am now running Ryzen and only getting 600MB/s writes according to the write speed test script. I used to get 1.5Gb/s on my x99 rig. I checked that the m.2 slot or nvme drive isn't throttling - seems fine when testing speed on Windows 10 and Magician which leads me to UNRAID being the possible issue. My nvme drive is being used as an array device with a bunch of vms on it. Any clues?

1 minute ago, mikeyosm said:

 

JB, are you still getting slow writes on your nvme? I am now running Ryzen and only getting 600MB/s writes according to the write speed test script. I used to get 1.5Gb/s on my x99 rig. I checked that the m.2 slot or nvme drive isn't throttling - seems fine when testing speed on Windows 10 and Magician which leads me to UNRAID being the possible issue. My nvme drive is being used as an array device with a bunch of vms on it. Any clues?

 

Nope, don't know what the problem was at the time but now I'm getting >900MB/s using 10GbE, didn't test locally.