Cache pool write speed via /mnt/user vs /mnt/cache ?

[glennv]

Trying to understand where the huge speed diff comes from.

Having a btrfs raid10 4x 1TB ssd samsung evo 860's cache pool. (connected via 12G sas card)

Writing directly to it (freshly trimmed and about 20% full) via /mnt/cache/sharename/ gets me a solid 1.1 GB/s speed as expected (about 500+ per ssd divided by 2 for raid10).

Writing to the cache via a share set to cache yes or only and /mnt/user/sharename/ gets me about 50-60% of that.

 

(testing via dd if=/dev/zero of=file1.txt count=25k bs=1024k, but similar via real world data speedtests via 10G ether )

 

So what is this huge overhead as parity is not involved here and can this be tuned to get me optimised cache write speeds ?

Edited September 20, 2019 by glennv

[Squid]

Writes to /mnt/user will always be slower than directly referencing the disk because it has to go through the fuse layer.  Enabling DirectIO in disk settings may help

[glennv]

Tnx. Wow that is a lot of overhead.  

Yeah i also just remembered direct I/O , but also vaguely remembered that in the past setting that caused issues with some of my dockers. But these have been moved off to zfs pools so will try and play with it again. 

 

Edited September 20, 2019 by glennv

[Squid]

6.7.x also is using Fuse 2.x  I believe that the next version of unRaid is updating that to 3.x and has lower overhead.

[glennv]

Ah good to know. Tnx for the info.

 

Btw just finished testing direct I/O and although it does improve artificial write speeds (using dd direct on server) it does not do a lot for real world over the net write speeds, but worse , it dropped my read speeds from 900MB/s to 230MB/s , so that idea goes in the bin

 

At least i know the cause and dont keep looking for a misconfiguration that is not there.

Tnx mate. Waiting for fuse 3.x it is i guess......