Very slow cache write

[Redspeed93]

Previously I had 2 SATA SSD's in BTRFS RAID 1 as my cache and performance was very slow so I decided to add 2 more SSDs and change to BTRFS RAID 5, however write speeds still seem to be abysmal.

 

To confirm that I didn't just have unrealistic expectations I tried writing an identical folder to first my Unraid cache and then a Windows 10 PC with a singular budget NVME SSD.

 

It took 65 second to write the folder to the Unraid Raid 5 cache and 37 seconds to write the folder to the Windows 10 PC with the lone budget NVME SSD.

 

Looking at the transfers themselves Windows reports that the part of the transfer dealing with thousands of smaller files the Unraid cache slows down to a grinding halt with transfer speeds of just a few hundred kilbytes per second, while the cheapo NVME SSD manages to stay above at least 1 MB/s at all times.

 

I know Windows transfer window is just a guesstimate but giving the total transfer time it seems to be accurate.

 

Surely 4 SATA SSD's working together in RAID 5 can't be H A L F the speed of a cheap, low capacity NVME SSD? Surely...?

vault-diagnostics-20200725-2338.zip

Edited July 25, 2020 by Redspeed93

[JorgeB]

Small files are not ideal to test speed, is speed normal with large files? You can also try the new beta, it aligns SSDs on the 1MiB boundary (requires reformatting the pool) and that usually results in better performance.

 

Example of a transfer of 3 large files totaling about 15GB, destinations is a raid5 pool with 5 cheap 120GB SSDs:

 

 

 

[Redspeed93]

4 hours ago, johnnie.black said:

Small files are not ideal to test speed, is speed normal with large files? You can also try the new beta, it aligns SSDs on the 1MiB boundary (requires reformatting the pool) and that usually results in better performance.

 

Example of a transfer of 3 large files totaling about 15GB, destinations is a raid5 pool with 5 cheap 120GB SSDs:

 

 

 

I would disagree completely. If the point of the cache was simply to receive large files I might as well not have a cache or just use HDDs as cache. The entire point of having a cache and using SSDs, at least for me, is to avoid the slow write speed that I would otherwise encounter with HDDs when writing many smaller files.

 

While I don't have any data I also feel like the write speeds now, after having added 2 more SSDs to the cache, is basically identical to what it was previously when only 2 SSDs were used in RAID 1, so that adds further to my belief that there is either a configuration error or a bug.

[JorgeB]

I didn't say you can't copy small files, but small files will always be much slower, does the pool perform normally with large files?

 

Also try this:

7 hours ago, johnnie.black said:

You can also try the new beta, it aligns SSDs on the 1MiB boundary