My configuration:
hp microserver gen8
cpu: e3-1265l v2
memory: 16g ddr3l
storage: WDC_WD10EZEX(1t hdd) x4, 2 x data, 2 x parity
WDC_WDS120G2G0A(120g ssd) x1, as cache
way to produce this problem:
I use Dynamix System Statistics to monitor the memory utilization and disk stats.
I use command below to test write speed.
dd if=/dev/zero of=/mnt/user/test/test.dbf bs=8k count=800000 conv=fdatasync
I use command below to test read speed.
dd if=/mnt/user/test/test.dbf of=/dev/null bs=8k count=800000
I have two shared folders, /mnt/user/test1 has cache set to No, and /mnt/user/test2 has cache set to Prefer.(write to SSD first)
Both of these partitions are btrfs.
I found that:
a) When the memory is enough
Both of them have 600MBps read and 600MBps write.
b) When the memory is fully occupied for cache
test1 has 170MBps read, test2 has 300MBps read
the write speed is werid.
test1 has only 40MBps write, test2 has 70MBps write
Dynamix System Statistics says that the size of memory cache is the same, but disk has both read and write at 100MBps.
I thought it might be a problem of the memory cache.
When the memory is full and new files are writing, it writes to both the memory for caching and the HDD.
As the memory is full, out-dated memory cache writes to HDD, which takes up half of the HDD write speed.
However, as the file is too large, the memory cache is useless(not going to be used), instead it write back to the HDD again, which causing the low writing performance.
In my opinion, memory cache may be the source of the problem, thus memory cache should be stop, or manually stop on this occassion.
This didn't happened on other OS on my server.