February 26, 20206 yr Ran with a single XFS-encrypted cache disk for a while, then moved to a 3-disk BTRFS-encrypted cache pool. Since that change, I've noticed a constant write IO on the cache disks of about 2-3MB/s (per the Main page). IOStat on the unraid host confirms the IO is happening. No other tool can give me any idea on what is doing the writing (eg: iotop points at shfs). I have 3 VMs with disks on the cache pool (that were there prior to the change). Monitoring on the VMs, iostat says there is maybe 10kB/s average write IO on each. I NFS-mount in a bunch of stuff, but monitoring nfsiostat tells me that there's not much going on there, and mostly read traffic. Since Btrfs is copy-on-write, I checked, and made sure that the VM disks, libvirt, docker, and other random-write stuff is COW-disabled (chattr +C, and verify with lsattr). With no IO coming from SMB clients, an only minor IO coming from VMs and NFS mounts, and COW disabled... how can I tell where all the extra IO is coming from? (I can account for, say, 100kB/s, but I'm seeing 3MB/s (1.5MB/s when you account for btrfs writing redundant blocks).
February 26, 20206 yr I am having a similar problem. Everything turned off, Dockers, VM, UD but still having high writes to my Crucial MX500. Spikes into the 20+MB/s. Second machine showing over all lower write number, but still higher than previously. Diags Attached tower01-diagnostics-20200226-0428.zip
February 26, 20206 yr Author 14 hours ago, johnnie.black said: Maybe same as this? Sounds similar, but I don't have any torrent clients, nor did I have any Windows machines on at the time.
February 29, 20206 yr Author Did some more testing on this, and there is definitely some write amplification going on here. I transferred everything off of the cache, and marked it as 'No' for all shares, and that dropped the write IO to zero on cache. At that point, there was reasonably constant 50-200kB/s write IO on the array (including parity). That IO only appeared on the disk that I moved the VM disks to. I moved everything except the VMs back, one at a time, and the write IO stayed on the array. Shut down the VMs, the write IO stopped, moved the disks to cache pool, set the 'No COW' flag (chattr +C) on the disks, and when I started up one, the IO was already above the 50-200kB/s that I saw as an average on the array. On cache, it's about 900kB/s-3.8MB/s ...which is about a 18-19x write amplification for being on the cache pool instead of the array.
Archived
This topic is now archived and is closed to further replies.