[[6.8.3] docker image huge amount of unnecessary writes on cache]

Two things I have found.

1. If your VM images are in a share with Copy on Write set to Auto, create a new share with this disabled, and move/copy the images to the new share.  Update your VMs to point to the new images in the new share.  This seems to reduce the write rate considerably.  I made these changes on all 3 VMs, and 2 out of the 3 reduced to practically nothing for an idle machine, as you'd expect.
2. The 3rd VM was still stubbornly writing out at a constant 3MB/s (~253GB/day).  I finally tracked this down to the Origin game client.  As soon as I signed out or shutdown the origin client, the excessive writes disappeared.

 

Lesson learned for point one - always read the tool tips as it says to set this to No for images.

 

I still think there is something weird going on, because the Windows VM was not registering 3MB/s of data being written by Origin - rather 0.1MB/s.  My suspicion is that some type of IO activity is being expanded dramatically at the filesystem layer for btrfs pools.

 

This is on the new build server.  Now I've got to get some outage on my older server to try and stop the crazy writing there and see if I can replicate the above success.

 

Edit: Also noticed the temp on my SSD pool has dropped by 2 degrees since making these changes.

[[6.8.3] docker image huge amount of unnecessary writes on cache]

This has been running since last night (12 hours roughly)

 

So 2 Windows 10 VMs responsible for 583G in 12 hours.

[[6.8.3] docker image huge amount of unnecessary writes on cache]

Brand new server (6.8.3) and suddenly noticed 111 million writes to my SSD pool (2 * 2TB WD Blue 3D NAND) after little more than a week.

 

Power on hours : 205 (8d, 13h)

233 NAND_GB_Written_TLC     -O--CK   100   100   ---    -    1393
234 NAND_GB_Written_SLC     -O--CK   100   100   ---    -    6789

 

Both drives are almost identical (as you'd expect).

 

I only have some Windows 10 VMs running, which I stopped and restarted.  It seemed to write a lot during windows boot up and then calm down.  However over time, the rate has increased again.

 

No dockers are running, only VMs.  I do not think this is exclusively a docker issue.

 

The loop2 process is nowhere to be seen on iotop, it's the VMs (which are idle most of the time).

 

This is a disaster in the making and will toast the drives long before the 5 year warranty expires.  Where is LT on this thread?  Is it worth a new one?  This is surely the single most important thing to look at right now, as customers out there with cache pools might be silently ruining their hardware.