I'm having a similar issue.
I have 16GB of RAM and after about a day after a reboot it has allocated over 12GB for cache. I can watch it go up if say I have something sitting on my cache drive and press the mover button, it takes more cache RAM but then never lets it go.
Storage wise I have five 4TB drives in the array (one is parity) and one 500gb ssd as a cache drive.
I am not running any VM's. I only have 4 dockers running. (couch potato, sonarr, sabnzb, plex)
This just started recently too. Before it would only use about 1gb of RAM...so now the problem is if Sabnzb is processing a file I run out of RAM and it slows down. Plus I'm seeing a stuttering from plex during playback that didn't used to exist.