this is not true if you have set shares to the 'Prefer' setting. That explicitly says that there are files on the data disk that you want to be moved to the cache disk whenever this becomes possible. That is the only setting that can cause files to be automatically be moved in that direction.
And there are other cache settings, like No, Yes, and Only. Each user share can have different cache settings, and if you don't set this for a particular user share the default is No, which means it won't use cache at all.
There is a Help button in the upper right of the webUI which will explain the current page to you. When setting up your user shares turn Help on. It it much more flexible (complex) than you thought.