Search for transcoding in the thread. Plex removed the cache location I think. Also, is your server up to the task? Might be hardware limitation.
I have set the cache to be on the cache drive on the unraid server so no problems with the space there.
The server is porwerfull enough for it, and it only happens then the cache folder gets close to 4GB, so I have not had this problem before since I mostly see series with episodes around 30min and the cache folder does not get over 3GB on episodes, only on long movies. (1.5hours+)