So it turns out I needed to re-install the client (in the meantime there was a newer version) and reboot my VM. Client now works without issue.
A lot of back and forth with PIA support, including using the OpenVPN terminal method and Firefox browser extension which both worked immediately.
Coincidentally the noVNC copy/paste clipboard now works, which didn't on my old VM.