Dear Lord, what a pain in the ass. I'm not sure what I did but after deleting and re-adding the config in VNC Viewer and turning off the setting Connect via Proxy, which was on by default, everything seems to be working. I still can't connect to the VM using through the Unraid VNC Remote, but that's fine. I'll play more with VNC Viewer. I'm sure it will work fine.
Thanks for all the help @Andrew Clarke