Just bumped into this.
It has been a while since i played around with this, but the problem of your keyboard and mouse seemed something i had encountered before on a project i was working on.
Don't know if this is still the case for ubuntu 20.04, but in Ubuntu 16.04 i had the same issue of the keyboard and mouse not working when working with Nvidia grid licenses.
Was able to solve this by :
- Updating the VM , but NOT restarting it yet.
- Before restart run: apt-get install xserver-xorg-input-all
- Reboot
After doing this i was still able to use keyboard and mouse in the ubuntu vm.
But still , seeing as this was on 16.04 , not sure if this is still applicable.