There are two things to consider when passing through a GPU to a Linux VM.
1. The GPU needs to be capable of being passed through to a VM
2. The GPU needs to be supported under Linux
To find an answer to (1), these forums are a good place to start. In my experience, NVIDIA GPUs have fewer issues with GPU pass-through. I use an NVIDIA Quadro K2200 and a GTX 1060 without issue - they are very stable. I haven't tried recent AMD GPUs though.
As for (2), phoronix.com has lots of information on GPU support under Linux. Typically AMD GPUs have much better open-source Linux driver support. This means that the GPU will be detected correctly and just work when installing Linux. If you choose NVIDIA, then you *will* want to install the nVidia binary drivers. This is quite easy on well-known modern Linux distros (e.g. Ubuntu, Mint, openSUSE). The problem you are most likely to have is a 'black screen' on boot (look up 'nomodeset' if this affects you), which makes installing anything difficult!
So the reason this might be a tricky decision is because the answer to (1) is NVIDIA, and to (2) is AMD!