Just in case anybody has had the same experience as me; none of the posted suggestions or steps helped me while attempting to fix this GPU passthrough problem (on both Mac and W10 VMs). After countless hours of troubleshooting, my solution (frustratingly simple) was booting Unraid into legacy mode instead of UEFI/efi mode. The moment I decided to boot into legacy and then fire up the vm's, I had no issues with installing the appropriate Nvidia drivers on either operating system's vm.