I've finally got it working 100% now.
To fix the problem with legacy boot mode I had to re-enable the embedded graphics on the motherboard in the bios, then that allowed me to boot into unraid. However, I still had the same issues with the VM. I tried a bunch of different things including enabling Message Signaled-Based Interrupts (MSI) but nothing worked.
I eventually found a solution from someone with a similar Gen9 HP server, except they're using Proxmox: https://forum.proxmox.com/threads/gpu-passthrough-issue.109074/post-469825
They suggest putting this in syslinux config:
video=simplefb:off
I put that in along with intel_iommu=relax_rmrr video=efifb:off video=vesafb:off. I then installed the latest Nvidia drivers whilst in safe mode and they seemed to install fine. For the first time the Nvidia audio drivers had installed. When I rebooted I encountered the same problem I had before, just an endless bootloop.
I'd realised I forgot to re-enable MSI for the GPU, so I did that in safe mode, rebooted and it's all done! I'm going to see if I can revert back to using UEFI however, and I'll post back here if it works or not, just in case there's anyone else having the same issue.
Thank you @ghost82 for helping me out and pointing me in the right direction!
Update: I switched back to UEFI in BIOS and in Unraid, and it's all still working fine.
MustardTiger's post in Windows 10 VM crashes/breaks during Nvidia driver install for GPU passthrough. was marked as the answer