ALRIGHT, I just so happened to keep messing around with this config. I set up a VM with the exact specifications listed above (only with the audio passed through as well), installed the ARC drivers from a VNC terminal. The card showed as good to go, but after a reboot the card showed as code 43. I then went into the VM settings, turned off Hyper-V, and booted it up. It works!
I've now rebooted the machine 3+ times and the card is still functioning. Even rebooted unraid and everything works. Hope this works out for others...