I found out what is causing this, its something to do with the nvme drive itself. I was trying to pass-though a Mushkin Pilot (MKNSSDPL250GB-D8), uses a SMI SM2262 for its controller. However I just tried a samsung pm961 and it worked just fine. Very odd, this must be an issue with kvm or maybe the nvme is not made to spec.
Edit: Ill also add that i had 2 of the pilots in my system, not sure if that would some how make a difference but thought i would add it. I would like to test it again with only one but I have no way to.