After thinking really hard with that half COVID Information Technology degree I have, I think I know the issue. Like 90% sure. My server is on a server VLAN. The switch I use only actually has a 1Gb uplink to my router. Ubiquiti's Flex XG I use for my 10Gb switch does NOT support layer 3 so that data has to go back through the router to get to the correct VLAN leaving the uplink to the router the bottleneck... Curently using the second 10Gb port on that card to do to the same LAN as my PC and run the test again.