Ok more tests: since both have the same motheboard (Asus C60M1-I, love those paired with Node cases ) I swapped the mobo just in case the ethernet port on the board was bad (or wrong settings). Nope, same issue.
Putting back the other mobo and looking at the dashboard it's detected as gigabit so I think that leaves out the switch then?? The weird thing is that the other computers connected to the same switch run ok...
BTW, I'm using big files for testing, so no random seeks.