Here's what I did:
Added a secondary NIC
Started the server
Gave the new NIC an IP address I can access
Went to that IP-address
Switched the two IP addresses around so that the new NIC gets the IP address I want to access the server by.
Reboot, just in case, and disconnected the old NIC
The result:
Absolutely no network access. The startup screen at the local console tells me the IP address the server listens on, which doesn't work. I