OK - so something related to Wireguard where even when I was on the terminal, it would try and send pings from its 10.253.x.x address. Which is kind of crazy. Not sure how to file a bug report and what was going on.