I know it's an older thread but essentially, changing the port number of your SSH isn't doing much in terms of security. It just makes it a tiny bit harder to determine the right port. You do discourage very simple attacks that scan very broadly and don't bother with alternate ports but changing the default port does nothing for targeted attacks and also not much for more sophisticated, automated attacks, once an attacker figures out your true ssh port, you will get brute-forced just like normal.
If you are still looking for a very simple and elegant solution for remote ssh access, give Tailscale a try. It's FOSS, based on WireGuard, doesn't require port forwarding and has clients for basically all platforms under the sun.