binhex, I followed the FAQ again but still had no luck. I decided to try delugevpn instead and it's working great. I noticed STRICT_PORT_FORWARD key was there by default on delugevpn, which I think is where the source of my problem was.
Looking at my original rtorrentvpn docker, for some reason the STRICT_PORT_FORWARD key was not present at all by default. I went and looked at re-installing rtorrentvpn and the STRICT_PORT_FORWARD key is now present by default. I don't know what happened between when I originally installed it a year ago and now to cause that key to not be present, since I generally keep my dockers up to date by checking once a week.
Anyways, working now, so I'm happy!