If you can set up that 5 port switch upstairs, then plug your "access point" and that PC into that switch, I think that'll address the immediate need. I think you might be putting clients downstream from the "access point" into a double NAT situation (and maybe that has something to do with your current issue?) I'm not enough of a networking guy to explain why double NAT is a problem, I just know enough to know if you put yourself in that situation unintentionally with consumer gear, problems can arise