This can be really frustrating because, in the heat of the moment, you can't think clearly. Something as simple and basic as getting to your CA site. You can see the SharePoint site you created, but you can't see CA, so you can't even get started. You're stuck staring at the Internet Explorer connection problem page du jour.
So, in desperation you log in to the machine on your farm that is actually hosting Central Administration and lo-and-behold it is working.
Yes, I'm afraid it's those pesky firewall rules again. You need to open up the appropriate port.