As Christopher suggests, using a Terminal Server is the way to go. You can configure Remote Access to just use port 443. Ports 80 and 3389 are not necessary to open. Not opening them reduces the amount of attack vectors into your server. If you add the Remote Desktop Gateway sub role, then it'll be authenticating against the gateway and you can access all computers on your network including the server without having to open port 3389.
Agree with reducing the attack surface. Go one better and configure a random high port and port forward that internally to 3389. You have then greatly reduced the brute force attacks that machine will see.
