VPN isn't a necessity but it's nice to have extra layer of security. The biggest thing would be to limit login attempts. Changing the port to a random one isn't on that list, but it makes a big difference with respect to random attacks. People are constantly scanning for common ports 21 (FTP), 22 (SFTP/SSH), 3389 (Remote Desktop), etc. Once they find one open, they just keep trying random username and password combinations.

James made a nice summary of the measures you should take:
Originally Posted by JamesNT
* Keep your system patched every month.
* Have a STRONG password for all user accounts allowed RDP access. And also have a lockout policy to lock the account after 5 or so bad attempts.
* Set the encryption level for RDP connections to HIGH.
* Optional: install an SSL cert to use for encryption from a certifying authority such as Godaddy.
* Do not allow any computer less than Windows 7 to connect. No Windows Vista or XP machines.
* Turn on Network Level Authentication.