Great points.
HIPAA does not make any specific technical requirements for security. So it is not absolutely required. One of the benefits of running an EMR that is non-cloud is that the majority of your traffic is offline.
The biggest difference between a router firewall and a hardware firewall is that they operate on a different core assumption. Routers firewalls/cheap firewalls 'allow' by default and firewalls 'deny' by default.
Most firewalls do not work out of the box for that reason. They need to be configured to allow access. However, knowing exactly what is needed to go in and out can be confusing for even IT professionals. I've been to several offices with SonicWalls only to see that they are configured exactly the same as a consumer router which defeats the point of buying a 1000 dollar router/firewall.