You can deactivate, but to my knowledge you cannot remove. One of the reasons for this is that records are tied to that person, so to remove them completely would cause breaks in the database.
They are deactivated similar to making a patient inactive.
Even more frustrating is that their templates are not removed and can glom up a lot of space and time. You can spend a long time just removing them.