I use a voicemail pager.
If you call the office the message says: "If you have an urgent problem, going on for less than 24 hours call our pager at 555-555-5555."
When they call the pager the message states: "You have reached the urgent pager. You can put in your number or a text message at the beep. We will not give refills on non urgent medicines, call the office."
The pager goes as a text message to both my and my NP's phone. On my iPhone all I have to do is press the highlighted number and they are called back* Frequently they leave a text message so I know the issue before they call.
I use DirectPage (directpage.com) Their listed cost is $143.00 per year but I usually have done it with discounts for under $100/yr.
*(I have caller ID blocked (Settings - Phone - Show My Caller ID - OFF)