Bert,
A few thoughts on the matter:
If it is a rare event, you have a couple of options. You could blow it off and not take it personally. Afterall, if they are not happy with you, the relationship is no longer a good relationship and may open you up to liabilty (am I rationalizing now!?). Another option is to call the patient either personally(especially if you thought you had a close relationship) or have a staff person call (perhaps less intimidating). The problem here is that if they haven't given you constructive criticism so far, it likely still won't happen. A third option is to drown your sorrows with your favorite drug of choice (not sure if posting on the board counts Bert!), but I don't think that would be very constructive.
If this is a frequent problem for you, then you have major problems! (I doubt this applies to you Bert. The few docs I've seen who had patients leaving in droves didn't seem to be in the least bit bothered by it or blamed others....which explains their problems!)
In the end, I think it is a normal and healthy thing to feel something when a patient leaves, whether it be sadness or joy. I fondly remember the day one patient told me he was going to see another doctor after I wouldn't refill his narcotics! Our office happily had his chart printed and ready to go.
Don't know if this helped, but to quote the former president "I feel your pain"....or was he the one that said "where's my intern!"