You are right, it is indeed a double-edged sword.
Its nice of you to meet with them. I am sure it encourages them to take the extra step, and like you said, breaking bread with someone will make for a much more positive relationship for both parties.