Don't be embarrassed to meet such colleagues. Just refuse him directly. You tell him I don't want to bring you any more food. I think it is a burden for me to bring you food. Feel tired. You'd better buy your own food.

You really can't be too tactful to such an unconscious colleague, because he thinks it's natural for you to bring food. One cannot take everything for granted. Like you said, once in a while, it can last for a long time. You have no obligation.

So tell him how you feel. And he also made it clear that he would never bring him food again. Don't care about his feelings, because he never cares about your feelings.