When you enter a restaurant and find it is too expensive, what reason can you find to leave without hurting your self-esteem?
If it were me, I would find a reason to leave because of things at home. For example, the waiter was standing next to you to order. You can suddenly tell your friend that I suddenly have some things to deal with at home, then invite my friend to help, and then you two can go. I think the waiter has actually encountered a similar situation and should be able to understand it.