You can totally ignore his wife's remarks. If you help and you don't do well in some places, maybe his wife will dislike it again. After all, her husband is your friend. You don't have to worry about what he said. If he says that about you next time, say you can't do it or can't do it well. I think the most important thing in this matter is your friend's attitude. If your friends also think you should help, you can choose to visit less in the future. After all, whether you help or not may attract their gossip.