If an experience leaves a bad taste in your mouth, you have an unpleasant memory of it.

I think we all felt he'd been treated very unfairly and it left a bad taste in our mouths. 我想我们都觉得他受到了极不公平的待遇，这件事给我们留下了很不愉快的记忆。