I think the issue here is reconciliation rather than forgiveness. You can forgive from the heart and then, with your anger diminished, ask for fairness from the other person. As you stand firm in the request for justice, you are giving the kind of message that I think is your intention, that the behavior is inappropriate. So, consider forgiving as soon as you sense anger arising in you from the injustice. Then have reconciliation in mind by pointing out the behavior that you would like to see him change so that you can again come together in mutual trust.