Non Agression Principle is not pacifism, because it allows defence against violence.

Violence, whether physical or psychological, is wrong unless it is self-defence.

Unfortunately, we only respond to violence with violence again, preferably even more violence to discourage further attempts at violence against us.

Psychological violence is more complex than physical violence, so this does not apply in all cases, I will not respond to a person who shouts abuse at me, because he would win because he is provoking and wants to see my reaction.