I don't think you read it. He is suggesting that women reacting to actual real danger is damaging to him. If he understood then it wouldn't be hurtful.
Do you get upset when a strange child is scared of you? Of course not, because you understand that new people are scary to children.