The hyper focus on seemingly trivial things may not be so trivial. Many times it is a trivial thing that is sort of the straw that breaks the camels back with neurotypicals. So we remember small events that maybe we didn't quite understand or that may have been different when looked at in retrospect that seemed to cause someone to pull away or cause us to lose support of a group. We can see patterns, so often the thing that we focus on is the thing that happened before the pattern changed. We may not know why that item was setting people off when it was seemingly unimportant, often it's a case of something being said behind our backs or something that someone is questioning about us because our behavior is strange and then that small item seeming to prove that point, when really a larger event that happened much earlier is what created the question that was proven in people's minds.

Meanwhile we act differently from others so we have a completely different reason than what neurotypicals would have for our Behavior.
The double empathy situation is also the reason why we often have to deal with people or groups not accepting us and we may take a long time to get over that because neither they nor we were able to stop the problems from happening. It's the feeling of repeated failure in our relationships that makes each new failure however small and stupid to seem worse then it is.