I think this is a great question.
As I personally gradually move from the mindset of my family growing up (the adults believed all sorts of negative racist stereotypes) toward becoming anti-racist I find there lots of internal barriers. It is clearly a practice of rewiring and rethinking things that were woven into me as a child. There are biases and assumptions behind every internal nook and cranny. Some of it is "inherited" from my family, some of it comes from just living with privilege. It is hard work to uncover them and sometimes it is hard to be patient with myself and give myself the grace to move through the process instead of "being" there.
And then, despite my conscious intent and best efforts when I'm called out for not being "there" enough, or accused by someone who has trouble sorting out the role their own internal oppression in a conflict, it takes a lot to stay curious, learning and forgiving of myself and the other person. I remind myself regularly that this small work compared to the work of POCs dealing with microaggressions continually...and that it takes patience and practice. I think it would be easier to just retreat into privilege and blame the other person (and I'd be less aware of the pain mine and theirs....) and that is not the road I currently choose to travel. I do have compassion for those that do, and I think part of my work is inviting them to be braver and join for the good of all.