I'm a white man who wants to have the racism talk/discussion. I'm willing to accept my own racism and be called out on it; and happy to engage with any Black person who will not "cancel" me for my views.
If I think about why I resist talking about racism it is for exactly the same reasons you feel happens in your own racism/sexism--that I am not heard.
I am happy to be criticized but not to be silenced; I am happy to listen more than I talk, but when I talk I would like to be acknowledged as a legitimate and whole human being--what this means, fundamentally, is that I am not just here to listen, but also sometimes to criticize the criticizers--to say, "Often I am wrong, but sometimes I am right; often you are right, but sometimes you are wrong."
Are there huge power imbalances between white men and black women? (yes, absolutely) and I have a lot to learn; at the same time as a white man, I am not responsible for EVERYTHING that happened to Black people.
There seems to be a very powerful contingent (see e.g. "White Fragility") that would suggest that as a white man my only job is to be instructed; that the answer to your voice being silenced is for mine to be silenced. That's when it becomes nearly impossible to keep listening.