First of all you confuse empathy and sympathy. YOU can't feel empathy for a black guy being called a nigger.
You keep saying they will be affected but you can't tell me how they will be affected in any material way.
Well, that really depends on whether I am white, doesn't it?
First it doesn't take exactly the same experience to empathize, you only need a similar experience. Secondly, sympathy is when you try to comfort somebody, empathy is when you extrapolate emotion from a similar experience. Every human has had a time in their life where they were treated less than human by somebody around them, it isn't difficult to extrapolate meaningful empathy.
And "material" is your own straw man. That they must be "materially" affected, shows you lack empathy, nor do you even attempt to sympathize. In my world and religion, you are lacking humanity.