I would offer a little bit of a nuance. Of course, I can't control people who throw around slurs willy nilly, I can only judge them
But the thing that differentiates a slur from a regular disparaging remark is historical and institutional baggage. That's where the nuance and asymmetry come from
Which is why a white person calling a black person the n word is different from black people calling each other the n word. One's echoing centuries of oppression and dehumanization, while the other's a reclamation of a slur into a term of endearment. You see this as well in the queer community adopting the f slur, though not universally.
Also it's not the history that's offensive, it's the context. If a black guy referred to another black guy in a derogatory manner for being black that is as offensive as a white or asian guy doing it.