I think you are right here. By cherry-picking gender minorities we sow a dissent and we underline their "otherness" from everyone else.
I am from an ethnic minority myself, and somehow people perceive us as maniacs and killers, try to burn our houses and fire us from work. Often it's the same people who preach equality and diversity. Somehow in my case they don't care about "empowerment" or "representation" at all. And somehow, the only place I feel comfortable is 4chan, where no one gives a damn who you are, and everyone's racist and sexist. To everyone else. Equally.