People with abilities do tend to gravitate towards each other.

[Or get inexorably linked through the mysterious energy that grants them powers. You know. Either or.]

I'd be more surprised if you were treated equally by those who didn't have them.