Algorithmic Bias Monitoring

via “bias-resistant example curation through adversarial filtering”

44K pronoun resolution problems testing commonsense understanding.

Unique: Applies adversarial filtering specifically targeting statistical shortcuts (word frequency, syntactic position, gender stereotypes) through automated correlation analysis + human validation, rather than passive bias documentation; filtering is integrated into dataset construction rather than post-hoc

vs others: More proactive than datasets with bias documentation (e.g., BOLD) because biases are removed rather than flagged; more systematic than manual curation because automated detection identifies subtle correlations humans might miss