Data Contamination Avoidance Through Surface Level Problem Perturbation

via “data contamination avoidance through surface-level problem perturbation”

1,000 data science problems across 7 Python libraries.

Unique: Explicitly addresses data contamination risk through controlled perturbations rather than ignoring the problem or using completely synthetic data. Preserves authentic problem semantics and solution logic while modifying surface text, enabling safe evaluation of models trained on web-scale data.

vs others: More practical than synthetic benchmarks because it maintains real-world problem characteristics, while being more rigorous than unperturbed StackOverflow data because it mitigates contamination risks for models trained on web-scale corpora