Test Case Based Solution Filtering And Ranking

via “test-case-based solution filtering and ranking”

* ⭐ 02/2022: [Finetuned Language Models Are Zero-Shot Learners (FLAN)](https://arxiv.org/abs/2109.01652)

Unique: Uses empirical test execution as the primary ranking signal rather than model confidence scores, treating test pass rate as ground truth for solution quality; this is more reliable than likelihood-based ranking for algorithmic code where model confidence is poorly calibrated

vs others: More robust than confidence-based ranking because it grounds evaluation in actual execution results rather than model probabilities, but requires test case infrastructure that simpler code generation systems avoid