Dual Metric Truthfulness And Informativeness Evaluation

via “dual-metric-truthfulness-and-informativeness-evaluation”

817 adversarial questions measuring model truthfulness vs misconceptions.

Unique: Decouples truthfulness from informativeness as independent evaluation dimensions rather than conflating them into single quality score; explicitly measures the dangerous failure mode of confident-sounding false answers (high informativeness, low truthfulness) which single-metric benchmarks miss

vs others: More nuanced than accuracy-only benchmarks (MMLU, TriviaQA) because it captures whether models generate plausible-sounding falsehoods or uninformative truths, addressing the safety-critical distinction between wrong answers and low-quality correct answers