SafetyBench Eval

Evaluates LLM safety responses across seven orthogonal safety categories (offensiveness, unfairness, physical health, mental health, illegal activities, ethics, privacy) using 11,435 curated multiple-choice questions. Each question is tagged with its safety category, enabling granular analysis of model vulnerabilities across specific harm dimensions rather than aggregate safety scoring. The architecture supports both zero-shot and five-shot evaluation modes to measure both baseline safety and few-shot robustness.

Provides 11,435 safety questions in both English and Chinese with separate test sets (test_en.json, test_zh.json) and few-shot development sets (dev_en.json, dev_zh.json). The architecture includes a filtered Chinese subset (test_zh_subset.json with 300 questions per category) that removes sensitive keywords to enable evaluation in restricted deployment contexts. Questions are structurally identical across languages but culturally adapted to reflect region-specific safety concerns.

Implements two distinct evaluation protocols: zero-shot (questions presented directly without examples) and five-shot (five category-specific examples provided before test question). The architecture uses separate dev sets (dev_en.json, dev_zh.json) containing exactly 5 examples per safety category to construct few-shot prompts. The evaluation pipeline in evaluate_baichuan.py demonstrates prompt construction, model invocation, and answer extraction for both modes, enabling researchers to measure how few-shot examples affect safety performance.

Organizes 11,435 safety questions in a standardized JSON schema with fields: id (unique identifier), category (safety dimension), question (text), options (list of 1-4 choices), and answer (0-3 index for A-D). This schema enables programmatic question filtering, batch processing, and metric computation. The architecture supports both full datasets (test_en.json, test_zh.json with variable question counts per category) and filtered subsets (test_zh_subset.json with exactly 300 questions per category), allowing flexible dataset composition for different evaluation scenarios.

Provides a standardized submission format for evaluation results: a UTF-8 encoded JSON file mapping question IDs to predicted answers (0-3 for A-D). The leaderboard infrastructure aggregates submissions across models, computing per-category accuracy scores and overall safety metrics. The architecture enables comparison of model safety performance on identical question sets, with results published on llmbench.ai/safety. Submission format is language-agnostic, supporting any model that can generate multiple-choice predictions.

Provides carefully designed prompt templates for zero-shot and five-shot evaluation that can be adapted for specific model architectures. The evaluation code (evaluate_baichuan.py) demonstrates model-specific prompt construction, showing that some models require minor prompt modifications to enable accurate answer extraction. The architecture supports prompt templating with placeholders for questions, options, and few-shot examples, enabling systematic variation of prompt format while maintaining question content consistency.

Provides two download methods for SafetyBench datasets: shell script (download_data.sh) and Python script (download_data.py using Hugging Face datasets library). The architecture leverages Hugging Face Hub for dataset hosting and distribution, enabling one-command dataset acquisition with automatic decompression and directory structure creation. The Python method uses the datasets library for programmatic access, supporting integration into automated evaluation pipelines without manual file management.

Computes accuracy metrics stratified by safety category, enabling per-dimension performance analysis. The evaluation pipeline aggregates predictions across all questions in each category (offensiveness, unfairness, physical health, mental health, illegal activities, ethics, privacy) and computes category-specific accuracy scores. This architecture enables identification of category-specific vulnerabilities (e.g., a model may be robust on ethics but weak on physical health) without requiring separate evaluation runs.