Nuanced Preference Filtering

via “dimension-specific preference filtering and stratification”

64K preference dataset for RLHF training.

Unique: Provides explicit dimension labels on preference judgments, enabling dataset consumers to filter and stratify by specific behavioral objectives rather than treating all preferences as equivalent. This allows training models optimized for particular use cases without confounding signals from unrelated dimensions.

vs others: More flexible than monolithic preference datasets because it enables task-specific subset creation and objective-aligned training, whereas generic preference datasets force you to train on all dimensions simultaneously or manually re-annotate data.