Instruction Response Pair Extraction And Formatting

via “instruction-response pair extraction for supervised fine-tuning”

161K human-written messages in 35 languages with quality ratings.

Unique: Preserves conversation tree structure while enabling flat pair extraction, allowing users to choose between SFT (flat pairs) and preference learning (branching) without data duplication.

vs others: More flexible than single-format datasets — supports both SFT and preference learning from the same source, vs datasets optimized for only one approach.

via “instruction-response pair extraction and formatting”

Multi-turn conversation dataset for steerable models.

Unique: Preserves reasoning chain annotations and multi-turn context during pair extraction, rather than flattening conversations into isolated Q&A pairs. Enables training on 'how to think' patterns, not just 'what to answer'.

vs others: More sophisticated than simple dialogue-to-pairs conversion (like basic CSV extraction) because it maintains semantic relationships between turns and explicitly encodes reasoning steps, producing higher-quality instruction-tuned models.

via “instruction-response-pair-generation-with-template-control”

300K instructions extracted directly from aligned LLM outputs.

Unique: Uses a pre-filled assistant template as a structural constraint during generation, allowing the model to generate diverse content within a controlled format. This balances the need for consistency with the flexibility of emergent generation.

vs others: More structured and reproducible than free-form generation while maintaining diversity better than fully rigid templates, because the model's learned distribution operates within the template constraints.

via “preference pair extraction for alignment training”

183K multi-turn preference comparisons for alignment.

Unique: Provides structured preference pairs derived from GPT-4 rankings of seven models, enabling direct use with modern preference optimization algorithms without additional annotation or pair construction logic.

vs others: More directly applicable to DPO/IPO training than raw rankings, and more flexible than fixed pair construction because researchers can implement custom pair extraction strategies on the underlying ranked data