Instruction Diversity Sampling And Stratification

via “instruction diversity sampling and deduplication”

Stanford's 52K GPT-3.5-generated instruction dataset that started it all.

Unique: Achieves diversity through implicit sampling during batch generation rather than explicit task categorization. Simplified pipeline removes classification/non-classification distinction, reducing pipeline complexity while maintaining empirical diversity through iterative sampling.

vs others: Simpler than original Self-Instruct's task-based categorization while achieving comparable diversity through batch decoding. More scalable than manual curation because diversity emerges from the generation process rather than requiring post-hoc filtering.