ToxiGen

Generates adversarial hate speech examples using the ALICE (Adversarial Language-model Interaction for Classifier Evasion) framework, which implements a beam search algorithm that combines GPT-3 language model probabilities with toxicity classifier confidence scores to produce text that is both fluent and designed to evade existing hate speech detection systems. The framework iteratively refines candidate generations by weighting language model likelihood against classifier adversarial objectives, enabling discovery of subtle, implicit toxic content without explicit slurs.

Converts human-created text demonstrations into structured prompts that guide GPT-3 to generate similar toxic content across 13 predefined minority groups. The system reads demonstrations from a directory structure organized by target group, applies configurable few-shot prompting with a specified number of examples per prompt, and produces prompt files ready for text generation. This approach leverages in-context learning to transfer toxic patterns from seed examples to new variations targeting specific demographic groups.

Integrates pre-trained toxicity classifiers (HateBERT, RoBERTa) into the text generation pipeline to provide real-time confidence scores that guide adversarial example generation. The system interfaces with classifier models to extract confidence outputs during beam search, enabling the ALICE framework to weight generations based on how likely they are to fool the classifier. This integration allows the generation process to actively optimize for adversarial properties by treating classifier confidence as a scoring signal.

Generates and distributes a large-scale dataset of toxic and benign statements across 13 minority groups using the combined demonstration-based and ALICE-framework approaches. The system produces structured datasets with annotations, metadata, and versioning, and distributes them through HuggingFace Datasets for reproducible research. The pipeline orchestrates human demonstrations, prompt generation, text generation, and dataset packaging into a cohesive workflow that produces research-ready adversarial datasets.

Generates benign (non-toxic) text statements about minority groups to create balanced datasets with both positive and negative examples. The system uses similar prompting and generation techniques as the toxic generation pipeline but with different seed demonstrations and objectives, producing grammatically sound, contextually appropriate non-toxic content. This capability ensures datasets contain both toxic and benign examples, enabling classifiers to learn discrimination between harmful and harmless content.

Provides utilities to load the generated ToxiGen dataset from HuggingFace or local files, apply preprocessing transformations (tokenization, normalization), and prepare data for training toxicity classifiers. The system handles dataset format conversion, train/validation/test splitting, and batch creation for PyTorch or TensorFlow training loops. This capability abstracts away dataset format complexity and enables researchers to quickly integrate ToxiGen data into their classifier training pipelines.

Provides infrastructure for human annotators to review and label generated toxic and benign examples with toxicity severity, implicit/explicit classification, and group-specific annotations. The system tracks annotation agreement, flags low-confidence examples, and produces quality metrics that enable filtering of low-quality generated content. This capability ensures dataset quality through human validation while maintaining reproducibility through structured annotation workflows.

Classifies generated toxic examples as either implicit (subtle, indirect, without slurs) or explicit (containing profanity, slurs, or direct attacks) to enable fine-grained analysis of toxicity types. The system applies rule-based heuristics and optional classifier-based detection to distinguish between these categories, enabling researchers to study how well classifiers perform on implicit versus explicit toxicity. This capability supports the core research goal of improving detection of subtle, implicit hate speech.

Analyzes dataset coverage across the 13 minority groups, generates statistics on example distribution, and produces reports on group-specific toxicity patterns and classifier performance. The system computes metrics like examples per group, toxicity prevalence by group, and group-specific classifier accuracy, enabling researchers to identify coverage gaps and group-specific biases. This capability supports systematic evaluation of whether classifiers perform equally well across all demographic groups.