ShieldGemma

Classifies input and output text for sexually explicit content using a fine-tuned Gemma language model trained on safety datasets. The model processes natural language through transformer attention mechanisms to detect explicit sexual references, imagery descriptions, and adult content across multiple languages and contexts. Returns confidence scores and categorical severity levels (e.g., safe/unsafe) that can be thresholded for different deployment scenarios.

Identifies and classifies text containing instructions for violence, self-harm, illegal activities, or other dangerous behaviors using semantic understanding of intent and context. The model distinguishes between educational/informational content and actionable dangerous instructions through fine-tuned pattern recognition on safety-labeled datasets. Outputs severity scores and content category tags enabling graduated response policies (e.g., warning vs. blocking).

Detects targeted harassment, bullying, and abusive language directed at individuals or groups using contextual language understanding. The model identifies patterns of repeated negative targeting, personal attacks, and coordinated abuse through transformer-based semantic analysis of conversation context and user interaction history. Outputs harassment severity scores and target identification enabling context-aware moderation policies.

Classifies text containing hate speech, discriminatory language, and slurs targeting protected characteristics (race, ethnicity, religion, gender, sexual orientation, disability, etc.) using fine-tuned semantic understanding. The model recognizes both explicit slurs and coded language/dog whistles through pattern matching on safety-labeled datasets. Outputs hate speech severity, target group identification, and language category enabling nuanced moderation policies.

Enables fine-grained control over safety classification thresholds and policies through configuration parameters applied at inference time. Allows operators to adjust confidence score cutoffs per safety category (e.g., strict filtering for explicit content, lenient for dangerous content), define custom response policies (block/warn/log), and apply different thresholds to different user segments or content types. Implemented through post-processing of model confidence scores against configurable policy rules.

Applies safety classification across multiple languages using Gemma's multilingual capabilities, enabling consistent content moderation policies across global platforms. The model processes text in 40+ languages through shared transformer embeddings trained on multilingual safety datasets. Outputs language-agnostic safety classifications with per-language confidence adjustments reflecting training data coverage.

Processes multiple text inputs (messages, comments, completions) in batch mode with vectorized inference, returning safety scores and classifications for all inputs simultaneously. Implemented through batching at the inference layer to maximize GPU utilization and throughput. Outputs structured results with per-input classifications, confidence scores, and category breakdowns enabling efficient content moderation pipelines.

Integrates safety classification into LLM application workflows by filtering both user inputs (before reaching the model) and model outputs (before returning to user). Implemented as middleware in the inference pipeline that applies safety classifiers sequentially or in parallel, with configurable blocking/warning policies. Enables end-to-end safety without modifying the base LLM.

Generates quantitative safety metrics and reports from classification results, enabling monitoring of content safety trends and policy effectiveness. Computes aggregate statistics (% unsafe content by category, false positive rates, policy violation trends) and generates visualizations/dashboards. Implemented through post-processing of classification results with aggregation and statistical analysis.