koelectra-small-v2-distilled-korquad-384

Performs span-based extractive QA on Korean language documents using a distilled ELECTRA transformer architecture fine-tuned on KorQuAD dataset. The model identifies and extracts the most probable answer span (start and end token positions) from a given passage that answers a natural language question, outputting confidence scores for both span boundaries. Uses token-level classification with softmax scoring over sequence length to pinpoint exact answer locations within context.

Executes forward passes using a knowledge-distilled ELECTRA model with 40% parameter reduction compared to base ELECTRA, enabling deployment on resource-constrained devices. The distillation process transferred learned representations from a larger teacher model into this smaller student architecture, maintaining semantic understanding while reducing embedding dimensions and layer counts. Supports multiple inference backends (PyTorch, TensorFlow, TFLite) for flexible deployment across cloud, edge, and mobile environments.

Applies Korean-optimized WordPiece tokenization that preserves morphological structure and handles Korean-specific Unicode ranges (Hangul syllables U+AC00-U+D7A3). The tokenizer uses a Korean-specific vocabulary learned during ELECTRA pre-training, enabling accurate segmentation of Korean compound words, particles, and verb conjugations that would be fragmented by generic multilingual tokenizers. Handles both modern Hangul and legacy Korean text encoding.

Provides model weights in multiple serialization formats (PyTorch safetensors, TensorFlow SavedModel, TFLite) enabling deployment across heterogeneous infrastructure without conversion overhead. The safetensors format enables secure, fast weight loading with built-in integrity checking; TensorFlow format supports graph optimization and quantization; TFLite enables mobile/edge deployment. A single model checkpoint can be loaded into any supported framework via the transformers library's unified interface.

Predicts answer spans by computing logit scores for each token position as a potential answer start and end, then selects the span with highest combined probability. The model outputs two logit vectors (start_logits, end_logits) of length sequence_length; inference applies softmax to convert logits to probabilities and selects argmax for start/end positions. Confidence is computed as the product of start and end token probabilities, enabling ranking of multiple candidate answers or filtering low-confidence predictions.