Span Based Answer Annotation With Character Level Indexing

via “span-based answer annotation with character-level indexing”

150K reading comprehension questions including unanswerable ones.

Unique: Uses character-level span indexing rather than token-level, making answers independent of tokenization choices. This enables fair comparison across models with different tokenizers and avoids off-by-one errors from token boundaries.

vs others: More precise than free-form answer generation (which requires BLEU/ROUGE metrics) and more tokenizer-agnostic than token-level span prediction, enabling reproducible evaluation across different model architectures.

via “passage-level answer span extraction with position tracking”

question-answering model by undefined. 48,782 downloads.

Unique: Predicts token-level start/end positions which are converted to character offsets via the tokenizer's offset_mapping, enabling precise answer localization without post-hoc string matching; supports both token and character-level indexing for flexibility

vs others: More precise than regex-based answer extraction (handles tokenization edge cases); token-level prediction is more efficient than character-level models; offset tracking enables direct document highlighting without string search