Offset Tracking And Character To Token Mapping For Span Extraction

via “entity span reconstruction from subword tokens”

token-classification model by undefined. 18,78,235 downloads.

Unique: Requires custom post-processing logic to map BERT's subword token predictions back to character-level spans, as the model natively outputs per-token classifications without span boundaries. This is not built into the model itself — users must implement or use a library like seqeval or transformers.pipelines.TokenClassificationPipeline.

vs others: More accurate than regex-based entity extraction because it preserves model confidence and handles complex token boundaries, but requires more engineering than end-to-end span prediction models (which directly output spans without subword merging).