stanza

Splits raw text into sentences and tokens using language-specific neural models and rule-based segmentation. The tokenizer handles multi-word tokens (MWT) common in languages like Arabic and Czech, expanding them into individual words. It uses a two-stage approach: first identifying sentence boundaries, then tokenizing within sentences using pre-trained neural models that understand language-specific morphology and punctuation conventions.

Assigns part-of-speech tags and morphological features (case, gender, number, tense, mood, etc.) to tokens using neural sequence models, then constructs syntactic dependency trees showing grammatical relationships between words. The architecture uses a BiLSTM-based tagger followed by a transition-based or graph-based dependency parser that learns to predict head-dependent relationships. Both components are trained jointly on Universal Dependencies treebanks, enabling cross-lingual transfer and consistent annotation schemes.

Provides Python bindings to the Java Stanford CoreNLP library, enabling access to CoreNLP's advanced features (Semgrex pattern matching, Ssurgeon tree surgery, enhanced dependencies) while maintaining Stanza's Python API. The integration layer converts between Stanza's Python document model and CoreNLP's Java representations, allowing seamless use of CoreNLP processors alongside native Stanza processors. This enables leveraging CoreNLP's mature implementations of complex linguistic tasks while staying in Python.

Supports training custom NLP models on user-provided datasets using PyTorch, with utilities for dataset preparation, model configuration, and evaluation. The training framework includes dynamic oracles for transition-based parsers, which correct parser errors during training to improve robustness. Training pipelines handle data loading, batching, optimization, and evaluation metrics. Users can fine-tune pre-trained models on domain-specific data or train models from scratch for new languages or tasks.

Provides specialized pre-trained models for biomedical and clinical NLP tasks, trained on medical corpora and annotated with medical entity types and clinical terminology. These models include biomedical NER recognizing medical entities (drugs, diseases, procedures), POS tagging adapted for medical text, and dependency parsing trained on clinical notes. Models are available for English and trained on diverse medical sources (PubMed abstracts, clinical notes, biomedical literature).

Identifies and classifies named entities (persons, organizations, locations, etc.) in text using neural sequence labeling models trained on language-specific corpora. The NER processor operates on tokenized input and produces entity spans that may cover multiple tokens, with each entity assigned a type label. Models are trained using BiLSTM-CRF or transformer-based architectures on diverse treebanks, with specialized biomedical/clinical models available for English medical text.

Constructs constituency parse trees that represent the hierarchical phrase structure of sentences, showing how words group into noun phrases, verb phrases, and other constituents. The parser uses a neural chart-based or transition-based approach to build trees bottom-up from tokens, trained on treebanks with constituency annotations. Output is a tree structure where each node represents a phrase with a syntactic label (NP, VP, PP, etc.) and children are sub-constituents or words.

Determines the base/dictionary form (lemma) of each word using a combination of neural models and morphological rules. The lemmatizer takes POS tags and morphological features as input to guide lemmatization, handling irregular forms and language-specific morphology. For some languages, it uses rule-based approaches; for others, neural sequence-to-sequence models trained on morphological analyzers. Output is a lemma attribute on each word, enabling downstream tasks to work with canonical word forms.

Identifies mentions of the same entity across a document and groups them into coreference chains, enabling tracking of who/what is being discussed. The resolver uses a neural mention-ranking model that scores pairs of mentions for coreference likelihood, building chains by linking mentions to their antecedents. It operates on the full document context, using word embeddings, syntactic features, and semantic similarity to determine if mentions refer to the same entity. Output is a mapping of mention spans to coreference cluster IDs.

Classifies the sentiment polarity (positive, negative, neutral) of sentences using neural classification models trained on sentiment-annotated corpora. The sentiment analyzer takes tokenized sentences as input and outputs a sentiment label and confidence score for each sentence. Models are typically fine-tuned LSTM or transformer-based classifiers trained on domain-specific data (e.g., movie reviews, product reviews, social media).

Provides a unified data structure (Document → Sentence → Token/Word → Entity) that stores all linguistic annotations produced by pipeline processors. The model is hierarchical, with each level containing relevant metadata: Documents contain sentences, sentences contain tokens and words (tokens may expand to multiple words for MWT), and entities are associated with sentence spans. All annotations (POS tags, lemmas, dependencies, NER, sentiment, etc.) are stored as attributes on the appropriate level, enabling easy access and traversal of linguistic information.

Manages the initialization, configuration, and execution of NLP processors in correct dependency order, with automatic model downloading and caching. The Pipeline class coordinates processor dependencies (e.g., POS tagging must run before lemmatization), handles processor configuration via kwargs, and supports lazy loading where processors are only initialized when needed. The resource management system automatically downloads missing models from Stanford's servers on first use, caching them locally to avoid repeated downloads.

Provides pre-trained models for 60+ languages using Universal Dependencies (UD) treebanks as the standard annotation scheme, enabling consistent linguistic representations across languages. Models are trained on UD treebanks for each language, ensuring that POS tags, dependency relations, and morphological features follow the same standards. The unified API allows switching between languages by changing a single parameter, with all downstream code working identically regardless of language.