Interactive Hypothesis Testing

via “interactive model debugging with hypothesis testing”

Open-source tool for ML observability that runs in your notebook environment, by Arize. Monitor and fine tune LLM, CV and tabular models.

Unique: Integrates hypothesis formulation with trace filtering and metric computation, enabling iterative refinement of debugging hypotheses within notebooks. Supports both declarative filtering (e.g., 'where confidence < 0.5') and custom Python functions for flexible hypothesis specification.

vs others: More interactive and exploratory than batch-based debugging tools (MLflow, Weights & Biases) because it enables real-time hypothesis refinement in notebooks; more accessible than statistical testing frameworks (scipy, statsmodels) because it abstracts away statistical complexity.

via “interactive-hypothesis-testing”

via “interactive hypothesis testing and iterative design”