Recipe Based Reproducible Experiments With Configuration Management

via “evaluation reproducibility through configuration versioning”

Automatic LLM evaluation — instruction-following, LLM-as-judge, length-controlled, cost-effective.

Unique: Captures all evaluation parameters in version-controlled YAML configurations with metadata tracking, enabling reproducible evaluations and transparent methodology auditing. Configuration-based approach allows sharing evaluation setup without code, improving accessibility for non-engineers.

vs others: More reproducible than ad-hoc evaluation scripts; more transparent than implicit parameter defaults

via “configuration-driven training experiment management”

Fully open bilingual model with transparent training.

Unique: Provides open-source configuration-driven experiment management integrated directly into training pipeline — most research code uses ad-hoc scripts or external tools (Weights & Biases, MLflow), and few models publish complete configuration files for reproduction

vs others: Enables perfect reproducibility through configuration versioning and automatic logging, though requires more upfront design than ad-hoc scripting and may be less flexible for highly customized experiments

via “git-integrated experiment branching and reproducibility”

Git for data and ML — version large files, experiment tracking, pipeline DAGs, remote storage.

Unique: Stores experiments as Git commits with full code and parameter snapshots, enabling perfect reproducibility without external databases. The experiment registry maps Git commits to experiment metadata, making experiments shareable and auditable via Git history.

vs others: More reproducible than MLflow because all inputs are captured in Git, but less convenient than cloud-based platforms because experiments are stored locally and require Git operations.

via “experiment configuration and yaml-based declarative training specification”

Deep learning training platform — distributed training, hyperparameter search, GPU scheduling.

Unique: Uses a declarative YAML schema that captures the full experiment specification (model, hyperparameters, distributed settings, resource requirements) in a single file, enabling version control and reproducibility. The master service parses the configuration and uses it to instantiate trials without requiring users to write boilerplate code.

vs others: More declarative than programmatic configuration APIs because it separates experiment definition from code; more flexible than cloud provider templates because it supports arbitrary hyperparameter spaces and search algorithms.

via “recipe-based reproducible experiments with configuration management”

All-in-one speech toolkit in pure Python and Pytorch

Unique: Implements recipe-based experiment templates with YAML configuration that bundles model, training, and evaluation in a single file, enabling one-command reproducible experiments. Supports recipe inheritance and composition for systematic ablation studies without code duplication.

vs others: More structured than raw PyTorch scripts for reproducibility; simpler than Hydra-based configuration for speech-specific workflows; enables easy experiment sharing and version control compared to notebook-based experiments

via “configuration management with hierarchical settings”

Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Unique: Implements hierarchical configuration with clear precedence (code defaults < config files < command-line overrides) and automatic validation, enabling reproducible experiments and easy configuration sharing across teams

vs others: More structured than ad-hoc hyperparameter management while simpler than full experiment tracking systems like Weights & Biases, providing a good balance for research and production use

via “training-experiment-management”