Multi Phase Ranking Execution

via “multi-phase ranking with onnx model integration”

AI + Data, online. https://vespa.ai

Unique: Executes ONNX models natively on content nodes during query processing without external model serving infrastructure, with ranking expressions compiled to optimized C++ code. This eliminates network latency of calling external ML services and enables batched inference across candidate results.

vs others: Faster than calling external model serving APIs (Triton, KServe) because ONNX inference happens in-process on content nodes, eliminating network round-trips and enabling batched inference across top-K candidates in a single pass.

via “multi-phase-ranking-execution”