Perf Analyzer For Load Testing And Latency Measurement

NVIDIA inference server — multi-framework, dynamic batching, model ensembles, GPU-optimized.

Unique: Generates synthetic load against running inference servers with configurable concurrency patterns, measuring end-to-end latency including network overhead. Produces detailed latency distributions and performance curves.

vs others: Integrated load testing tool differs from generic load generators, with inference-specific metrics (batch sizes, model-aware requests) and latency measurement.