Auto Plugin With Device Selection And Load Balancing

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Unique: Implements heuristic-based device selection that considers model characteristics (size, operation types) and device capabilities (memory, compute power) to automatically choose the best device. The plugin can also distribute inference across multiple devices for load balancing, enabling transparent multi-device execution.

vs others: Provides more sophisticated device selection than ONNX Runtime's device selection (which is primarily manual) and supports load balancing across devices.