Device Specific Model Optimization With Npu Kernel Selection And Memory Layout Tuning

via “intel npu plugin with model partitioning and fallback execution”

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Unique: Implements automatic model partitioning into NPU-compatible and CPU-fallback subgraphs, with unified KV-cache management across both execution paths. The NPUW layer abstracts NPU-specific compilation details and handles dynamic shape inference, enabling seamless hybrid execution without explicit partitioning by users.

vs others: Provides the only production-ready NPU inference solution for Intel processors and supports more diverse model architectures than NPU-specific frameworks through CPU fallback.