Open Internet by MindsNet

Silent NPU Fallback Detection in ML on Snapdragon

The ONNX Runtime's QNN execution provider silently routes unsupported ops to the CPU, causing production latency to mysteriously increase. This issue is hard to detect with standard median latency gates, leading to a bimodal distribution of latency. The problem affects teams shipping ML to Snapdragon.

Computing & Technology, Computer Science, Machine Learning