Edge AI accelerator comparison: 21 NPUs, GPUs and TPUs
The 21 edge accelerators ModelVisio scores models against, with the numbers that decide how fast a model runs: dense INT8 TOPS, memory bandwidth, typical power and the toolchain you compile with. Each figure carries its source.
Score your model on all 21 Drop a model file in the browser. It is never uploaded.
Specifications
Sorted by dense INT8 TOPS. Ridge point = TOPS ÷ bandwidth, in operations per byte. "est." marks an estimated bandwidth.
| Device | INT8 TOPS | Memory | Bandwidth GB/s | Power W | TOPS/W | Ridge ops/byte | Toolchain and formats | Data |
|---|---|---|---|---|---|---|---|---|
| NVIDIA Jetson AGX Orin calibrated SoM | 137 275 sparse | 64 GB LPDDR5 | 204.8 | 40 | 3.4 | 669 | TensorRT, cuDNN, CUDA ONNX, TensorRT, PyTorch | MLPerf result |
| NVIDIA Jetson Orin NX 16GB SoM | 100 | 16 GB LPDDR5 | 102 | 20 | 5 | 980 | TensorRT, cuDNN, CUDA ONNX, TensorRT, PyTorch | MLPerf result |
| SiMa.ai MLSoC SoM/PCIe | 50 | 16 GB LPDDR4 | 30 est. | 10 | 5 | 1667 | Palette, ONE Platform ONNX, TFLite, SiMa | MLPerf result |
| NVIDIA Jetson Orin Nano SoM | 40 67 sparse | 8 GB LPDDR5 | 68 | 12 | 3.3 | 588 | TensorRT, cuDNN, CUDA ONNX, TensorRT, PyTorch | vendor datasheet |
| Mobilint MA35D M.2/PCIe | 30 | On-chip (20 MB) | 600 est. | 5 | 6 | 50 | Mobilint SDK ONNX, Mobilint | estimated |
| Hailo-8 calibrated M.2/mPCIe/PCIe | 26 | On-chip (32 MB) | 1000 est. | 2.5 | 10.4 | 26 | HailoRT, Dataflow Compiler ONNX, TFLite, HailoRT | independent benchmark |
| DeepX DX-M1 M.2 2280 | 25 | 4 GB LPDDR5 | 40 est. | 3 | 8.3 | 625 | DXNN ONNX, TFLite, DeepX | vendor datasheet |
| Blaize Pathfinder P1600 SoM | 16 | 8 GB LPDDR4 | 25 est. | 7 | 2.3 | 640 | Blaize Picasso ONNX, TFLite | estimated |
| Qualcomm RB5 SoM/board | 15 | 8 GB LPDDR5 | 44 est. | 10 | 1.5 | 341 | SNPE, QNN ONNX, TFLite, SNPE, QNN | vendor datasheet |
| Hailo-8L (RPi AI Kit) M.2 | 13 | On-chip (16 MB) | 1000 est. | 1.5 | 8.7 | 13 | HailoRT, Dataflow Compiler ONNX, TFLite, HailoRT | independent benchmark |
| Qualcomm RB6 (QCS6490) SoM/board | 12 | 12 GB LPDDR5 | 51 est. | 12 | 1 | 235 | SNPE, QNN ONNX, TFLite, QNN | vendor datasheet |
| Intel Core Ultra NPU Integrated (laptop SoC) | 11 | 32 GB LPDDR5X | 120 | 20 | 0.6 | 92 | OpenVINO, DirectML, ONNX RT ONNX, OpenVINO | independent benchmark |
| TI TDA4VM (Jacinto) SoM | 8 | 4 GB LPDDR4 | 17 | 15 | 0.5 | 471 | TIDL ONNX, TFLite, TIDL | vendor datasheet |
| Rockchip RK3588 NPU SoM/board | 6 | 8 GB LPDDR5 | 19 | 8 | 0.8 | 316 | RKNN ONNX, RKNN, TFLite | independent benchmark |
| MediaTek Genio 1200 APU SMARC SoM | 4.8 | 8 GB LPDDR4X | 17 est. | 6 | 0.8 | 282 | NeuroPilot ONNX, TFLite, NeuroPilot | estimated |
| Google Coral Edge TPU USB/M.2/PCIe | 4 | On-chip (6.91 MB) | 64 est. | 2 | 2 | 63 | EdgeTPU Compiler TFLite | independent benchmark |
| Amlogic A311D2 NPU SoM | 3.2 | 4 GB LPDDR4 | 12 est. | 6 | 0.5 | 267 | Tengine, Vivante ONNX, TFLite, Tengine | vendor datasheet |
| AMD Xilinx KV260 SoM | 1.4 | 4 GB LPDDR4 | 19.2 | 10 | 0.1 | 73 | Vitis-AI ONNX, Vitis-AI | independent benchmark |
| Intel Movidius Myriad X end of life USB/M.2 | 1 | 0.5 GB LPDDR4 | 6.4 est. | 1.5 | 0.7 | 156 | OpenVINO OpenVINO, ONNX | vendor datasheet |
| Kneron KL720 USB/M.2 | 1 | 0.5 GB LPDDR4 | 4 est. | 1 | 1 | 250 | Kneron SDK ONNX, TFLite | estimated |
| NXP i.MX 93 Neutron NPU SoM | 0.5 | 2 GB LPDDR4X | 6.4 | 2 | 0.3 | 78 | eIQ TFLite, ONNX | vendor datasheet |
Why TOPS doesn't tell you FPS
Peak TOPS is an upper bound that real networks rarely reach. Two limits sit below it:
- Memory bandwidth. In the roofline model (Williams, Waterman and Patterson, 2009), a layer that does fewer operations per byte it moves than the device's ridge point is memory-bound. It runs at bandwidth × intensity, however many TOPS the chip has. On the NVIDIA Jetson Orin Nano the ridge point is about 588 ops/byte, so a layer needs that much arithmetic intensity to use the full 40 TOPS.
- Utilization. Compilers rarely keep every MAC busy. ModelVisio assumes typical utilization per workload, for example 40% for CNN classification, 25% for detection and 18% for transformers on a mature toolchain.
Where a published benchmark exists, utilization is calibrated from it instead: NVIDIA Jetson AGX Orin (MLPerf Inference v3.1 closed-edge ResNet-50 offline); Hailo-8 (Hailo AI SW Suite v2024-10 ResNet-50 (batch 8)). Results for these devices are marked calibrated in the app. All other devices use estimates and are labelled that way. A model whose memory footprint exceeds the device's memory scores at most 15 out of 100.
What happens to unsupported ops
When the compiler meets an op the accelerator can't run, the device's fallback behaviour decides the cost:
- Per-op CPU fallback: NVIDIA Jetson AGX Orin, NVIDIA Jetson Orin NX 16GB, SiMa.ai MLSoC, NVIDIA Jetson Orin Nano, Mobilint MA35D, DeepX DX-M1, Blaize Pathfinder P1600, Qualcomm RB5, Qualcomm RB6 (QCS6490), Intel Core Ultra NPU, TI TDA4VM (Jacinto), Rockchip RK3588 NPU, MediaTek Genio 1200 APU, Amlogic A311D2 NPU, AMD Xilinx KV260, Intel Movidius Myriad X, Kneron KL720, NXP i.MX 93 Neutron NPU.
- One partition: everything after the first unsupported op runs on CPU: Google Coral Edge TPU.
- No CPU fallback: unsupported ops must be cut from the graph: Hailo-8, Hailo-8L (RPi AI Kit).
See unsupported operators on Coral, RKNN, Hailo and Kneron for the op lists ModelVisio checks.
Sources
- NVIDIA Jetson AGX Orin: MLPerf Inference v3.1 closed-edge ResNet-50 ~6,424 fps → ~51 achieved INT8 TOPS (~37% of 137 dense); developer.nvidia.com/embedded/jetson-benchmarks
- NVIDIA Jetson Orin NX 16GB: NVIDIA Jetson datasheet; 100 TOPS (157 super); MLPerf-class ResNet-50 ~2,640 fps 157 TOPS in JetPack 6.2 'super' mode.
- SiMa.ai MLSoC: MLPerf Inference v3.0 closed-edge; SiMa.ai press Gen1 MLSoC won MLPerf v3.0 closed-edge ResNet-50 single-stream vs Orin. Bandwidth estimated.
- NVIDIA Jetson Orin Nano: NVIDIA Jetson Orin Nano datasheet; 40 TOPS / 68 GB/s (67 TOPS / 102 GB/s super) Super mode (JetPack 6.2): 67 sparse TOPS, 102 GB/s, $249 devkit.
- Mobilint MA35D: Mobilint MA35D product brief Peak TOPS [V]; bandwidth & achieved throughput not sourced; low-confidence.
- Hailo-8: Hailo AI SW Suite v2024-10 benchmark table; HailoRT log (1371.26 fps, 3.39 ms, 3.96 W) DRAM-free dataflow: weights resident on-chip. Official: ResNet-50 1,371 fps @3.7ms 375 FPS/W (~42% util); YOLOv5m-640 242 fps 45 FPS/W (~24% util). Models too large for on-chip SRAM cannot deploy (no external memory).
- DeepX DX-M1: DeepX DX-M1 datasheet; DXNN SDK docs 25 INT8 TOPS; M1 = 4-8GB external LPDDR5, PCIe Gen3 x4. Bandwidth estimated.
- Blaize Pathfinder P1600: Blaize Pathfinder P1600 brief ~16 TOPS class; key specs incompletely sourced; low-confidence.
- Qualcomm RB5: Qualcomm RB5 datasheet (15 TOPS AI Engine) QRB5165 (Snapdragon 865-class). Achieved benchmarks not published. Peak TOPS [V], bandwidth estimated.
- Hailo-8L (RPi AI Kit): Hailo-8L datasheet; community RPi5 benchmarks Entry-grade Hailo-8. YOLOv8s-640 ~80-120 fps on RPi5 PCIe Gen3 [B].
- Qualcomm RB6 (QCS6490): Qualcomm QCS6490/Dragonwing datasheet (12 dense TOPS) QCS6490 6th-gen AI Engine = 12 DENSE TOPS (corrects the common 20-TOPS marketing figure). Bandwidth estimated.
- Intel Core Ultra NPU: Intel Core Ultra spec; PRoof ICPP'24 (perf 'significantly deviated from theoretical') Meteor Lake NPU 3720 'AI Boost'. PRoof: achieved ~24-29% of peak; many models failed to run under OpenVINO 2024 → low-confidence (nn-Meter: analytical prediction least accurate on NPU/VPU).
- TI TDA4VM (Jacinto): TI TDA4VM datasheet; TIDL docs ~8 TOPS DLA (C7x DSP + MMA). Automotive-grade.
- Rockchip RK3588 NPU: Rockchip RK3588 TRM; community RKNN benchmarks (~19 GB/s LPDDR) 6 TOPS (3-core). ResNet18 ~244 fps @4.09ms; TinyLlama-1.1B ~10-15 tok/s. RKNN v1.x uses CPU fallback for SiLU (v2.0+ native).
- MediaTek Genio 1200 APU: MediaTek Genio 1200 (MT8395) brief ~4.8 TOPS APU. MediaTek does not disclose its TOPS methodology; low-confidence.
- Google Coral Edge TPU: coral.ai/docs/edgetpu/models-intro (single-partition fallback); coral.ai/docs/edgetpu/compiler (6.91 MiB RAM); MobileNet v2 ~400 fps INT8-only; 6.91 MiB usable param cache; once an unsupported op appears, it + the rest of the graph run on host CPU.
- Amlogic A311D2 NPU: Amlogic A311D2 datasheet 3.2 TOPS Vivante NPU. Bandwidth estimated.
- AMD Xilinx KV260: Kria KV260 / K26 datasheet; Xilinx DPU B4096 benchmarks; SOM power measured >10W [B] Zynq UltraScale+ DPU (~1.4 TOPS-class for typical B3136 config; corrects the 3.2 figure). Custom (non-Model-Zoo) models documented as hard to deploy.
- Intel Movidius Myriad X: Intel Movidius Myriad X (MA2485) datasheet EOL (~2020). ~1 TOPS DNN (4 TOPS aggregate). Legacy analysis only.
- Kneron KL720: Kneron KL720 product page ~0.9-1.4 TOPS class; bandwidth not sourced; low-confidence.
- NXP i.MX 93 Neutron NPU: NXP i.MX 93 datasheet; eIQ docs (op CPU-fallback list) eIQ Neutron NPU 0.5 TOPS (256 MAC/cycle @1GHz). Documented CPU fallback for ARG_MAX, RESIZE_BILINEAR, etc.
Full methodology and citations: roofline scoring research. Spot an outdated number? Open an issue with the datasheet.