ModelVisio Open the app
ModelVisio / Edge AI accelerators

Edge AI accelerator comparison: 21 NPUs, GPUs and TPUs

The 21 edge accelerators ModelVisio scores models against, with the numbers that decide how fast a model runs: dense INT8 TOPS, memory bandwidth, typical power and the toolchain you compile with. Each figure carries its source.

Score your model on all 21  Drop a model file in the browser. It is never uploaded.

Specifications

Sorted by dense INT8 TOPS. Ridge point = TOPS ÷ bandwidth, in operations per byte. "est." marks an estimated bandwidth.

DeviceINT8 TOPSMemoryBandwidth GB/sPower WTOPS/WRidge ops/byteToolchain and formatsData
NVIDIA Jetson AGX Orin calibrated
SoM
137
275 sparse
64 GB LPDDR5204.8403.4669TensorRT, cuDNN, CUDA
ONNX, TensorRT, PyTorch
MLPerf result
NVIDIA Jetson Orin NX 16GB
SoM
10016 GB LPDDR5102205980TensorRT, cuDNN, CUDA
ONNX, TensorRT, PyTorch
MLPerf result
SiMa.ai MLSoC
SoM/PCIe
5016 GB LPDDR430 est.1051667Palette, ONE Platform
ONNX, TFLite, SiMa
MLPerf result
NVIDIA Jetson Orin Nano
SoM
40
67 sparse
8 GB LPDDR568123.3588TensorRT, cuDNN, CUDA
ONNX, TensorRT, PyTorch
vendor datasheet
Mobilint MA35D
M.2/PCIe
30On-chip (20 MB)600 est.5650Mobilint SDK
ONNX, Mobilint
estimated
Hailo-8 calibrated
M.2/mPCIe/PCIe
26On-chip (32 MB)1000 est.2.510.426HailoRT, Dataflow Compiler
ONNX, TFLite, HailoRT
independent benchmark
DeepX DX-M1
M.2 2280
254 GB LPDDR540 est.38.3625DXNN
ONNX, TFLite, DeepX
vendor datasheet
Blaize Pathfinder P1600
SoM
168 GB LPDDR425 est.72.3640Blaize Picasso
ONNX, TFLite
estimated
Qualcomm RB5
SoM/board
158 GB LPDDR544 est.101.5341SNPE, QNN
ONNX, TFLite, SNPE, QNN
vendor datasheet
Hailo-8L (RPi AI Kit)
M.2
13On-chip (16 MB)1000 est.1.58.713HailoRT, Dataflow Compiler
ONNX, TFLite, HailoRT
independent benchmark
Qualcomm RB6 (QCS6490)
SoM/board
1212 GB LPDDR551 est.121235SNPE, QNN
ONNX, TFLite, QNN
vendor datasheet
Intel Core Ultra NPU
Integrated (laptop SoC)
1132 GB LPDDR5X120200.692OpenVINO, DirectML, ONNX RT
ONNX, OpenVINO
independent benchmark
TI TDA4VM (Jacinto)
SoM
84 GB LPDDR417150.5471TIDL
ONNX, TFLite, TIDL
vendor datasheet
Rockchip RK3588 NPU
SoM/board
68 GB LPDDR51980.8316RKNN
ONNX, RKNN, TFLite
independent benchmark
MediaTek Genio 1200 APU
SMARC SoM
4.88 GB LPDDR4X17 est.60.8282NeuroPilot
ONNX, TFLite, NeuroPilot
estimated
Google Coral Edge TPU
USB/M.2/PCIe
4On-chip (6.91 MB)64 est.2263EdgeTPU Compiler
TFLite
independent benchmark
Amlogic A311D2 NPU
SoM
3.24 GB LPDDR412 est.60.5267Tengine, Vivante
ONNX, TFLite, Tengine
vendor datasheet
AMD Xilinx KV260
SoM
1.44 GB LPDDR419.2100.173Vitis-AI
ONNX, Vitis-AI
independent benchmark
Intel Movidius Myriad X end of life
USB/M.2
10.5 GB LPDDR46.4 est.1.50.7156OpenVINO
OpenVINO, ONNX
vendor datasheet
Kneron KL720
USB/M.2
10.5 GB LPDDR44 est.11250Kneron SDK
ONNX, TFLite
estimated
NXP i.MX 93 Neutron NPU
SoM
0.52 GB LPDDR4X6.420.378eIQ
TFLite, ONNX
vendor datasheet

Why TOPS doesn't tell you FPS

Peak TOPS is an upper bound that real networks rarely reach. Two limits sit below it:

Where a published benchmark exists, utilization is calibrated from it instead: NVIDIA Jetson AGX Orin (MLPerf Inference v3.1 closed-edge ResNet-50 offline); Hailo-8 (Hailo AI SW Suite v2024-10 ResNet-50 (batch 8)). Results for these devices are marked calibrated in the app. All other devices use estimates and are labelled that way. A model whose memory footprint exceeds the device's memory scores at most 15 out of 100.

What happens to unsupported ops

When the compiler meets an op the accelerator can't run, the device's fallback behaviour decides the cost:

See unsupported operators on Coral, RKNN, Hailo and Kneron for the op lists ModelVisio checks.

Sources

Full methodology and citations: roofline scoring research. Spot an outdated number? Open an issue with the datasheet.