Blog 

Datasheet TOPS is a vendor peak claim, not a prediction of sustained FPS, end-to-end latency, or multi-camera capacity. Freeze the workload first: model and input size, precision and accuracy floor, end-to-end latency and throughput, camera or stream count, and the exact runtime path you will ship. Then apply four gates—convertible, accurate enough, full pipeline meets the deadline, and thermals/power stable in the real form factor. Compare only under the same workload and conditions; never multiply or rank raw TOPS. An RK3588 board such as the youyeetoo YY3588 lists up to 6 TOPS plus an RKNN path—a candidate route to validate, not verified FPS.

Why peak TOPS does not answer “is it enough?”

TOPS (tera-operations per second) usually describes a theoretical INT8 multiply-accumulate peak under vendor assumptions. Those assumptions often do not match a real vision pipeline. If you need how to evaluate edge AI NPU performance for computer vision without TOPS, start here: separate peak arithmetic claims from what your pipeline actually delivers.

What TOPS can do:

• Place a part in a coarse compute class so you can shortlist similar accelerators.
• Suggest a rough upper bound on arithmetic throughput for supported integer precisions.

What TOPS cannot do:

• Prove that your operators compile to the NPU.
• Predict end-to-end FPS after capture, resize, NMS, tracking, encode, or network send.
• Guarantee accuracy after quantization.
• Guarantee sustained performance inside a sealed, warm enclosure.
• Reveal memory bandwidth, host overhead, or silent CPU fallback.

Vendors may count a multiply-accumulate as one or two operations, quote different precisions, assume sparse utilization, or use peak figures that no customer pipeline sustains. Cross-vendor TOPS comparisons are therefore easy to misread. If two datasheets both say “6 TOPS,” that alone does not tell you which board will meet your deadline.

For board selection by task (OS path, camera path, x86 vs ARM), see Affordable SBCs for Real-Time Computer Vision: What to Check Before You Buy. This article answers a narrower question: once you are looking at an NPU, how do you decide whether your computer-vision workload will fit without trusting marketing TOPS.

Freeze the computer-vision workload (what to write down before shopping)

If you cannot state the model, input rate, and latency budget in one sentence, you are not ready to select an NPU.

Record these non-negotiables before reading comparison tables:

Keep this contract fixed while you evaluate candidates. Changing the model mid-comparison restarts the test.

A four-gate sufficiency test (convert → correctness → pipeline → soak/thermal)

An NPU is sufficient only when all four gates pass on the target hardware (or a close development board) under your frozen workload. Reuse this NPU benchmarking methodology for detectors and classifiers alike—YOLO-family models, ResNet-style backbones, or a custom graph—as long as the workload contract stays fixed.

Gate 1 — Convertible

Check graph compatibility before buying hardware: the production graph must compile for the candidate’s toolchain and target SoC. For Rockchip RK3588-class boards, the documented path is conversion with RKNN-Toolkit2 to an RKNN model, then inference with RKNN-Toolkit-Lite2 or the RKNN Runtime—not dropping a raw .onnx or PyTorch checkpoint onto the NPU. Ask for a compilation report, the share of the graph that executes on the NPU, and known unsupported operators. A conversion success is only Gate 1. For the ONNX→RKNN convertability checklist, see Can You Run an ONNX Model on the RK3588 NPU? RKNN Compatibility Checklist.

Gate 2 — Correct enough

Compare the converted model’s accuracy on your validation set against the FP32 (or desktop) baseline. Record mAP, IoU, classification accuracy, or the metric your product actually uses. INT8 often costs a few points; recovery depends on calibration and graph support. If accuracy is outside tolerance, the NPU is not sufficient for this model—even if latency looks good.

Gate 3 — Full pipeline meets the deadline

Measure capture → preprocess → NPU inference → postprocess → output action as one system. Include color conversion, letterbox/resize, NMS, tracking, overlays, encoding, and network or control I/O if they are in production. Pass only if P95/P99 latency and sustained throughput meet the budget with headroom while other required services run.

Gate 4 — Soak and thermal

Run long enough to expose throttling: typically tens of minutes to an hour or more under representative ambient and enclosure conditions. Record watts if power is constrained, board temperature, and whether FPS or latency drifts after warm-up. A peak demo that collapses in a sealed box fails Gate 4.

Decision rule: all four gates pass with margin → the NPU path is sufficient for this workload. Any gate fails → keep evaluating a different model scale, precision, duty cycle, cooling design, or a stronger compute path. Do not “fix” a failed gate with a higher TOPS number from another datasheet.

What to measure (and what not to trust): inference-only vs end-to-end

The realistic metrics for edge NPU computer vision—FPS, latency, and power under your duty cycle—come from the production pipeline, not from a datasheet row. Use the vendor production toolchain for the candidate—not a generic path that bypasses the NPU compiler if the product depends on that compiler.

Best practices:

• Warm up, then measure under realistic video or live camera input—not only synthetic tensors.
• Test with other processes and memory pressure if the device will share resources.
• Compare model scales (n/s/m) and precisions only when each variant is converted and validated.
• Check operator support early; unsupported layers that fall back to the CPU destroy latency and efficiency.

Do not trust:

• Datasheet TOPS as FPS.
• Vendor demo models as proof for your custom graph.
• Short burst scores without a soak.
• Relative claims such as “N× faster because TOPS is N× higher.” Arithmetic peaks do not scale linearly into pipeline FPS; ask memory bandwidth vs TOPS as a paired question, because a higher peak that starves the bus—or waits on the host—still fails Gate 3.

For multi-camera capacity specifically—ingress, decode, inference interval, and system overhead—use the method in How to Determine Camera Capacity on an Edge AI SBC (article-09). Do not derive camera count from a 6 TOPS figure.

Where an RK3588 / RKNN path fits as a conditional example

The following facts are taken from the official YY3588 materials (Wiki-grade product facts). They describe a candidate platform class. They do not prove a FPS or power winner for your model.

Worked-example stance: if your workload needs local NPU inference on an ARM board, your model converts cleanly through RKNN for rk3588, accuracy stays inside tolerance, and the full camera pipeline meets latency under soak in your enclosure, then an RK3588 board such as YY3588 is a reasonable candidate to prototype. If any gate fails, escalate model simplification, cooling, or a different accelerator class—do not multiply 6 TOPS into a promised speedup against an Intel N100 or any other part.

Related reading without rewriting those guides:

• Board-by-task selection: Affordable SBCs for Real-Time Computer Vision
• Conversion gates: ONNX on RK3588 NPU / RKNN checklist
• Camera capacity: How to Determine Camera Capacity on an Edge AI SBC

Product reference: YY3588 Wiki.

Purchase / validation checklist + when you still need a stronger path

Use the lists below as a realistic NPU evaluation checklist; they mirror the four gates rather than a TOPS scoreboard.

Before you buy

• ☐ Workload contract written (model, input, latency, accuracy, concurrency, runtime, envelope).
• ☐ Operator / compiler support checked for the exact graph; conversion command reproducible.
• ☐ Accuracy plan defined on your data, not only on a public demo set.
• ☐ Measurement plan includes e2e latency, sustained FPS, utilization, and soak—not TOPS.
• ☐ Cooling and enclosure constraints listed as acceptance tests, not afterthoughts.
• ☐ Fallback path identified if Gate 1–4 fails (smaller model, lower analysis rate, different SoC class, or x86/GPU path when Windows or existing software is mandatory).

On the board (or close EVK)

• ☐ Gate 1: production model converts for the target SoC.
• ☐ Gate 2: accuracy within tolerance.
• ☐ Gate 3: e2e P95/P99 and sustained throughput meet budget with headroom.
• ☐ Gate 4: soak shows stable thermals and no crippling throttle.
• ☐ Logs show no unexpected CPU-heavy fallback for critical operators.
• ☐ Memory footprint fits with the rest of the application.

When you still need a stronger (or different) path

Keep a stronger NPU, a discrete accelerator, or an x86/GPU host in evaluation when:

• The model will not convert, or too much of the graph stays on the CPU.
• Accuracy after quantization misses the product floor.
• End-to-end latency or multi-stream load fails even after model and pipeline tuning.
• Fanless or high-ambient constraints force throttling under soak.
• The application hard-requires Windows or an existing x86 stack (YY3588’s current official OS list does not include Windows; use an N100-class path such as youyeetoo K1 when that constraint dominates—K1 has no independent NPU in current Wiki material).

Bottom line

An edge AI NPU is sufficient when your exact quantized model, in the vendor’s compiler and runtime, meets your required end-to-end latency, accuracy, memory, power, and thermal limits on the actual board and enclosure. TOPS helps you shortlist. The four-gate test decides.

Identity: evidence-based how-to guide. Not a lab report. Product numbers above are vendor-documented capabilities for the YY3588 / RK3588 example and must be re-checked on the selected SKU and Wiki revision before purchase.