What NPU TOPS Ratings Really Mean: How to Judge an AI PC Beyond the 40 TOPS Line

Laptop spec sheets now lead with a single AI number: TOPS, or trillions of operations per second. Microsoft draws its Copilot+ PC line at an NPU (neural processing unit) that can run at more than 40 TOPS, and the latest chips from Qualcomm, Intel and AMD advertise anywhere from 50 to 80. Those figures are real, but they are peak numbers measured in a specific way, and they say less about how fast your own AI workload will run than the marketing suggests.

This guide explains what an NPU is, how to read a TOPS rating, why the "platform TOPS" figure on a box is not the NPU figure, and how to check whether an app is actually using the NPU on a Windows machine. It sticks to what the chipmakers and Microsoft publish in their own documentation.

Key facts at a glance

• The 40 TOPS line: Microsoft says many new Windows AI features require an NPU able to run at 40+ TOPS, the definition of a Copilot+ PC.
• TOPS are usually INT8 peak figures: Intel, AMD and Qualcomm quote NPU throughput for 8-bit integer maths, not the 16- or 32-bit formats many models ship in.
• Platform TOPS are bigger than NPU TOPS: AMD's Ryzen AI 9 HX 370 lists up to 80 TOPS overall but up to 50 on the NPU; Intel's Core Ultra X7 358H lists 50 NPU TOPS and 122 on its GPU.
• Software decides: an NPU does nothing unless the app targets it; Microsoft now recommends Windows ML, which picks a hardware-specific execution provider automatically.
• You can check it: Task Manager on NPU-equipped PCs shows NPU utilisation in real time.

Who this guide is for

It is written for IT buyers choosing a fleet of AI PCs, developers deciding whether to run a model locally or in the cloud, and anyone in the US, UK, Canada or Australia comparing laptops whose spec sheets now quote TOPS next to battery life. If you mainly run cloud AI services in a browser, the NPU matters little to you today. If your organisation wants on-device features such as local transcription, image effects or small language models, it matters a lot.

What an NPU actually does

An NPU is a processor built for the dense matrix maths inside neural networks. Microsoft describes it as a specialised chip for AI-intensive work such as real-time translation and image generation, able to process large amounts of data in parallel while using less energy than a CPU or GPU for the same AI task. That efficiency is the point: on a laptop, an NPU lets an AI feature run continuously without draining the battery or spinning up the fans.

The NPU works alongside, not instead of, the CPU and GPU. Microsoft notes that Windows 11 assigns processing to the most appropriate place. In its own mapping, the NPU is meant for battery-efficient, sustained on-device inference; the GPU handles high-throughput image, video and generative AI work; and the CPU is the universal fallback, often the lowest-latency choice for small models.

That division explains why a machine with a modest NPU but a strong GPU may still run a large image model faster than a rival with a bigger NPU figure. The NPU wins on efficiency for always-on tasks. It does not automatically win on raw speed for everything.

How to read a TOPS rating

TOPS counts operations, at a given precision, at peak. The chip vendors' own spec pages show the precision: Intel lists "NPU Peak TOPS (Int8)", and Qualcomm's Snapdragon X2 Elite product brief quotes up to 80 TOPS at INT8. INT8 means 8-bit integers. Microsoft's developer guide explains why that matters: many NPUs support only lower-bit integer maths, so models trained in larger formats such as FP32 must be converted, or quantised, before they can run on the NPU at all.

Platform TOPS add the CPU and GPU together. Here is how three current chips describe themselves on their makers' spec pages:

Sources: Qualcomm, Intel and AMD product specifications.

The practical rule: when you compare laptops, compare NPU TOPS with NPU TOPS. A box that says "80 TOPS" may be quoting the whole platform, and the NPU alone may be well under that.

Peak is not sustained, and TOPS is not everything. A peak figure assumes the chip is fully fed with work. Real models also depend on memory bandwidth, the precision the model was converted to and how well the software maps each layer onto the hardware. Microsoft's own performance tooling lists NPU memory bandwidth as a separate metric to examine, which is a hint that compute alone does not set the speed.

How software reaches the NPU

Apps must be written to use it. Microsoft is explicit that the NPU, like any hardware resource, needs software specifically programmed to take advantage of it. An older app that only knows CPU and GPU will never touch the NPU, however many TOPS it has.

Windows ML is now the recommended route. Microsoft says the recommended way to access the NPU and GPU for AI has shifted from DirectML to Windows ML. Windows ML uses ONNX Runtime as its inference engine and pairs it with hardware-tuned execution providers: OpenVINO for Intel NPUs, QNN for Qualcomm and VitisAI for AMD. It detects the hardware, selects the most performant provider, downloads it if needed through Windows Update, and falls back to the GPU or CPU if the preferred provider fails.

Models usually need preparing first. Microsoft points developers to Qualcomm AI Hub and the ONNX Model Zoo for ready-converted models, and to its Olive tool for compressing and optimising their own. It also recommends pre-compiling a model for its target provider and caching the result, because compilation can otherwise turn the first launch into a multi-second, or even multi-minute, cold start.

For Microsoft's own built-in features, the Windows AI APIs check readiness at runtime. An app asks for one of four states: Ready, NotReady (the model must be downloaded), DisabledByUser or NotSupportedOnCurrentSystem. Microsoft says most of these APIs require a Copilot+ PC with an NPU, while some can also run on a GPU or CPU.

How to check whether the NPU is being used

Use Task Manager first. Microsoft says that on devices with NPUs, Task Manager now shows NPU resource usage alongside CPU, memory, disk and GPU, including utilisation and driver version. Open it, go to the Performance tab, start the AI feature you care about, and watch whether the NPU graph moves. If it stays flat while the GPU or CPU spikes, the app is not using the NPU.

Developers can go deeper. Windows Performance Recorder includes a Neural Processing profile that records NPU activity, and Windows Performance Analyzer can show which processes used the NPU and the call stacks that submitted the work. Microsoft's guidance also lists ONNX Runtime events, from version 1.17, that report model load time and per-inference timings.

What to do if things go wrong

• The NPU never shows activity: confirm the app supports your NPU vendor, and update Windows so the right execution provider can be delivered. Windows ML is designed to fall back silently to GPU or CPU, so a working feature does not prove the NPU is in use.
• A Windows AI feature is missing: check whether the device meets the 40+ TOPS Copilot+ requirement, and whether the feature has been switched off by the user or by policy, which the API reports as DisabledByUser.
• A custom model runs slowly or fails on the NPU: check its precision. Microsoft says many NPUs need integer formats such as INT8, so an unconverted FP32 model may fall back to another processor.

The vendors' view, and a counterpoint

Microsoft presents the NPU as the foundation for all-day battery life and its most advanced on-device features, and the chipmakers compete on ever-higher NPU figures. That is a fair reflection of where efficiency gains are coming from. The counterpoint is in the vendors' own fine print: Intel's Core Ultra X7 358H quotes more than twice as many TOPS on its GPU as on its NPU, and AMD's overall figure is 60% higher than its NPU figure. For heavy, bursty AI work, the GPU remains the bigger engine. The NPU's case is efficiency and always-on availability.

Conclusion

TOPS is a useful first filter, not a verdict. When you buy or build for on-device AI:

1. Compare like with like: NPU TOPS at INT8, not platform totals.
2. Treat 40+ NPU TOPS as the entry ticket for Microsoft's Copilot+ features on Windows.
3. Check software support for your NPU vendor before you count on it, and prefer apps built on Windows ML or ONNX Runtime.
4. Verify in Task Manager that the NPU graph actually moves when your AI feature runs.
5. Budget for model preparation: quantise to integer formats and pre-compile to avoid slow first launches.