What is TOPS?
TOPS stands for trillions of operations per second. Chipmakers use it to rate peak AI compute, especially for NPUs (neural processing units). The figure is usually measured at low precision, such as INT8 or lower. For scale, Microsoft's Copilot+ PC label requires an NPU rated at 40 TOPS or more. TOPS is a theoretical peak, and figures quoted at different precisions cannot be compared directly. It also says little about local LLM generation speed. Generating each token is mostly limited by memory bandwidth, not raw compute. Many popular LLM runners also use the GPU rather than the NPU. That is why this site lists memory bandwidth and measured tokens per second instead. For example, the Mac Studio (M3 Ultra) has 819 GB/s and reaches 14 tok/s on a 70B model. The GMKtec EVO-X2 has 256 GB/s and reaches 5 tok/s.