What does 70B mean?
70B means a model has roughly 70 billion parameters, the learned numbers stored in its weights. Size sets the memory you need. At 16-bit precision, 70 billion weights take about 140 GB. Quantized to around 4 bits (for example Q4_K_M), they shrink to roughly 40–43 GB, plus extra room for context. That is why 70B is a useful benchmark for this site's hardware. Machines with 64 GB or more of unified memory, such as the Apple Mac mini (M4 Pro, 64GB), can load a 4-bit 70B model. The Apple Mac mini (M4, 16GB) cannot, so its 70B speed is listed as 0 tok/s. Across the catalog, 70B generation speeds run from 4.5 tok/s (NVIDIA DGX Spark) to 14 tok/s (Apple Mac Studio M3 Ultra).