What is Q4_K_M?
Q4_K_M is a GGUF quantization type used by llama.cpp and tools built on it, such as Ollama and LM Studio; that exact string appears in GGUF filenames. "Q4" means 4-bit, "K" is llama.cpp's k-quant method, and "M" the medium variant, which keeps some sensitive tensors at higher precision, averaging roughly 4.8 bits per weight. It is a common default because quality stays close to the 16-bit original. A 70B model in Q4_K_M is around 42 GB, which fits on 64 GB machines such as the Apple Mac mini (M4 Pro, 64GB) or the Minisforum MS-S1 Max, at about 5 to 5.5 tok/s per the published numbers. On the Mac, macOS by default caps GPU memory at about 75%, roughly 48 GB, leaving only ~6 GB for the KV cache, so long contexts get tight. The base Mac mini (M4, 16GB) cannot load it. Compared with Q4_K_S, files are slightly larger and generation slightly slower.