LocalLLM Rigs

Bandwidth explained

Bandwidth is the rate at which data can travel over a connection, measured as an amount per second. On this site it usually means memory bandwidth, the speed at which the processor reads from RAM, given in GB/s. In this catalog that ranges from 120 GB/s on the Apple Mac mini (M4, 16GB) to 819 GB/s on the Apple Mac Studio (M3 Ultra). The term also shows up for networking. The Minisforum MS-S1 Max has dual 10GbE ports, which run at 10 gigabits per second each. That is roughly 1.25 gigabytes per second, far below memory bandwidth, so the units are easy to mix up. Watch whether a spec uses bits (Gb, lowercase b) or bytes (GB, uppercase B). Network bandwidth matters when you cluster several machines. Memory bandwidth matters for every machine running a model.