LocalLLM Rigs

What is a GPU?

A GPU (graphics processing unit) is a processor with many small cores built to run the same calculation across large amounts of data at once. That parallelism suits the matrix math in language models, so inference tools offload as much of the model as possible to the GPU. Every machine in this site's catalog uses an integrated GPU that shares unified memory with the CPU rather than a separate graphics card with its own VRAM. For example, the Apple Mac Studio (M3 Ultra) has an 80-core GPU. GPU compute mainly speeds up prompt processing, which is reading your input. Token generation, the speed of the reply, depends more on memory bandwidth. Software support also differs by vendor: NVIDIA's CUDA is the most widely supported, and Apple machines lack it, so some AI tools do not run on them.