Which Mac runs which AI model?

How we work it out

Memory needed = parameters × bits per weight ÷ 8 + KV cache + about 1 GB for the app's working buffers.

Left for other apps = the Mac's memory − memory needed − about 4 GB for macOS itself.

An 8-billion-parameter model at 4-bit (Q4_K_M, 4.89 bits a weight) is 8 × 4.89 ÷ 8 ≈ 4.9 GB of weights. The KV cache is the model's short-term memory of the conversation: for each token of context, every attention layer stores 2 × KV heads × head size numbers, 2 bytes each (the apps' 16-bit default). We read each model's layer counts and head sizes from its config.json; layers with sliding-window attention stop growing at their window, and the newer hybrid models (Qwen3.5 and later) keep a cache in only some layers. Mixture-of-experts models (marked "active") must hold all their parameters in memory but read only the active ones for each token, which is why they're fast for their size.

GPU memory limit: by default macOS lets the GPU use only part of the memory, reportedly about two-thirds on Macs with 36GB or less and three-quarters on bigger ones (Apple doesn't publish the figure; this is what llama.cpp users report, unverified). A model above that share is marked Tight even with memory to spare; LM Studio and similar apps explain how to raise the limit.

Speed (rough): writing a reply reads the active weights once per token, so tokens a second ≈ memory bandwidth × 0.65 ÷ GB of weights (× 0.4 for mixture-of-experts models). It's a ballpark from bandwidth alone, not a benchmark; reading a long prompt first is a separate wait that depends on the GPU.

We count memory in GB of a billion bytes, and Apple's "16GB" is 16 GiB (about 17.2 GB), so the results err on the safe side. Bits per weight are llama.cpp's own figures (llama.cpp quantize README (bits per weight table), checked Oct 23, 2026); MLX's 4-bit and 8-bit files are close. Vision and audio parts of multimodal models are counted in the parameters when the model card counts them.