For a long time, there has been a fundamental bottleneck when using Ollama to execute large language models (LLM) on Mac: the bandwidth advantage of Unified Memory has never been fully utilized. Although the traditional llama.cpp with Metal acceleration can run the model on Apple chips, its performance has not been ideal because it has not been optimized for the Appke MLX architecture. Recently, Ollama Officially announcing the integration The Apple MLX framework brings a qualitative leap to local AI inference on Apple Silicon through a design directly optimized for unified memory architecture.
The latest version of Ollama integrates the Apple MLX framework: Mac native AI inference speed soars two times
Ollama said on its official blog that the new version of Ollama 0.19 uses the MLX unified memory architecture, which directly solves the performance loss problem of traditional inference engines that must frequently copy data between the CPU and GPU. Based on multiple independent testsdata, the speed advantage of MLX is quite significant: on the M4 Max chip, the inference speed of MLX can reach 60 to 70 tokens per second, while the traditional Ollama (llama.cpp with Metal) is about 35 tokens/s, a gap of nearly twice; if you look only at the prompt word processing (prefill) stage, the gap widens to five times.
Performance breakthrough of Qwen3.5-35B on M5 chip
According to official information released by Ollama, the Qwen3.5-35B-A3B model equipped with int4 quantization format can achieve a feedforward speed (prefill rate) of 1851 tokens per second and a decoding speed of 134 tokens per second on the M5 chip. This means that even a large model with a scale of 35 billion parameters can now achieve smooth real-time interaction on Apple’s high-end hardware, which is no longer just theoretically possible.
However, a complete 35B model requires a large amount of memory support.According to test, to take full advantage of this new architecture, users need a Mac device with more than 32GB of unified memory.
| Chip | Qwen3.5-35B-A3B Q4 via Ollama | Qwen3.5-35B-A3B Q4 via MLX | MLX advantage |
|---|---|---|---|
| M1 Pro (16GB) | ~15-18 tok/s | ~25-35 tok/s | ~1.5-2x |
| M3 Pro (18GB) | ~18-22 tok/s | ~35-45 tok/s | ~1.8-2x |
| M4 Pro (24GB) | ~25-30 tok/s | ~45-55 tok/s | ~1.8x |
| M4 Max (64GB) | ~30-38 tok/s | ~60-75 tok/s | ~2x |
| M3 Ultra (192GB) | ~35-45 tok/s | ~70-90 tok/s | ~2x |
Why MLX is faster: Deep optimization of the unified memory architecture
To understand the speed advantage of MLX, you must first understand the structural issues that traditional inference engines face on Apple Silicon. In the past, when llama.cpp was used with the Metal solution, the model weights and data generated during the calculation process still needed to be copied back and forth between the CPU memory space and the GPU memory space. This data copying process will not disappear in the unified memory architecture. Instead, it will cause additional overhead due to the special memory management design of Apple chips.
The core breakthrough of MLX is that it is designed from the ground up based on Apple’s unified memory architecture. MLX directly completes the storage and calculation of model weights in unified memory, no longer distinguishing between CPU memory and GPU memory in the traditional sense, completely eliminating the performance loss of cross-space copying4]. This kind of native optimization based on hardware characteristics is the fundamental reason why MLX can show overwhelming speed advantages on the same hardware.
The design direction of this Ollama 0.19 update not only improves performance, but also targets the actual needs of developers’ workflow. The new version is specially optimized for coding agent scenarios and supports mainstream development tools such as Claude Code and OpenClaw.
Caching and Checkpointing Technology: Comprehensive Improvements to the Conversation Experience
In addition to pure inference speed, Ollama 0.19 also makes important improvements to the memory management mechanism. The new version of the caching mechanism allows the reuse of calculated content between multiple rounds of dialogue, greatly reducing the resource consumption of repeatedly processing the same prompt words. In addition, “smart checkpointing” technology can reduce redundant calculations in key aspects of prompt word processing, further shortening response delays.
The combined effect of these improvements means that users will experience a smoother conversation experience in actual operations. Not only will the response speed be faster, but the model will also be able to maintain more stable performance when processing lengthy conversation contexts, without significant latency increases due to context accumulation.
Impact on local AI inference ecology and future prospects
Ollama’s integration of MLX has far-reaching implications for the entire local AI inference ecosystem. Before this, if developers and researchers wanted to run large models on Apple hardware, they often had to make a difficult choice between “speed” and “convenience”: traditional solutions were limited in speed, and complete optimization solutions often required tedious manual compilation and parameter tuning. The launch of Ollama 0.19 has lowered this technical threshold to some extent, allowing more users to directly benefit from high-performance local inference in the Apple ecosystem.
However, this technology only increases the feedforward and decoding speed of the model. If you want to use a model with larger parameters, you still need to use a Mac model with higher specifications and larger memory. This will not be different due to the progress of MLX. Just like the memory compression technology just launched by Google, it will not allow models with smaller memory capacity to smoothly use models with larger parameters (but it will be somewhat improved). Interested friends can try it themselves.
Source: KOCPC Chinese