As large language models become increasingly powerful, many people want to set up AI models locally. However, running an AI model with over 100 billion parameters requires at least six AI GPUs—one is definitely not enough—and the total cost easily exceeds one million. Recently, someone on China’s Bilibili shared a test of the full-blooded M3 Ultra running the 671B-parameter DeepSeek R1 model locally. The results showed speeds faster than eight A100 GPUs, but at a total price several times cheaper.
This shows that, compared with GPUs, the M3 Ultra is currently one of the most cost-effective options for running large language models locally; in fact, many M-series chips are more cost-effective than GPUs.
The M3 Ultra is arguably one of the most cost-effective ways to run large language models locally.
Recently, the Chinese Bilibili channel “Although But Zhang Heihei” uploaded a video titled “Speed Comparable to the Official Website? Full-Power M3 Ultra Reasoning with DeepSeek R1’s 600 Billion Parameters,” sharing his real-world benchmark data from running DeepSeek R1 on the M3 Ultra.
First of all, you need to understand that, generally speaking, to run a DeepSeek R1 large model with up to 600 billion parameters locally, you would likely need a professional server costing over a million yuan, equipped with 6 to 8 A100 GPUs. This is simply not something an ordinary person can afford.
But with the full-spec M3 Ultra, you only need this one Mac to achieve speeds comparable to 8 A100s.
Currently, DeepSeek has two most powerful models: one is R1, and the other is V3. In simple terms, when you open the DeepSeek official website to chat, the model used is V3, and if you enable deep thinking below, it becomes the R1 model:
With 8 A100s, running DeepSeek R1 671B delivers 16.41 Tokens/s. Meanwhile, the M3 Ultra in the same GGUF format achieves 15.78 Tokens/s, but after switching to the MLX format, which leverages the advantage of unified memory, the speed surpasses the 8 A100 setup, reaching 19.17 Tokens/s.
DeepSeek V3 671B is also the same, with speed reaching 19.66Tokens/s:
Of course, this doesn’t mean the full-power M3 Ultra completely defeats the A100. The video also explains that for a single user running inference on a single model, memory bandwidth and capacity are what matter most, which can’t fully showcase the true strength of eight A100 GPUs.
If it comes to multi-user inference and large model training, then the M3 Ultra simply can’t compete.
The full-spec M3 Ultra he uses is equipped with 512GB unified memory, along with other customized specifications, for a total price of 74,249 RMB, approximately 336,000 NT dollars:
For most people, this one really does have luxurious specs. If you don’t need to run models with such large parameter counts, you don’t need to max out the unified memory, and you’ll save a lot on the price.
As for whether the processor has to be the M3 Ultra? If you can, of course it’s recommended.
The figure below shows LLM inference speed tests comparing the M3 Ultra with other M-series chips. As can be seen, whether it’s Llama 3.1 70B, Gemma2 27B, or Qwen 2.5 14B, the M3 Ultra is clearly faster than the other M-series models. Compared with the M2 Ultra, speeds have improved by 13%, 34%, and 18%, respectively.
“Speed comparable to the official website? Full-power M3 Ultra inference of DeepSeek R1 with 600 billion parameters” Full video:
Source: KOCPC Chinese