Dark Side of the Moon (Moonshot AI) has been officially open sourced these days Kimi K3 Model weights and technical reports, this hybrid expert (MoE) SOTA open source model with 2.8 megabytes of parameters has immediately become a dream product that everyone in the AI field wants to install and set up by themselves. However, the native weight file is as high as 1.6TB. Officials recommend using a supernode configuration composed of more than 64 accelerators for inference. The hardware threshold is easily tens of millions of Taiwan dollars (We also have relevant analysis), leaving the vast majority of “ordinary” developers and companies intimidated. However, decentralized infrastructure startup Pipe Network and the well-known open source quantitative team Unsloth AI Both have released their own ported quantified versions, using self-developed technology to compress the model from 1.6TB to 350GB~594GB, so that a Mac Studio M3 Ultra equipped with 512GB of unified memory can theoretically load and execute this giant model.
Kimi K3: The technical details of the largest open source model in history
Dark Side of the Moon’s most powerful Kimi K3 so far, the parameter scale is about 3 times that of the previous generation K2.5. The model adopts MoE architecture and has 896 routed experts internally. Only 16 of them plus 2 shared experts are activated each time a token is processed. The actual operating parameters are about 100 billion. This design allows the model to maintain a huge knowledge capacity while controlling the computational complexity of a single inference within a reasonable range.
In terms of benchmarks, Kimi K3 scored 57.11 points on the Artificial Analysis Intelligence Index, ranking fourth, behind Claude Fable 5 (59.86) and GPT-5.6 Sol (about 59), but surpassing models such as Claude Opus 4.8. In the Frontend Code Arena, K3 even defeated Claude Fable 5, demonstrating its strong code generation capabilities.
Pipe Network’s approach: Cutting from 896 experts to 179, compressing 1.5 TB to 350 GB
Pipe Network was founded by David Rhodus and was originally a decentralized CDN infrastructure company. This Kimi K3 MLX transplant is a landmark work of the team’s shift from CDN to AI infrastructure. Their entire transplantation includes three core technical links. The first is the streaming converter: Since K3’s BF16 full-precision representation occupies approximately 5.6TB, far exceeding any single-machine memory, the converter processes the model layer by layer and does not need to load the complete model into memory at once. The second is REAP expert pruning: K3’s 896 routing experts are scored, the experts most relevant to the target workload are retained based on the calibration corpus, and the rest are deleted. The third is MXFP4 encoding preservation: K3’s routing experts have adopted Dark Side of the Moon’s MXFP4 encoding, and MLX can directly preserve this encoding format without the need for another lossy quantization.
Pipe Network uploaded three pruned versions to Hugging Face. The smallest version retains 179 routing experts (80% pruning rate), is about 350GB compressed, and can be loaded into Mac Studio with 512GB of memory. The other two 451GB versions retain 242 experts each, one of which is calibrated and optimized for Chinese and coding scenarios.
Speed is the biggest bottleneck: one token every 5 seconds
However, being able to load has nothing to do with being able to run smoothly. The test data published by Pipe Network on Hugging Face shows that the 350GB version takes about 66 seconds to load on the 512GB M3 Ultra Mac Studio, and the generation speed is about 0.16 tokens per second. The 451GB version is slower, about 0.14 tokens per second. Pipe Network itself describes the three pruned versions in the GitHub document as “non-interactive,” meaning that this speed is completely unavailable for real-time conversations. Test data published by Pipe Network itself shows that the fastest version generates only about 0.20 tokens per second, which is equivalent to producing one word every 5 seconds. Coupled with the quality degradation caused by pruning, this transplant is still a long way from being a practical tool. It is a technical demonstration at most.
In contrast, the unpruned K3 model can achieve an inference speed of dozens of tokens per second on a 64-piece H100 GPU cluster, and the previous generation K2 series can also achieve a generation speed of 23 tokens/sec when executed through MLX on Mac Studio. 0.20 tokens/sec means that it takes nearly 17 minutes to generate a 200-word response, which is unacceptable in any practical application scenario.
Not only is it super slow, but the quality of the responses is also poor: cutting off 80% of the cost of experts
The quality degradation caused by REAP pruning cannot be ignored. The Pipe Network document admits that the pruned model will have problems such as repetitive list output, responses deviating from the original prompts, and overall language ability decline. Although code completion remained structurally coherent, performance on the general language task was significantly impaired.
The reason is that K3’s 896 routing experts are each responsible for different fields of knowledge, and only 16 are started for each inference, but the combination of routing experts is dynamically selected based on input. When 80% of the experts are removed, the model’s knowledge in many fields is directly missing, and the routing mechanism also loses most of its choice space. This is like cutting down a hospital with 896 specialists to only 179. Although the core departments are still there, they are helpless when it comes to rare diseases.
Additionally, even if Mac Studio can accommodate model weights, the KV cache requirements in long-context scenarios will further compress available memory. one articleAnalytical articles pointed out, a prompt word of 2,000 tokens may work, but when it reaches 32,000 tokens, it may fail due to insufficient memory, and K3 supports a maximum contextual window of 1 million tokens.
Unsloth AI: 1-bit extreme quantization, retaining approximately 78.9% of original capabilities
Another route comes from the well-known open source quant team Unsloth AI. They used self-developed Dynamic quantization technology to compress K3 from 1.56TB to 594GB (a 62% reduction), but retained about 78.9% of the original top-1 accuracy with a perplexity of 2.58 (the original model was 1.46). Different from Pipe Network’s pruning scheme, Unsloth’s approach retains all 896 routing experts, but reduces the weight accuracy of each expert to 1-bit. The completeness of the model knowledge is much higher than that of the pruned version, but the price is that the compression ratio is not as extreme as the pruned version. The problem is that a 594GB file still requires more than 512GB of memory to load, so a single 512GB M3 Ultra cannot be loaded. It requires at least “Mac Studio + 128GB RAM device”, that is, more than two Macs running EXO.
Kimi K3 can now be run locally! ✨
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio + 128GB RAM device.
Kimi K3 is the strongest open model to date.
Guide: https://t.co/1mVwOMLpDW
GGUF: https://t.co/bt1c1ADdCZ https://t.co/yFJ5TTeNz5 pic.twitter.com/rdvVlDIitX— Unsloth AI (@UnslothAI) July 29, 2026
Unsloth also released GGUF versions with multiple precisions: the Dynamic 2-bit XXS version retains 81.2% accuracy, the 2-bit XL version reaches 84.1% (about 861GB), and the near-lossless Q8 version reaches 90.4% (1.56TB). The measured results of Unsloth are quite usable (mainly because the K3 body is so good). In addition, Unsloth The quantization quality is also far better than other community versions. It is also the community IQ1_M quantification on Hugging Face. The file size of 618.9GB is larger than Unsloth’s 594GB, but the perplexity has increased to 54.56, with a quality gap of 21 times.
Unsloth 1-bit quantized version Kimi K3 official website
On the X platform, netizensExpressed after test The 1-bit version “can still complete difficult creative tasks in one go, and the tool calls are quite reliable.” Someone also tested the 1-bit version on four NVIDIA B200 GPUs and achieved a generation speed of 36 tokens per second, which is far better than Pipe Network’s 0.20 tokens/sec on Mac Studio.
Currently, these local-end Kimi K3-based solutions each have obvious trade-offs: Pipe Network’s REAP pruning compresses the model to 350GB, which can be stuffed into a 512GB Mac Studio, but sacrifices 80% of the experts, resulting in significant quality degradation and a speed of only 0.20 tokens/sec (should be the least recommended); Unsloth’s 1-bit quantization retains all experts and higher accuracy, but 594GB Files require more memory and require multiple units to handle (although expensive, but the cost is the lowest); with the same 1-bit quantization on a GPU cluster (such as 4 B200s), the Unsloth version can reach a practical speed of 36 tokens/sec, but that returns to the hardware requirements at the data center level. Regardless of the solution, running K3 on consumer-grade hardware is still in the stage of “technical exploration” rather than “productivity tool”. At least for ordinary people, the threshold is still too high to be realistic.
The reality of hardware threshold: GPU deployment costs nearly 20 million
Return to the deployment requirements of native K3. The official technical documentation of Dark Side of the Moon recommends using 64 or more accelerators, and the native MXFP4 weight is about 1.4TB to 1.5TB. Calculated based on H100 (80GB), approximately 19 images are needed simply to place weights, and additional KV cache space is required for actual execution. The hardware purchase cost for 64 H100s exceeds US$2.4 million (approximately NT$78 million). With the addition of servers, network and power infrastructure, the total cost easily exceeds tens of millions of US dollars.
The weight is released, but who can run?
The open source of Kimi K3 triggered a fundamental discussion: What is the meaning of “open source” when a model is so large that only data center-level hardware can execute it? The Q4 quantized version of DeepSeek R1 671B is about 350GB to 400GB, and a 512GB Mac Studio can barely run it; the native weight of K3 is 3 to 4 times that of DeepSeek R1, directly pushing the threshold back to the data center level.
However, the value of open source lies not only in local execution. In addition to using K3 through the official API of Dark Side of the Moon, developers can also rent GPUs and run quantified models on specific hardware. This is at least “optional” for small and medium-sized enterprises or developers with strong financial resources who cannot access the Internet due to privacy and security considerations. As for the average person, it is more practical to just look for a cheap API or Coding Plan, because it is too difficult to get back the money.
Source: KOCPC Chinese