Traditionally, if you want an AI model to be smarter, you must make it larger, which means more parameters, more GPUs, more power, more memory, and higher costs. This path is indeed working, and it has given rise to many large models capable of long-context reasoning, solving complex problems, and generating software, research reports, and creative content. However, this also creates a deep structural limitation: the most powerful intelligence is trapped in huge data center clusters and professional infrastructure, and edge devices such as mobile phones, laptops, vehicles, robots… have almost no opportunity to use better local models. Recently, a new AI startup company incubated from the California Institute of Technology (Caltech) “PrismML“Officially released a disruptive product:1-bit Bonsai 8B Model. This model condenses the advanced LLM model into a very small size of only 1.15 GB. It can run directly on the iPhone 17 Pro at a conversation speed of about 40 tokens per second. It is called “the world’s first commercially viable 1-bit LLM.”
What is a 1-bit model?
To understand the breakthrough of Bonsai 8B, we must first understand the essential difference between traditional models and 1-bit models.
AI models based on the Transformer architecture contain millions or even billions of “weights”, which control the strength of the connections between neurons and determine how the model performs tasks during the training process. The traditional model represents each weight as a 16-bit or 32-bit floating point number: this means huge storage space and computing power requirements.
Bonsai 8B adopts a revolutionary 1-bit architecture: each weight is represented by only two values {-1, +1}, and the scale factor shared by the group is used to maintain model performance. The Embedding layer, attention layer, MLP layer and LM head all use 1-bit design, without any higher-precision “escape hatch”. This is truly a 1-bit model from start to finish, spanning 8.2 billion parameters.
This concept did not come out of nowhere. Research on 1-bit networks can be traced back to the 2017 BitNet paper, and the 2024 “The Era of 1-bit LLMs” paper (also known as BitNet b1.58) established the design direction of 1.58 bits. The breakthrough of PrismML is that a team led by Babak Hassibi, a professor in the Department of Electrical Engineering at Caltech, spent several years developing the necessary mathematical theory to finally compress the neural network without losing its reasoning ability.
Intelligence Density: A Key Metric to Re-Measure AI Efficiency
PrismML proposes a new measurement standard: “Intelligence Density”. Traditionally, we are used to using benchmark scores to compare model capabilities, but this ignores the efficiency difference caused by model size. PrismML defines wisdom density as the negative logarithm of the model’s average error rate, divided by the model size in GB. Using this indicator to measure, the smart density of Bonsai 8B reaches 1.06/GB, while the Qwen3 8B with the same parameter level only has 0.10/GB. Bonsai isn’t just ahead in this metric, it’s into an entirely different category.
According to the test data published by PrismML, the actual performance of Bonsai 8B is equally impressive:
- The size is only 1.15 GB, about 14 times smaller than models with the same performance
- Inference speed increased by about 8 times
- Energy efficiency improved by about 5 times on edge hardware
- Runs at about 40 tokens per second on iPhone 17 Pro (comparison: 16-bit 8B model won’t fit on any iPhone)
There is also a significant difference in energy consumption. The energy consumption of 1-bit Bonsai 8B is much lower than its 16-bit full-precision version, and the energy efficiency is approximately 4-5 times higher. On the M4 Pro, it consumes 0.074 mWh/s, while on the iPhone 17 Pro Max, it consumes just 0.068 mWh/s.
The official test example, their 1-bit Bonsai 8B model running on the iPhone 17 Pro is about 40 toks/second, the standard 16-bit 8B model won’t work on any iPhone. For comparison, a 16-bit 1B model running at the same MATH-500 prompt at 23 toks/second is also shown, highlighting the huge difference in accuracy and speed:
Bonsai also saw significant improvements in throughput across devices. On the M4 Pro Mac, the throughput is up to 131 Tokens per second; on the RTX 4090, it is up to 368 Tokens per second; on the iPhone 17 Pro Max, the throughput is about 44 Tokens per second. Here is the performance of 1-bit Bonsai 8B running on the M4 Pro Mac versus the standard 16-bit 8B model:
Agent capability test, execute 1-bit Bonsai 8B on M4 Pro Mac, and execute a standard 16-bit 8B model at the same time to simulate long-term agent tasks running locally:
The smart density score of Bonsai 8B is 10.6 times that of Qwen3 8B; if you only look at the original benchmark average score, the improvement can reach 12.7 times. PrismML explains that the smart density indicator places more emphasis on improvements in high-precision intervals because the closer to perfect performance, the more difficult each point of improvement is.
Break the black curse of “compression will reduce energy”
Model compression techniques have always been associated with painful trade-offs in the past. Low-bit models often lose too much ability in instruction following, multi-step reasoning and reliable tool use, and cannot be used as the basis of the main model. But Bonsai turned that around. PrismML claims that 1-bit models don’t have to be victims of narrow-mindedness: they can be capable, production-deployable systems. In standard benchmark tests, Bonsai 8B is comparable to 16-bit models of the same parameter level, but achieves all this with extremely small size and power consumption.
“We regard 1 bit as the end point, but also as the starting point.” Babak Hassibi, CEO and founder of PrismML, said: “We believe that major breakthroughs in AI in the future will be driven by orders of magnitude improvements in intelligence density, rather than solely relying on the expansion of parameter counts.”
The era of edge AI is officially here
When advanced models become small enough, fast enough, and energy efficient enough to run locally, the design space for AI will immediately change.
PrismML describes the application blueprint that Bonsai enables:
Product response is faster:Intelligence can run on the device, significantly reducing latency.
The system pays more attention to privacy:Sensitive data never needs to leave the device or cross organizational boundaries.
Application more reliable:No more reliance on constant cloud connectivity.
AI finally works in remote areas:Scenarios where server-side deployment costs were too high in the past are now affordable.
Entire categories of applications will emerge: persistent on-device agents, real-time robotics, secure enterprise copilot, offline intelligence, and AI-native products that have historically been unable to use advanced models due to bandwidth, power, or compliance constraints. The future of AI won’t be limited to the cloud. It will span the cloud, the edge, and everything in between.
PrismML was founded to solve a fundamental imbalance in the development of AI: the most powerful intelligence is trapped in the most expensive infrastructure. CEO Hassibi noted that there is a profound misalignment between where AI is being deployed and where it really needs to happen.
Bonsai models are currently available at Hugging Face Upload and download, supporting GGUF format and MLX format, developers can deploy it to various edge devices and local environments.
Conclusion
From the high-efficiency model trend set off by DeepSeek to the current 1-bit LLM of PrismML, the AI industry is undergoing a paradigm shift. The linear growth logic of “bigger is better” in the past is being challenged, replaced by the pursuit of “intelligent density”: providing more powerful AI capabilities in a smaller size and lower power consumption. The emergence of Bonsai 8B proves that this path is not only feasible, but also ready for commercialization. The next step for AI will no longer be just a cloud arms race, but a smart network that extends to every mobile phone, every car, every factory and every office. The gunshots of this revolution have already sounded.
Source: KOCPC Chinese