Bonsai 27B Just Changed What's Possible on Your Phone — Here's Why Everyone's Talking

A 27-billion-parameter AI model that fits in under 4 GB and runs on an iPhone. Not a research demo. Not a proof of concept. PrismML released Bonsai 27B on July 14, and the internet hasn't stopped talking about it since.

The Caltech spinout dropped two variants of an extreme compression of Alibaba's Qwen3.6 27B — a 1-bit binary build at just 3.9 GB and a ternary version at 5.9 GB — both under the Apache 2.0 open-source license. And within hours, reports emerged that Apple is actively evaluating the technology for its on-device AI roadmap.

Bonsai 27B: The First 27B-Class Model to Run on a Phone

Pasadena, California — PrismML CEO Babak Hassibi confirmed to CNBC that Apple and other companies have been testing the startup's models, measuring speed, energy efficiency, and device-level performance. "They're really evaluating our technology right now," Hassibi said, describing the discussions as early-stage but noting that "things are progressing nicely." Apple declined to comment.

How 27 Billion Parameters Fit Into 4 GB

Here's the scale of what PrismML pulled off. Standard large language models store each weight as a 16-bit floating-point number. Alibaba's Qwen3.6 27B — the open-source base model — sits at roughly 54 GB at that precision. That's not just too big for a phone; it's too big for most laptops.

Even a good 4-bit quantization build of the same model still needs about 18 GB. Better, but still not phone territory.

PrismML took a more radical approach. Bonsai 27B reduces each weight to a single bit in its binary variant — the value is either +1 or -1, nothing else — or to one of three values ({-1, 0, +1}) in its ternary variant, which works out to about 1.58 bits per weight. This compression is applied end-to-end: embeddings, attention layers, MLP blocks, the language model head — everything uses the low-bit format. There are no higher-precision escape hatches left in the architecture.

To keep the model accurate despite such extreme compression, PrismML uses FP16 group-wise scaling. Each cluster of weights retains its own full-precision 16-bit scaling factor, anchoring the model's outputs even as individual weight values collapse to discrete steps. The 1-bit binary model comes in at 3.9 GB; the ternary variant is 5.9 GB.

The Numbers That Matter

PrismML evaluated Bonsai 27B across 15 benchmarks in thinking mode using EvalScope with vLLM on H100 GPUs. The ternary variant retains 94.6% of the full-precision Qwen3.6 27B baseline. The 1-bit variant retains 89.5%.

But the per-category breakdown tells a sharper story than the averages. Math holds up almost untouched: GSM8K, MATH-500, AIME25, and AIME26 collectively score 93.4 on the ternary build and 91.7 on 1-bit, compared to the baseline's 95.3. Coding scores 86.0 on ternary and 81.9 on 1-bit versus 88.7 at full precision. Tool-calling and agentic tasks — BFCL v3, TauBench — land at 74.0 on ternary and 66.0 on 1-bit versus 80.0 at full precision.

On an iPhone 17 Pro, the 1-bit model runs at 11 tokens per second. On an M5 Max, it reaches 87 tokens per second. On a desktop RTX 5090, the 1-bit variant hits 163 tok/s, and the ternary build delivers 134 tok/s.

By PrismML's own intelligence-density metric — capability per gigabyte of deployment footprint — the 1-bit Bonsai 27B delivers 0.53 per GB. That is more than 10 times the full-precision baseline and roughly 2.7 times the best conventional low-bit alternative at the same parameter count.

The Apple Connection Everyone's Talking About

The timing here is precise. Apple opened the first public beta of iOS 27 on July 13 — just one day before PrismML dropped Bonsai 27B. The new iOS revision is built around Siri AI, a rebuilt personal assistant that depends on a hybrid of on-device and cloud infrastructure. Whether PrismML's compression technique could reduce the cloud portion of that equation is exactly what Apple's evaluation is probing.

This is the same Apple that announced AFM 3 Core Advanced at WWDC 2026 on June 8 — a 20-billion-parameter model that uses Instruction-Following Pruning. That architecture stores the full model in flash and loads only 1-4 billion parameters into active DRAM per request. Two different technical bets on the same problem: how do you get real AI capability on a device with limited memory and power?

Sparse Activation vs. Dense Compression: Two Bets, One Goal

The technical distinction between PrismML's approach and Apple's current on-device architecture reveals how differently engineers are solving the same problem.

Apple's AFM 3 uses sparse activation: keep all 20B parameters in flash, load what you need for each prompt. The bandwidth between NAND and DRAM is too slow to swap parameters token-by-token, so the system commits to a specific parameter subset per prompt rather than per token.

PrismML's approach is the opposite: compress everything so aggressively that the entire model fits in memory at once. No routing decisions, no flash swapping, no parameter selection — just a small, dense, fully-loaded model matching what a bigger model can do, in a tighter footprint.

Both approaches carry tradeoffs. Sparse activation keeps more total parameters available but introduces routing complexity and flash access latency. Dense compression eliminates the routing overhead but accepts the fidelity loss from extreme quantization. Which approach wins depends on whose benchmarks you trust and what use case you are optimizing for.

The Open Source Factor

Both Bonsai 27B variants ship under the Apache 2.0 license — free to download, modify, and deploy. The models are available on Hugging Face, compatible with Apple's MLX framework and Nvidia's CUDA, with GGUF builds for llama.cpp. PrismML also released a demo application on GitHub that downloads models from HuggingFace and runs them locally.

This licensing choice matters strategically. Apache 2.0 means developers can integrate Bonsai 27B into commercial products without royalty concerns. It means the community can fine-tune, quantize further, and build tooling around it. And it means PrismML's compression technique is now available for anyone to study and improve upon — an approach that accelerates the entire field compared to proprietary alternatives.

The open-source release also puts pressure on Apple and other hardware vendors. If a third-party model can run competitively on their devices, the argument for tightly integrated proprietary AI stacks weakens. Developers may choose to build around Bonsai 27B instead of waiting for Apple's native solutions, creating an ecosystem that operates independently of any single platform vendor's roadmap.

The Tradeoffs Nobody's Discussing

Hassibi disclosed a tradeoff that is easy to miss in all the excitement: factual recall degrades before higher-order skills like reasoning, mathematics, and coding. The 1-bit model MMLU-Redux score drops to 73.4 from the baseline's 83.1 — a noticeable gap on knowledge-intensive tasks. Agentic task performance — tool use, multi-step planning — holds up better, but if you need your model to reliably retrieve specific facts, the 1-bit build may disappoint.

Independent third-party verification of PrismML's benchmarks has not yet been completed. The company's numbers are impressive, but they are also self-reported. How the model holds up under independent testing will be the real story in the weeks ahead.

There is also the question of energy efficiency on device. Running at 11 tok/s on an iPhone 17 Pro sounds promising, but sustained agentic workloads will draw power continuously. For phone use cases, that battery drain adds up fast.

What This Means

Bonsai 27B crosses a practical threshold. It is not the first model to run locally, and it will not be the last. But it is the first 27B-class model bringing real reasoning, tool-calling, and agentic capability to the hardware people already carry.

The implications go beyond Apple. If a 27B model can run on a phone, the entire economics of AI deployment shifts. On-device agents with zero latency, zero cloud costs, and zero privacy exposure become feasible. Hybrid architectures that use local models for non-frontier work and cloud APIs only for the hardest problems could slash operating costs for AI-native products.

PrismML's CEO makes the point directly: the most valuable AI workloads are shifting from single responses to sustained work — assistants that operate real tools, workflows that run unattended, research that synthesizes dozens of documents. Those workloads make hundreds of model calls per task. When each call runs locally, the marginal cost drops to zero and the user's data never leaves the machine.

The Shanghai World AI Conference kicks off July 17, and major announcements are expected. Bonsai 27B has set the tone for that conversation: intelligence density is the next frontier, and the question is no longer whether capable models can run on devices, but how fast the ecosystem will build for them.

— Nova Chen, Global 1 News