PrismML released Bonsai 27B earlier this week, a 27-billion-parameter AI model compressed to 3.9 GB that runs on an iPhone 17 Pro Max at 11 tokens per second. The ternary variant retains 94.6% of full-precision benchmark performance while fitting within smartphone memory constraints, marking the first time a model at this capability tier has cleared a consumer device's memory budget. The compression method, built on Caltech intellectual property, reduces each model weight from 16 bits to a single sign value, landing the binary variant at 1.125 bits per weight—14 times smaller than the full-precision original.
Compression Technology Reduces Model Weights to Sign Values
The compression method reduces each model weight from 16 bits of floating-point precision to a single sign—+1 or -1 in the binary build, one of three values in the ternary variant. Each group of 128 weights shares a 16-bit scaling factor. The ternary model adds a zero state for slightly more expressive power and settles at 1.71 bits per weight. The ternary variant, at 5.9 GB, hits around 26 tokens per second on an M5 Pro laptop.
Nothing gets a higher-precision escape hatch: embeddings, attention, and the full language model head are all compressed end-to-end. Most quantized builds keep certain sensitive layers at full precision, which increases their size as a tradeoff for better quality. The model uses a hybrid attention backbone where roughly 75% of the layers are linear rather than full quadratic attention, making a 262K-token context window practical on-device.
In March, PrismML shipped Bonsai 8B, a 1.15 GB model that proved the 1-bit architecture could survive at 8 billion parameters. Both models are free under Apache 2.0 license.
Benchmark Performance Averages 80.49 Across 15 Tests
Across 15 benchmarks evaluated in thinking mode on NVIDIA H100 GPUs—spanning knowledge, math, coding, and tool use—Ternary Bonsai 27B averages 80.49, or 94.6% of the full-precision model. The 1-bit variant hits 76.11. AIME25 and AIME26, modeled on the American Invitational Mathematics Examination, come in at 93.7% for Ternary Bonsai 27B versus 95.3% for Qwen 3.6B. Bonsai scores 86 points in coding versus 88 for Qwen 3.6 and 77% on general knowledge versus 83 for Qwen 3.6.
PrismML ships a DSpark speculative decoding layer alongside the model—a lightweight drafter that proposes blocks of candidate tokens, which the main model verifies in a single forward pass rather than generating token-by-token. On an H100 that adds a 1.37x throughput boost with no change in output quality. On Apple Silicon it's not yet enabled by default.
Real-World Testing Produces Functional Code and Creative Writing
The source tested Bonsai 27B on a Zombie Type game—a first-person typing-horror browser game. Two coding rounds produced clean collision detection, proper scoring logic, and graphics that held together. The model grasps structure early; the second pass refines rather than rebuilds. Some models looked more elaborate than the ones from GPT 5.6 Sol.
For creative writing, Bonsai produces stories with consistent internal logic, pacing, and arc—on par with Claude Haiku or even Sonnet on lower effort on comparable prompts. The results aren't particularly imaginative with zero-shot prompts.
Apple Evaluates Technology for On-Device Use
Apple is in early talks with PrismML about the underlying compression technology, per CNBC. PrismML CEO Babak Hassibi confirmed to CNBC that the company is in early talks with Apple, which is evaluating the compression technology for potential on-device use. Hassibi said a compressed Gemma model is next in the pipeline, followed by larger frontier models.
FAQ
What is PrismML's Bonsai 27B model?
Bonsai 27B is a 27-billion-parameter AI model compressed to 3.9 GB that runs on an iPhone 17 Pro Max at 11 tokens per second. The ternary variant retains 94.6% of full-precision benchmark performance using compression technology built on Caltech intellectual property that reduces model weights to sign values.
How does Bonsai 27B perform on benchmarks?
Across 15 benchmarks evaluated on NVIDIA H100 GPUs, Ternary Bonsai 27B averages 80.49, or 94.6% of the full-precision model. On AIME mathematics tests it scores 93.7%, on coding tasks it scores 86 points, and on general knowledge it scores 77%—all while requiring significantly less memory than comparable models.
