Source: MarkTechPost — 2026-07-14
Summary
PrismML released Bonsai 27B, extreme-quantization builds of Qwen3.6-27B that shrink a 27-billion-parameter multimodal model down to 5.9GB (ternary, {-1,0,+1} weights at 1.71 bits/weight) or 3.9GB (1-bit, {-1,+1} weights at 1.125 bits/weight). The ternary build retains 94.6% of FP16 baseline performance and the 1-bit build retains 89.5% — PrismML claims the 1-bit variant is the first 27B-class model that actually fits on a phone. Both ship Apache 2.0 and run via llama.cpp (CUDA/Metal) or MLX.
Key Takeaways
- Two quantization tiers: ternary weights ({-1, 0, +1}) at 1.71 bits/weight producing a 5.9GB file, and true 1-bit binary weights ({-1, +1}) at 1.125 bits/weight producing a 3.9GB file — both derived from the same Qwen3.6-27B base.
- Performance retention is the headline number: 94.6% of FP16 baseline for the ternary build, 89.5% for the 1-bit build — meaning most of a 27B model's capability survives a roughly 8-9x size reduction.
- The model is multimodal: ~24.8B language weights, a 0.46B vision tower (kept at 4-bit, not further quantized), and 2.5B in embeddings/LM head — so the vision component is deliberately spared the extreme compression applied to the language backbone.
- Apache 2.0 licensing plus llama.cpp and MLX support means this is immediately runnable on consumer hardware, not a research artifact behind an API.
Reel Script
Hook: A 27-billion-parameter model that used to need a data-center GPU now fits in 3.9 gigabytes — small enough for a phone — and still keeps almost 90% of its original capability.
Core Concept: Quantization means storing a model's weights with fewer bits per number, trading some precision for a much smaller file. Most quantization you've heard of goes from 16-bit down to 4-bit. Bonsai 27B goes much further: a "ternary" build where every weight is just one of three values (-1, 0, or +1), and a "1-bit" build where every weight is just one of two values (-1 or +1) — think of it like rounding every measurement in a recipe down to "a little," "none," or "a lot" instead of exact grams, and finding the dish still tastes almost the same.
Hands-On: The numbers are the whole story here. The ternary build sits at 1.71 bits per weight, comes out to 5.9GB, and keeps 94.6% of the full-precision (FP16) model's performance. The 1-bit build goes further — 1.125 bits per weight, 3.9GB, 89.5% retention — and PrismML says that's the first 27B-class model small enough to actually run on a phone. What's notable is what they didn't compress: the model's vision tower, a 0.46-billion-parameter component, stays at 4-bit while the 24.8B-parameter language backbone gets crushed down to near-binary — a sign the team identified which parts of the model tolerate extreme compression and which don't, rather than quantizing everything uniformly. It ships Apache 2.0 and runs today via llama.cpp on CUDA or Metal, or through MLX on Apple silicon.
Takeaway: This is a genuine engineering result, not a marketing number — a real, licensed, runnable model that keeps ~90-95% of a 27B model's ability at under a tenth of the size. If you're evaluating on-device or edge deployment for a multimodal model, Bonsai 27B is worth benchmarking against your own workload before assuming you need cloud inference.