--- title: "Bonsai 27B Review: 1-Bit LLM on a Phone" canonical: https://wavect.io/blog/bonsai-27b-phone-local-ai-review/ language: en description: "Can Bonsai 27B really run on a phone? Verified size, memory, benchmarks, WebGPU speed, limitations, variants and production pilot checklist." image: "https://wavect.io/img/general/bak/open_graph_preview.jpg" --- [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 13 min read · 16 Jul 2026 Last reviewed August 11, 2026 [**Next**](/blog/soofi-s-european-sovereign-llm/) # Bonsai 27B Review: Can a 27B LLM Really Run on a Phone? TL;DR Bonsai 27B is PrismML's low-bit model family derived from Qwen3.6-27B. Its binary language weights occupy about 3.9 GB, and PrismML reports roughly 11 tokens per second on an iPhone 17 Pro Max. The 1-bit build scores 89.5% of the Qwen3.6 FP16 average in PrismML's suite, but an independent 98-question community benchmark scored it at 82.9% versus 94.2% for Qwen3.6-27B. Binary Q1_0 works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. For ternary, the 7.17 GB group-128 Q2_0 file still needs PrismML's fork, while the 7.59 GB group-64 file now works across those mainline backends. Bonsai remains a strong pilot candidate, but production requires pinned artifacts, full peak-memory measurement, task-specific evals, sustained latency tests, and cost per successful outcome. **Yes, a 27B-class model can now run on a phone, but the useful answer needs more context than the headline.** PrismML reports that its 1-bit Bonsai 27B language weights occupy about 3.9 GB and generate roughly 11 tokens per second on an iPhone 17 Pro Max. The model is derived from Qwen3.6-27B and published under Apache 2.0. It is a real on-device milestone, not proof that every 27B workload now fits comfortably inside every phone. The same evidence also shows where the compression hurts. Across PrismML's own 15-benchmark suite, the 1-bit variant scores 89.5% of the full-precision average. Math and coding retain more of their baseline than instruction following, vision, and multi-step tool use. The quality-first ternary variant retains 94.6%. Its group-128 GGUF is about 7.17 GB, while the mainline-compatible group-64 file is about 7.59 GB. Both are larger than the 5.9 GB ideal representation often repeated in launch posts. | Question | Verified answer | What it means for a buyer | | --- | --- | --- | | Does it run on a phone? | PrismML reports about 11 tok/s on iPhone 17 Pro Max via MLX Swift | Interactive local text generation is possible on a current high-end phone | | Is it really 3.9 GB? | Yes, for the resident 1-bit language model weights | Runtime buffers, context cache, and optional vision components need additional memory | | Does it retain 90%? | 89.5% of the FP16 average in PrismML's 15-benchmark evaluation | Use-case quality still needs an independent eval because losses are uneven | | Is ternary 5.9 GB? | 5.9 GB ideal; 7.17 GB for the group-128 fork file or 7.59 GB for the group-64 mainline file | Plan capacity from the exact artifact, not the information-theoretic target | | Is it production-ready? | Promising, with custom low-bit runtimes and explicit model-card limitations | Pilot first, especially for agents, vision, and long context | ## What is Bonsai 27B? [Bonsai 27B](https://prismml.com/news/bonsai-27b) is PrismML's low-bit version of [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B), a dense multimodal model with a 27B language backbone, a vision encoder, and a native 262,144-token context window. Qwen's architecture uses linear attention in most blocks and full attention in 16 of 64 blocks. That hybrid design limits KV-cache growth compared with a conventional all-attention model. PrismML publishes two operating points. The [1-bit build](https://huggingface.co/prism-ml/Bonsai-27B-gguf) uses binary weights from {−1, +1} with one FP16 scale for each group of 128 weights. That produces 1.125 effective bits per weight and about 3.9 GB of resident language weights. The [ternary build](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf) adds zero, using {−1, 0, +1}. Its representation carries 1.71 effective bits per weight and retains more quality. This is deeper than taking a normal FP16 checkpoint and exporting a generic 1-bit file. PrismML says low-bit weights extend across embeddings, attention projections, MLP projections, and the language-model head. Custom CUDA, Metal, MLX, and WebGPU kernels consume the packed weights directly instead of expanding them back to FP16 for every operation. Deployment support has also matured since launch. [PrismML's current compatibility matrix](https://github.com/PrismML-Eng/Bonsai-demo) says binary `Q1_0` works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. Ternary needs an artifact-level distinction: the 7.17 GB group-128 `*-Q2_0.gguf` still requires PrismML's fork, while the 7.59 GB group-64 `*-Q2_0_g64.gguf` now runs in mainline on CPU, Metal, CUDA, and Vulkan. The experimental `PQ2_0` files are not yet a stable deployment target. Buyers should pin the model file and runtime commit together. ## How does a 27B model shrink from 54 GB to 3.9 GB? The rough FP16 calculation is simple: 27.3 billion language parameters multiplied by two bytes is about 54 GB before runtime overhead. The binary model stores one sign bit per weight plus a 16-bit scale shared across every 128 weights. That works out to 1.125 bits per weight, roughly 14.2 times smaller than FP16. The headline footprint is not the entire application footprint. PrismML's model card separates four memory layers: - **Resident language weights:** about 3.9 GB for 1-bit, 7.17 GB for the ternary group-128 fork file, or 7.59 GB for the group-64 mainline file. - **Runtime and activations:** the published measurements include roughly 1.3 GB across backends. - **KV cache:** grows with the active context. With 4-bit KV compression, the full 262K window still brings the 1-bit build to about 9.4 GB peak memory. - **Optional components:** the compact vision projection is about 0.63 GB. The speculative-decoding drafter is another optional artifact and is not free memory. That distinction matters commercially. A model file can fit inside a phone's app budget while its maximum advertised context cannot. Capacity planning must use weights plus context plus runtime, measured on the exact backend and device. ## Does Bonsai 27B really retain 90% of Qwen3.6 intelligence? **It retains 89.5% of one vendor-reported benchmark average, not 90% of every capability.** PrismML evaluated all variants in thinking mode on the same H100 setup with EvalScope and vLLM. The aggregate is useful because the conditions are matched, but it has not yet been independently reproduced and it hides large category differences. | Capability | Qwen3.6 FP16 | Ternary Bonsai | 1-bit Bonsai | Commercial reading | | --- | --- | --- | --- | --- | | Math | 95.33 | 93.40 | 91.66 | Strong retention in the published suite | | Coding | 88.74 | 85.96 | 81.88 | Promising for bounded code work, not proof of repository-level autonomy | | Knowledge and reasoning | 83.15 | 76.96 | 73.39 | Noticeable quality trade-off | | Instruction following | 78.47 | 71.77 | 65.74 | Test formatting, constraints, and refusal behavior carefully | | Agentic tool use | 80.00 | 74.01 | 66.03 | Multi-step reliability is a material risk | | Vision | 72.61 | 65.19 | 59.57 | Do not transfer the text footprint story to multimodal quality | The per-benchmark results make the risk clearer. On the 1-bit model, MATH-500 moves from 99.40 to 98.00 and BFCL v3 from 77.10 to 70.72. But the harder multi-step τ²-Bench falls from 82.90 to 61.34, IFBench from 68.03 to 52.36, and MMMU-Pro from 79.94 to 60.48. Compression loss lands precisely where a long-running agent can compound small mistakes. A small independent community check points in the same direction. [ArmanJR's reproducible 98-question benchmark](https://github.com/ArmanJR/PrismML-Bonsai-vs-Qwen3.5-Benchmark) on a Jetson Orin scored the 1-bit model at 82.9% and the ternary model at 86.2%, versus 94.2% for Qwen3.6-27B. The sample is too small and heterogeneous to replace PrismML's suite, but it is useful outside-vendor evidence that footprint retention is not the same as task-level parity. PrismML's own model card says long-horizon, multi-file agentic coding is not yet a strong target for this release. That caveat should carry more weight in a procurement decision than the rounded 90% headline. ## Can Bonsai 27B really run locally on a phone and laptop? **Yes, on supported high-end hardware and with the right runtime.** PrismML reports about 11 tok/s for the MLX Swift build on an iPhone 17 Pro Max. For the 1-bit GGUF build, its llama-bench results report 26.0 tok/s on an M4 Pro, 44.2 on an M5 Pro, and 66.4 on an M5 Max for 128-token generation. The ternary GGUF reports 18.0, 26.2, and 44.0 tok/s on the same laptop classes. These figures are not directly interchangeable. Phone, Metal, CUDA, and WebGPU tests use different kernels, context lengths, thermal envelopes, and measurement protocols. Prompt processing and token generation are also different phases. A fast decode rate does not guarantee a short time to first token for a long document. ## What does the WebGPU demo actually prove? [Joshua Lochner's Hugging Face Space](https://huggingface.co/spaces/webml-community/bonsai-webgpu-kernels) confirms the important part: Bonsai 27B can execute locally in a browser through custom WGSL compute shaders. The page says Fable 5 and GPT 5.6 Sol authored and optimized the kernels before correctness tests and shape-specific benchmarks. The model and conversation stay on the device after the weights are fetched. The public demo does not publish a hardware-specific throughput result or a repeatable benchmark protocol. Treat browser speed claims as demonstrations, not capacity-planning constants. For a buying decision, record the browser, GPU, power mode, prompt length, output length, first-token latency, sustained decode rate, memory peak, and thermal throttling over repeated runs. ## Where can Bonsai 27B create commercial value? - **Private document assistance:** summarize, classify, or extract from sensitive local files without sending the content to a model API, provided the entire surrounding application also stays local. - **Offline field software:** support technicians, inspectors, or mobile workers in locations with unreliable connectivity. - **On-device product features:** make text assistance available with predictable latency and no per-token network charge after the model is deployed. - **Hybrid agent routing:** keep private and bounded steps local, then route difficult reasoning or fragile tool workflows to a stronger cloud model. - **Single-machine pilots:** evaluate a 27B-class model on laptops or modest GPUs before paying for a permanent inference fleet. The valuable product is still the system around the model. Wavect's [AI enablement service](/services/ai-enablement/) covers use-case selection, evals, data access, routing, deployment, observability, and team handover. Our [Twinsoft AI case study](/case-studies/twinsoft-ai/) shows why production value comes from the workflow and controls around the model, not its parameter count. ## Which Bonsai 27B variant should a company choose? | Workload | Best starting point | Why | Gate before production | | --- | --- | --- | --- | | Phone-local, offline text features | 1-bit MLX | Only variant designed for the phone-class memory budget | Thermals, peak memory, battery, and task quality on the target phone | | Laptop-local private assistant | Ternary if memory allows | Better instruction, tool, and vision retention | Test the 7.17 GB fork file or 7.59 GB mainline file with the expected context | | Long-horizon coding agent | Compare against full precision or a managed frontier model | The release explicitly flags this as a weak target | Repository-level run-test-repair eval with failure costs | | High-volume enterprise serving | Benchmark 1-bit, ternary, and hosted alternatives | Single-stream speed does not predict concurrent throughput or operations cost | Cost per successful task, SLO, redundancy, and fallback | | Multimodal mobile feature | 1-bit plus optional vision component | Smallest language footprint leaves the most memory headroom | Vision quality, camera privacy, and total resident memory | This article intentionally owns the Bonsai-specific deployment decision. For the wider family choice, use our [2026 open-weight LLM comparison](/blog/open-weight-llm-comparison-2026/). For the financial decision between hardware and APIs, use the [local-model versus API break-even calculator](/blog/local-models-vs-apis-break-even-eu-2026/). Keeping those intents separate prevents one product review from pretending to answer the whole model market or your infrastructure economics. ## What should a Bonsai 27B pilot measure? 1. **Freeze the artifact and runtime.** Record the Hugging Face revision, weight hash, backend fork, kernel commit, device, OS, and browser or MLX version. 2. **Measure the whole memory envelope.** Include weights, runtime, expected context, KV-cache format, vision projection, and speculative drafter. 3. **Build a task eval.** Score at least 50 to 100 representative tasks, including formatting constraints, tool schemas, refusal cases, and unacceptable failures. 4. **Separate latency phases.** Capture time to first token, prompt processing, sustained decode, and performance after repeated warm runs. 5. **Test offline and privacy boundaries.** Verify model fetches, analytics, crash reporting, logs, embeddings, and fallback routes. A local model inside a cloud-dependent app is not fully local. 6. **Price successful outcomes.** Compare device support, engineering, update distribution, monitoring, and fallback costs against a hosted API. 7. **Choose a rollback path.** Keep a known-good model and a server-side route for unsupported devices, overload, or quality failures. Our [MVP technology selection guide](/software-development-guide/how-to-choose-a-tech-stack-for-mvp/) applies the same rule: decide first on the constraint that is expensive to reverse. For on-device AI, that constraint is usually the supported hardware floor and the quality of the weakest critical workflow, not the model download size. ## Frequently Asked Questions About Bonsai 27B ### What is Bonsai 27B? Bonsai 27B is PrismML's binary and ternary low-bit model family derived from Qwen3.6-27B. The 1-bit language weights occupy about 3.9 GB. Ternary targets 5.9 GB ideally, while current GGUF files use 7.17 GB for group-128 or 7.59 GB for the mainline-compatible group-64 pack. ### Can Bonsai 27B run on an iPhone? PrismML reports about 11 tokens per second on an iPhone 17 Pro Max using MLX Swift. That result applies to a supported high-end phone and a specific runtime. Model weights, context, runtime buffers, and optional vision components all count toward the app's real memory budget. ### Is Bonsai 27B really a 1-bit model? The binary language weights use one sign bit plus one FP16 scale shared across every 128 weights, for 1.125 effective bits per weight. Some small supporting tensors remain at higher precision, and the optional vision component uses 4-bit HQQ. ### Does Bonsai 27B retain 90% of Qwen3.6 quality? It scores 89.5% of Qwen3.6-27B FP16 on PrismML's own average across 15 thinking-mode benchmarks. A separate 98-question community benchmark scored the 1-bit model at 82.9% versus 94.2% for Qwen3.6-27B. Retention is workload-dependent, so independent task-specific evaluation is still required. ### Is Bonsai 27B free for commercial use? The Hugging Face model cards list Apache 2.0. A commercial team should still pin the exact artifacts, retain notices, review upstream and dependency licenses, and confirm that its intended data and use case meet all applicable obligations. ### Should I use 1-bit or ternary Bonsai 27B? Start with 1-bit when phone-class footprint is the hard constraint. Start with ternary when laptop-class memory is available and task quality matters more. Compare both on your own eval because the published aggregate does not predict the failure cost of your workflow. ## Final thoughts Bonsai 27B is a genuine systems milestone: PrismML has put 27B-class language weights into a 3.9 GB package, demonstrated interactive inference on a high-end phone, and moved binary plus a mainline-specific ternary pack into upstream llama.cpp. That changes which private, offline, and hybrid product designs are technically possible. It does not remove the engineering decision. The 90% claim is a vendor-reported aggregate, a small independent benchmark also shows a meaningful task-level gap, quality loss is largest in several capabilities that agents need, and the full context does not fit inside the phone-sized headline. Treat Bonsai as a strong pilot candidate. Pin the exact artifact and runtime, measure peak memory, test real tasks, and compare cost per successful outcome before making it a production dependency. ## You may also like.. [**When Local Models Beat APIs** Calculate the real break-even across GPU utilization, engineering, eval upkeep, data residency, and hosted alternatives.](/blog/local-models-vs-apis-break-even-eu-2026/) [**AI Enablement vs a generic AI consultancy** A model demo proves feasibility. Production needs evals, routing, deployment, observability, and a team that can own the stack.](/compare/ai-enablement-vs-generic-ai-consultancy/) Models and infrastructure ## Continue through this cluster Model selection, inference economics, local deployment, compression and serving architecture. [Start with the cornerstone**Self-Hosting LLMs in the EU: When Open Weights Actually Pay Off**](/blog/self-hosting-llms-eu-cost/) - [FreeToken AI Review: Run Frontier MoE Models on Consumer GPUs?](/blog/freetoken-ai-inference-engine-review/) - [Darkbloom AI Review: Private Inference on Idle Macs](/blog/darkbloom-ai-private-inference-mac/) - [Ox Alpha Free AI Model: Setup, Privacy and Buyer Guide](/blog/ox-alpha-free-ai-model-guide-2026/) - [Pika Audio Models API Pricing: Is SFX Really 20x Cheaper?](/blog/pika-audio-models-api-pricing-2026/) - [Thunder Compute's $13M GPU Virtualization Bet: Enterprise Buyer's Guide](/blog/thunder-compute-gpu-virtualization-series-a/) Inbox, without the noise ## Follow the work that matters to you Get a short email when we publish something new. Follow the whole blog or only the problems you care about. [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 13 min read · 16 Jul 2026 Last reviewed August 11, 2026 [**Next**](/blog/soofi-s-european-sovereign-llm/) New posts by email × × Get new posts by email A short email when we publish. Free, no tracking. ## Structured Data ```json { "@context": "https://schema.org", "@graph": [ { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ], "employee": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "founder": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "legalRepresentative": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "name": "Wavect GmbH", "subjectOf": { "@id": "https://wavect.io/verified-claims.json#dataset", "@type": "Dataset", "creator": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "description": "A machine-readable registry of quantitative and qualitative claims published by Wavect, with review dates, localized page appearances and public third-party citations where available.", "inLanguage": "en", "isAccessibleForFree": true, "license": "https://creativecommons.org/licenses/by/4.0/", "name": "Wavect verified publication claims", "url": "https://wavect.io/verified-claims.json" }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "sameAs": [ "https://www.wikidata.org/wiki/Q139796367", "https://www.linkedin.com/in/jocr77/", "https://github.com/jo-chris" ], "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/#website", "@type": "WebSite", "inLanguage": [ "en", "de", "es", "zh" ], "name": "Wavect", "potentialAction": { "@type": "SearchAction", "query-input": "required name=search_term_string", "target": { "@type": "EntryPoint", "urlTemplate": "https://wavect.io/search/?q={search_term_string}" } }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/blog/bonsai-27b-phone-local-ai-review/#webpage", "@type": "WebPage", "dateModified": "2026-07-16", "inLanguage": "en", "isPartOf": { "@id": "https://wavect.io/#website", "@type": "WebSite" }, "lastReviewed": "2026-07-16", "url": "https://wavect.io/blog/bonsai-27b-phone-local-ai-review/" } ] } ``` ```json { "@context": "https://schema.org", "@type": "BlogPosting", "abstract": "Bonsai 27B is PrismML's low-bit model family derived from Qwen3.6-27B. Its binary language weights occupy about 3.9 GB, and PrismML reports roughly 11 tokens per second on an iPhone 17 Pro Max. The 1-bit build scores 89.5% of the Qwen3.6 FP16 average in PrismML's suite, but an independent 98-question community benchmark scored it at 82.9% versus 94.2% for Qwen3.6-27B. Binary Q1_0 works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. For ternary, the 7.17 GB group-128 Q2_0 file still needs PrismML's fork, while the 7.59 GB group-64 file now works across those mainline backends. Bonsai remains a strong pilot candidate, but production requires pinned artifacts, full peak-memory measurement, task-specific evals, sustained latency tests, and cost per successful outcome.", "articleBody": " Blog overview/AI and agents/Models and infrastructure Bonsai 27B Review: Can a 27B LLM Really Run on a Phone? TL;DR Bonsai 27B is PrismML's low-bit model family derived from Qwen3.6-27B. Its binary language weights occupy about 3.9 GB, and PrismML reports roughly 11 tokens per second on an iPhone 17 Pro Max. The 1-bit build scores 89.5% of the Qwen3.6 FP16 average in PrismML's suite, but an independent 98-question community benchmark scored it at 82.9% versus 94.2% for Qwen3.6-27B. Binary Q1_0 works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. For ternary, the 7.17 GB group-128 Q2_0 file still needs PrismML's fork, while the 7.59 GB group-64 file now works across those mainline backends. Bonsai remains a strong pilot candidate, but production requires pinned artifacts, full peak-memory measurement, task-specific evals, sustained latency tests, and cost per successful outcome. Yes, a 27B-class model can now run on a phone, but the useful answer needs more context than the headline. PrismML reports that its 1-bit Bonsai 27B language weights occupy about 3.9 GB and generate roughly 11 tokens per second on an iPhone 17 Pro Max. The model is derived from Qwen3.6-27B and published under Apache 2.0. It is a real on-device milestone, not proof that every 27B workload now fits comfortably inside every phone. The same evidence also shows where the compression hurts. Across PrismML's own 15-benchmark suite, the 1-bit variant scores 89.5% of the full-precision average. Math and coding retain more of their baseline than instruction following, vision, and multi-step tool use. The quality-first ternary variant retains 94.6%. Its group-128 GGUF is about 7.17 GB, while the mainline-compatible group-64 file is about 7.59 GB. Both are larger than the 5.9 GB ideal representation often repeated in launch posts. Bonsai 27B in one minute, verified 11 August 2026 QuestionVerified answerWhat it means for a buyer Does it run on a phone?PrismML reports about 11 tok/s on iPhone 17 Pro Max via MLX SwiftInteractive local text generation is possible on a current high-end phone Is it really 3.9 GB?Yes, for the resident 1-bit language model weightsRuntime buffers, context cache, and optional vision components need additional memory Does it retain 90%?89.5% of the FP16 average in PrismML's 15-benchmark evaluationUse-case quality still needs an independent eval because losses are uneven Is ternary 5.9 GB?5.9 GB ideal; 7.17 GB for the group-128 fork file or 7.59 GB for the group-64 mainline filePlan capacity from the exact artifact, not the information-theoretic target Is it production-ready?Promising, with custom low-bit runtimes and explicit model-card limitationsPilot first, especially for agents, vision, and long context What is Bonsai 27B? Bonsai 27B is PrismML's low-bit version of Qwen3.6-27B, a dense multimodal model with a 27B language backbone, a vision encoder, and a native 262,144-token context window. Qwen's architecture uses linear attention in most blocks and full attention in 16 of 64 blocks. That hybrid design limits KV-cache growth compared with a conventional all-attention model. PrismML publishes two operating points. The 1-bit build uses binary weights from {−1, +1} with one FP16 scale for each group of 128 weights. That produces 1.125 effective bits per weight and about 3.9 GB of resident language weights. The ternary build adds zero, using {−1, 0, +1}. Its representation carries 1.71 effective bits per weight and retains more quality. This is deeper than taking a normal FP16 checkpoint and exporting a generic 1-bit file. PrismML says low-bit weights extend across embeddings, attention projections, MLP projections, and the language-model head. Custom CUDA, Metal, MLX, and WebGPU kernels consume the packed weights directly instead of expanding them back to FP16 for every operation. Deployment support has also matured since launch. PrismML's current compatibility matrix says binary Q1_0 works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. Ternary needs an artifact-level distinction: the 7.17 GB group-128 *-Q2_0.gguf still requires PrismML's fork, while the 7.59 GB group-64 *-Q2_0_g64.gguf now runs in mainline on CPU, Metal, CUDA, and Vulkan. The experimental PQ2_0 files are not yet a stable deployment target. Buyers should pin the model file and runtime commit together. How does a 27B model shrink from 54 GB to 3.9 GB? The rough FP16 calculation is simple: 27.3 billion language parameters multiplied by two bytes is about 54 GB before runtime overhead. The binary model stores one sign bit per weight plus a 16-bit scale shared across every 128 weights. That works out to 1.125 bits per weight, roughly 14.2 times smaller than FP16. The headline footprint is not the entire application footprint. PrismML's model card separates four memory layers: Resident language weights: about 3.9 GB for 1-bit, 7.17 GB for the ternary group-128 fork file, or 7.59 GB for the group-64 mainline file. Runtime and activations: the", "articleSection": "Engineering", "author": { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/" }, "citation": [ { "@type": "WebPage", "name": "Bonsai 27B", "url": "https://prismml.com/news/bonsai-27b" }, { "@type": "WebPage", "name": "Qwen3.6-27B", "url": "https://huggingface.co/Qwen/Qwen3.6-27B" }, { "@type": "WebPage", "name": "1-bit build", "url": "https://huggingface.co/prism-ml/Bonsai-27B-gguf" }, { "@type": "WebPage", "name": "ternary build", "url": "https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf" }, { "@type": "WebPage", "name": "PrismML's current compatibility matrix", "url": "https://github.com/PrismML-Eng/Bonsai-demo" }, { "@type": "WebPage", "name": "ArmanJR's reproducible 98-question benchmark", "url": "https://github.com/ArmanJR/PrismML-Bonsai-vs-Qwen3.5-Benchmark" }, { "@type": "WebPage", "name": "Joshua Lochner's Hugging Face Space", "url": "https://huggingface.co/spaces/webml-community/bonsai-webgpu-kernels" } ], "dateModified": "2026-08-11", "datePublished": "2026-07-16", "description": "Bonsai 27B is PrismML's low-bit model family derived from Qwen3.6-27B. Its binary language weights occupy about 3.9 GB, and PrismML reports roughly 11 tokens per second on an iPhone 17 Pro Max. The 1-bit build scores 89.5% of the Qwen3.6 FP16 average in PrismML's suite, but an independent 98-question community benchmark scored it at 82.9% versus 94.2% for Qwen3.6-27B. Binary Q1_0 works in mainline llama.cpp on CPU, Metal, CUDA, and Vulkan. For ternary, the 7.17 GB group-128 Q2_0 file still needs PrismML's fork, while the 7.59 GB group-64 file now works across those mainline backends. Bonsai remains a strong pilot candidate, but production requires pinned artifacts, full peak-memory measurement, task-specific evals, sustained latency tests, and cost per successful outcome.", "headline": "Bonsai 27B Review: Can a 27B LLM Really Run on a Phone?", "image": "https://wavect.io/img/blog/headers/header_bonsai-27b-phone-local-ai-review.svg", "inLanguage": "en", "mainEntityOfPage": { "@id": "https://wavect.io/blog/bonsai-27b-phone-local-ai-review/", "@type": "WebPage" }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/blog/bonsai-27b-phone-local-ai-review/", "wordCount": 2627 } ``` ```json { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "item": "https://wavect.io/", "name": "Home", "position": 1 }, { "@type": "ListItem", "item": "https://wavect.io/blog/overview/", "name": "Blog overview", "position": 2 }, { "@type": "ListItem", "item": "https://wavect.io/blog/topics/ai-agents/", "name": "AI and agents", "position": 3 }, { "@type": "ListItem", "item": "https://wavect.io/blog/clusters/models-infrastructure/", "name": "Models and infrastructure", "position": 4 }, { "@type": "ListItem", "item": "https://wavect.io/blog/bonsai-27b-phone-local-ai-review/", "name": "Bonsai 27B Review: 1-Bit LLM on a Phone | ", "position": 5 } ] } ``` ```json { "@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [ { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Bonsai 27B is PrismML's binary and ternary low-bit model family derived from Qwen3.6-27B. The 1-bit language weights occupy about 3.9 GB. Ternary targets 5.9 GB ideally, while current GGUF files use 7.17 GB for group-128 or 7.59 GB for the mainline-compatible group-64 pack." }, "name": "What is Bonsai 27B?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "PrismML reports about 11 tokens per second on an iPhone 17 Pro Max using MLX Swift. That result applies to a supported high-end phone and a specific runtime. Model weights, context, runtime buffers, and optional vision components all count toward the app's real memory budget." }, "name": "Can Bonsai 27B run on an iPhone?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "The binary language weights use one sign bit plus one FP16 scale shared across every 128 weights, for 1.125 effective bits per weight. Some small supporting tensors remain at higher precision, and the optional vision component uses 4-bit HQQ." }, "name": "Is Bonsai 27B really a 1-bit model?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "It scores 89.5% of Qwen3.6-27B FP16 on PrismML's own average across 15 thinking-mode benchmarks. A separate 98-question community benchmark scored the 1-bit model at 82.9% versus 94.2% for Qwen3.6-27B. Retention is workload-dependent, so independent task-specific evaluation is still required." }, "name": "Does Bonsai 27B retain 90% of Qwen3.6 quality?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "The Hugging Face model cards list Apache 2.0. A commercial team should still pin the exact artifacts, retain notices, review upstream and dependency licenses, and confirm that its intended data and use case meet all applicable obligations." }, "name": "Is Bonsai 27B free for commercial use?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Start with 1-bit when phone-class footprint is the hard constraint. Start with ternary when laptop-class memory is available and task quality matters more. Compare both on your own eval because the published aggregate does not predict the failure cost of your workflow." }, "name": "Should I use 1-bit or ternary Bonsai 27B?" } ] } ```