Bonsai 27B's 1-bit quantization: the reality of phone-class local inference

What we could verify

On July 14, 2026, PrismML announced Bonsai 27B, a multimodal model based on Qwen3.6 27B. It offers a 1-bit Binary Bonsai and a ternary Bonsai using -1, 0, and +1 weights. The official announcement describes their weight footprints as approximately 3.9GB and 5.9GB respectively.

The official figures are 1.125 effective bits per weight for the 1-bit version and 1.71 effective bits per weight for the ternary version. PrismML describes the low-bit representation as covering the whole language network—embeddings, attention, MLPs, and the LM head—without higher-precision escape hatches. The model supports a 262K-token context and image input, and is released under the Apache 2.0 license.

Those 3.9GB and 5.9GB figures are primarily model-weight footprints. Context length, KV cache, runtime, image components, and the operating system require additional memory, so “runs with only 3.9GB of VRAM” does not follow. The official material does not establish one universal VRAM requirement for every computer.

The official GGUF model card identifies the 1-bit build as Q1_0_g128 and documents CUDA or Metal use through the PrismML llama.cpp fork. The official demo repository also lists build paths for Windows, Linux, macOS, and iOS, but actual support still needs to be checked for each model-format and runtime combination. The official material does not position Bonsai 27B as an NSFW-specific or uncensored model.

In the model card’s benchmark table, PrismML reports a 15-task average of 76.11 for 1-bit Bonsai 27B versus 85.07 for its FP16 baseline; Agentic / tool calling is 66.03 versus 80.00. These are PrismML’s reported evaluations, not a direct guarantee of quality in a creative workflow. The model card also says that long-horizon, multi-file agentic coding with run-test-and-repair loops is not a strong target for this release.

What users reported in the community

In r/LocalLLM, a user ran the Q1 build on a Xiaomi 14T Pro and reported that the fact it ran on a smartphone was impressive, while the speed was mediocre. A comment on the same post added a separate production experience: interactive use depended more on time to first token, KV-cache reuse, and sustained thermals than on peak decode speed. These are reports from particular devices, not evidence about smartphones in general.

In a separate r/LocalLLaMA post, a user benchmarked the ternary build on an 8GB RTX 5070 Laptop. They reported that it fit on the GPU, but that its Terminal-Bench 2.0 score was below a 9B model that also fit on the same card. The same author reported clean tool-call syntax but an agent loop in the 1-bit build that failed to stop and expanded into a very long completion. This is a user report from one harness, configuration, and test condition—not an official benchmark.

Another r/LocalLLM post reported that fused Gated Delta Net operations were disabled on a Vulkan run and became enabled after switching to CUDA. The author attributed generation loops to that difference, but this is a user’s diagnosis. Because the model card documents the PrismML llama.cpp fork, the runtime and build that implement the model’s hybrid structure need to be checked alongside sampling parameters.

A 4060 Ti 16GB user reported that the ternary build was usable for small tasks and document processing, while instruction following and reliability lagged behind another 35B-class model. The same report gave a concrete example of using it for a tool-assisted summarization task. Across these posts, the recurring pattern is that shrinking the weights and making long agentic or tool-driven work reliable are separate questions.

Implications for llimu’s creative and local workflow

Bonsai 27B is a candidate for organizing unpublished setting material or adult-project drafts locally, without sending them to an external API, because a 27B-class model can be stored with a small weight footprint. The value is not “27B means high quality”; it is that more local experiments become possible within a given memory budget.

A careful rollout should separate these checks:

1. Fix the binary or ternary variant, quantization format, context length, and KV-cache precision, then measure actual memory use and long-form consistency
2. Specify the runtime and GPU backend, including the PrismML fork, and do not treat Vulkan, CUDA, and Metal as equivalent configurations
3. Evaluate continuation, setting-table cleanup, image description, and small tool-assisted tasks separately; do not delegate long agent loops without verification
4. Check the Apache 2.0 terms, and do not infer uncensored behavior or adult suitability from the model name
5. Even with local inference, review age, consent, prohibited content, and continuity in generated material; add a separate permission boundary before automating publication or file operations

This story shows that quantization is no longer only about storing a large model more compactly. The runtime and the evaluation method are part of the choice. For creative work, the practical standard is not the headline that it runs on a phone, but a small private evaluation set that measures speed, consistency, and whether tool-driven generation stops correctly.